Exploiting producer-consumer pattern to achieve intra-process data parallelism.
<< Previous Article (Rendezvous with JSON) |
We have a long way to go, thus, in order to quickly set the stage, we consider following psuedo code we often encounter:
List<T> data = CreateData(...)
//where T is some known datatype,
// CreateData is some function which returns a collection of instances of T
ProcessList(data)
//where ProcessList performs required processing on the generated data
As per above example, we declare, "CreateData" as our Data-Producer (or simply Producer) and "ProcessList" as our Data-Consumer (or simply Consumer). Then we consider following description of our goal:
Our Goal:Given two (2) pieces of codes, one generates data (thus, calling producer) and another performs data-procesing (thus, calling consumer) and assuming that each data instance can be processed independently; we are interested in designing a generic mechanism that leverage data-parallelism, enables concurrent data-processing, and at the same time hides associated thread-synchronization intricacies and offers a simplified API.
Now after the goal is announced, we need to decide on the approach and for that when we look into the literature, we take inspiration from long known concept of Producer-consumer (one can find information of its generic case in this Wikipedia article). We would like to adapt this concept, to distribute workload (recieved from producers) among consumers while letting all the entities (all producers/consumers) running CONCURRENT
ly. During our discussion, we will go through, step-by-step, a possible implementation of it, i.e. to have our producer(s)
running independently, yet in harmony, of our consumer(s)
. We will see how to create such a concert (here onward, calling it Pipeline
) while satisfying following design requirements:
datatype
) Vs discordant (produced items requires some sort of possible data transformation to match consumable item's datatype
)compile-time
and bound to the chain) Vs detached (producers, perhaps ephemeral, appear, possibly in parallel, at run-time) : Implementation of these two cases we will discuss separatelyAlthough the original producer-consumer problem considered only the CONCORDANT case (third (3rd) point among above characteristics). Nevertheless, thanks to GoF's famous Adapter design pattern
(see GoF design patterns), we would like to extend the idea, while stretching the original philosophy of the design pattern, in order to create a data oriented adapter (here onwards simply calling it adapter unless specified otherwise), in order to create such a pipeline among given discordant producer/consumers (given a condition we identify an adapter). The point of interest in doing so is to maintain separation of concerns, and, thus, to achieve simplified pipeline with detached data transformation logic.
Why ".Pipe"
? To understand it, for a moment, imagine the date-flow and think of a simplistic view of data originating at producer and absorbed by consumer. Think of, as if there are two person (P and C) and P is handing over whatever he got to C. With this, I think of UNIX.
As, with UNIX
terminal when we want to create such a chain of actions (passing data between commands); it exactly lets us do that, thanks to famous "|" (a.k.a. pipe)
syntax (see some examples here), and, so I thought of this name.
When we talk about why we need such an implementation, we need to consider several factors:
Now, talking about our pipeline, lets consider a task at hand (perhaps trivial):
Assume, we need to parse a file, lets say CSV for simplicity sake, that contains some considerable number of records (i.e. rows). Further assume that we need to store these records to some database; without any additional computation on the data.
Here we observe two (2) distinct and uncorrelated sub-tasks: read the file (producer) & save data in database (consumer). Now, consider two(2) classic (non-concurrent) implementation approaches:
Most importantly, both approaches ignore the fact that both, file and database operations, are I/O operations, and, given even a single core processor, concurrency can be achieved through thread-interleaving thanks to non-blocking I/O. It is also possible to design yet another balance approach where instead of pushing single record to db, we will push some fixed-size (chunk) lists to database. However, as we describe about our pipeline approach next (below), one can see that it remains less attractive in term of performance.
Assuming that we have our .Pipe
implementation available. We can design a producer method (reading the file) and consumer method (making db transaction), we can simple write above code as: producer.Pipe(consumers)
(lets call it Approach3)
.Pipe
will facilitate the channeling of chunks (lists) from producer to consumers => assuming this data transfer time is negligible and buffer is unboundedWith such an approach we make following significant observations:
In general, as total number of records increases, we would notice (left-most is lowest and right-most is highest): (lower the better)
Thus, perhaps, it might be safe to say that our concurrent pipeline approach is a balanced approach. Thus, the why.
However, before we discuss implementation, we need to consider/make following limitations/assumptions:
broadcasting
(see here). In our approach, each produced item will be consumed (accepted/treated/processed) by one and only one consumer among all available consumers.The idea of producer-consumer is actually language neutral and can be developed in several programming languages. However, to achieve our goal, we opt to implement it in C# .Net Framework 4.6.1 while leveraging several TPL features (especially async-await
) and inherent language capability to create extension methods. If you are interested to consume this implementation. Based on your choice of language you may achieve different usage forms.
During our discussion, we have provided a lot of comments along with C#.Net code snippets and added some amusing images (showing conversation among entities). Even if you feel uncomfortable with .Net syntax, do NOT be worried, you would be able to get the essential while reading this article.
During my college days, I always asked myself, every time I took the operating system book in my hand, why the dinosaurs? (unfortunately, I cannot find the original cover but this picture should do for the moment) And, I used to cajole myself that the book is not as terrifying as Jurassic Park of Steven Spielberg. I still wonder, sometimes, was it to symbolize operating system as gigantic/fascinating/stupefying as dinosaur or was it just to overwhelm a sophomore. Nonetheless, it is NEITHER the right time NOR the subject of our discussion, thus, whatever the case, during this discussion there are no dinosaurs and we will try our best to keep things simple.
To begin with, lets have a look at following simple picture to understand a few of our design choices and more importantly what we are actually trying to build:
So based on above picture, we want:
In order to design our solution, we would like to focus on the buffer
as it is going to be the central piece of our solution; and, its implementation is going to be impacted by the producer side requirements as well as of consumer, plus we should not forget that we need to diffuse all the features to our design. Thus, in order NOT to complicate the discussion with everything explained in a single silo based proposed solution, we further sub-divide the discussion into several smaller pieces as follow:
As our solution is producer agnostic, i.e. we do not know how exactly the producer would produce an item (i.e. the actual producer implementation). In this case, we can only define a generic signature of it and thus our producer can be defined as simple as following delegate
:
//NOTE: Some explanation are provided as comments
public delegate Task ProduceAsync<TP>(IProducerBuffer<TP> buffer, CancellationToken token);
//accepts buffer and cancellation token as inputs and returns a Task
// where TP is the datatype of item produced by producer
// and IProducerBuffer is an interface to our Buffer implementation
//we add CancellationToken as an input parameter in order to support
// interruptible pipeline feature
//In this way, by simply supplying CancellationToken.None to the pipeline
// we can create uninterruptible pipeline.
Based on above delegate signature, we can create following interface for our producer:
//IDisposable to avail Dispose method to perform resource clean-up
public interface IProducer<TP> : IDisposable
{
//to perform some pre-processing initialization
Task InitAsync();
//actual data generating method
Task ProduceAsync(IProducerBuffer<TP> buffer, CancellationToken token);
}
Now actual producer implementation can simply inherit IProducer<TP>
interface. Though, we have designed how to provide buffer access to our producer, however, we haven't yet know how to populate the buffer. Thus, our first requirement at buffer side, i.e. to have some method for population. Lets look at it:
//NOTE: Some explanation are provided as comments
public interface IProducerBuffer<T>
{
//adds an item to the buffer
//it blocks, if buffer is full, until the item can be added
void Add(T item, CancellationToken token);
//adds an item to the buffer with given millisecond timeout
//if the item was added with in timeout period returns true else false
bool TryAdd(T item, int millisecTimeout, CancellationToken token)
//we add this second method to support our lossy pipeline feature
// millisecTimeout=0 means immediately add or discard
//based on boolean outcome, the actual producer implementation can
// decide the fate of produced yet discarded item
//we also add CancellationToken to support cancellation based on
//transient method token (we will see an example when we talk about
// detached pipeline)
}
Thus, till now, we have producer and it's buffer interfaces, and so, the means to add produced items to buffer. Now, lets look at the consumer side requirements next.
Similar to producer, our solution is also consumer agnostic (i.e. unaware of the the actual consumer implementation), thus, in a similar way we can define following consumer interface:
//IDisposable to avail Dispose method to perform resource clean-up
public interface IConsumer<TC> : IDisposable
{
//to perform some pre-processing initialization
Task InitAsync();
//actual data consuming method
Task ConsumeAsync(TC item, CancellationToken token);
}
while deciding about the consumer interface, especially the signature of ConsumeAsync
, we had a choice to pass the buffer as method parameter as we did for producer. However, doing so we noticed such design:
thus, finally we decided to hide such complexity within the API and obtained a callable consumer. In such a way, the concrete consumer implementation shall focus on the business logic.
Though, no apparent requirement visible at consumer side, yet we can infer from point (2) above, that we need to loop over the items in order to drain the buffer. Thus, we need:
So, we create our ConsumerBuffer interface:
public interface IConsumerBuffer<T>
{
//true when all items are drained (producers are done producing too!)
bool Finished { get; }
//to retrieve an item
//returns true when item was available within given millisecond timeout
bool TryGet(int millisecTimeout, CancellationToken token, out T data);
//we add millisecond timout to support a special case of our discordant pipeline feature
// anyway we can always pass millisecTimeout=Timeout.Infinite i.e. wait infinitely
}
Until this point, we are trying to fulfill all the requirements, and following item list quickly covers those points:
Ctor
parameter.TryAdd
method. (Note: Add
method is similar to TryAdd(item, Timeout.Infinite)
)CancellationToken
Now, the only remaining point is Concordance. In fact, the way we have defined our interfaces above, we have intentionally kept TP
as producer type parameter and TC
as consumer type parameter. Although, such different symbols (type placeholders) hardly matters in generics
, nonetheless, it is to impose the idea that we will inject IDENTICAL <data-type>
for both TP
and TC
during concordant pipeline construction and different <data-types>
for discordant pipeline. Furthermore, for a rapid understanding of such a conflict, we offer following illustration:
Now, we see that:
datatype <TP>
is same as buffer's datatype <T>
datatype <TC>
is same as buffer's datatype <T>
<TP> = <TC> = <T>
our current pipeline can workThus, above design will not work in case of discordant pipeline. With this idea in mind, we keep both shards of our *Buffer interface
, for the moment. Even from the point of view of "abstraction
", we would be wise to NOT to expose TryGet
method to producer whose only interest is to fill the buffer.
To fulfill our last requirement, we need to review our Image 2 as shown above; as having different datatype
s will create conflict. But, before we talk about how to overcome this limitation using adapter, let's visualize what adapter must do logically based on below picture:
Thus, if we consider provided adapter as a black box, we expect that by passing an object of type <TI>
it will output an object of type <TO>
. As per our requirements, thus, if we pass produced items of type <TP>
and convert those into consumer's type <TC>
; our pipeline should work.
IMPORTANT: In order to remain generic, for the concordant case, when TC = TP = T
(thus, TI = TO
), we prepare a default IDENTITY adapter that does NOTHING i.e. it returns us back the same item that we provided as input without making any change in it. Following C# .Net code snippet roughly represents this idea:
public static TI IdentityAdapter<TI>(TI input)
{
return input;
}
In order to plug such an adapter, we have following choices:
<TC>
(shown in image 4)<TP>
and second has type <TC>
(shown in image 5)<TP>
(shown in image 6)Among given implementation choices, we opt to choose the third (3rd) option, i.e. injecting adapter between buffer and consumer, because:
By injecting adapter between buffer and consumer, we only need to maintain a single buffer (thus, single drainage loop), but also, we make a transperant call to adapter just at right time (before feeding data to consumer). And, by doing so we hide all these intricated implementation details behind our .Pipe
call and offer complete seperation of concerns among producer, consumer and adapter so that all these three (3) pieces of code can evolve independently.
Up until now, we kept our design both producer and consumer agnostic, however, in order to keep the complexity out of our discussion we assumed a naive approach to the Adapter. As shown above, we provided an object instance, of some given type, to our adapter and receive back an object instance of a well defined type. However, as we are close to finalize our interface design, we would like to get rid of this give-n-take assumption about our adapter. In fact, we desire to finalize the design as Adapter agnostic too! And, this is the only way we are sure that we have provided full liberty to the end-user to achieve the desired end result from such pipeline without hacking/patching business logic. End-user can then focus on actual logic and associated data-model without worrying about mundane technical plumbing between producer, consumer, and adapter.
To achieve such Adapter agnostic design, we propose following interfacing:
//NOTE: Some explanation are provided as comments
public interface IDataAdapter<TP, TC>
{
//accept the buffer and cancellation token and outs consumable object
// returns true until buffer is NOT empty! else false.
bool TryGet(IConsumerBuffer<TP> buffer, CancellationToken token, out TC consumable);
//we notice that we provide buffer containing produced object instances
// with IConsumerBuffer interface, thus exposing TryGet method!
//Actual adapter implementation can then recover produced item (or several items)
// to construct an item of type TC
}
Now as we have defined all the three (3) key parts of the pipeline, given any task and assuming that our pipeline can be implemented, we can achieve an optimal solution by thinking in terms of these sub-components as shown below:
Following itemized list summarizes the above idea:
Until this point, we tried to use a lot of drawings to convey our ideas, but, unfortunately, now we are obliged to introduce the code, and, thus, below you will see some long code snippets. But do NOT be worried, we will add some amusing drawings to illustrate the same idea in psuedo manner; nonetheless, you must memorize below given pyramidical mind-map which is closely related to our concrete implementation:
As we are aware that our solution is producer/consumer/adapter agnostic, thus, their respective concrete implementation is not our concern; once we have exposed our interfaces, end-user will inherit those to use in the pipeline. However, it would be nice to implement some default Adapters in order to cover some mundane use cases. Thus, in this section we will propose following implementations:
To start simple, we choose to implement identity adapter first and if we remember from above, it should just return the produced item as it is. We achieve this as follows:
//NOTE: Some explanation are provided as comments
//generic adapter satisfying TP = TC = T, buffer type:<T>
public class IdentityAdapter<T> : IDataAdapter<T, T>
{
public bool TryGet(IConsumerBuffer<T> buffer, CancellationToken token, out T consumable)
{
//we just transfer the call on the buffer and return boolean
// status and also the object as it.
return buffer.TryGet(Timeout.Infinite, token, out consumable);
//NOTE: we pass INFINITE timeout on buffer, thus:
//if all buffered items are processed AND producers are done...
// buffer will return false. Thus satisfying adapter boolean status.
//else buffer will return True and out an instance of the produced object
// this again fulfils adapter behaviour.
}
}
Sometimes we encounter a case when consuming single item leads to a suboptimal solution; and processing those in group (chunks) is technically cost-effective. A few of such examples are:
In order to handle such use cases, we have decided to implement awaitable list adapter, so that end-user is relieved and use it out of the box. The idea is to recover List<TC>
on each TryGet
call on the adapter as shown below in image 7.
NOTE: Now onwards, we will use words "chunk" and "list" interchangeably, i.e., unless and until specified otherwise, list would mean a subset (a part of) of whole data.
As soon as we think about list, following design related options comes to mind related to TryGet
method:
For the first (1st) option, given the fact that it might NOT be possible to generate identically sized list (consider if we have total of 103 items and we fixed the list size to be 10, then the last list will contain ONLY 3 items instead of 10); yet, we choose to implement it based on belief that consumer logic is indifferent to the size of the chunk (and it should!) and the whole idea behind consuming chunks (instead of single instance) is to reduce associated technical latency.
The second (2nd) option is a generalized case of the first option, so we will implemented it but with some assumptions. These assumptions we will underline when we describe our implementation details.
We choose to opt out the third (3rd) option because it again questions the usefulness of spanning multiple consumers. Lets rethink that if we are able to supply unbounded lists to consumers then perhaps we are able to supply available items to a single consumer alone irrespective of the fact whether consumer has a capacity to handle such a list or not; then why to span other consumers concurrently? Thus, we observe that our design is going astray (based on our pre-decided goal).
NOTE: Perhaps due to our myopic vision, we dropped implementing third (3rd) option. Still, not to forget that our pipeline is Adapter agnostic, thus, end-user can always construct their own version of Adapter and plug it in.
The short answer is: No. it isn't!
If you have followed us till now, perhaps you might got an impression that this adapter is all about creating a list, then why we call it "Awaitable" list adapter? Ain't it as simple as spanning a loop to producea list? If you have got similar thoughts, then we assure you that its more than that; for the simplest fact that items we want to iterate might not be promptly awailable. In fact, to elaborate further let's consider following below listed arguments:
TryGet
method was called, the buffer was empty and producers were busy creating object instance, thus, soon there will be some items in the buffer but for the moment we need to wait (sleep)TryGet
call, we do NOT escape from this conundrum. And, all above listed questions fall back at caller
level (i.e. the code which called TryGet
at the first place).One thing is certain that if we want to reduce on latency (as a part of our goal) we need to have a some kind of notification when the items arrives in the buffer, while our thread is asleep. Similar suggestion can also be found in the original producer-consumer problem. Now, of course we do not want to build such a mechanism inside the adapter else it would fail the whole purpose (imagine, everytime end-user/we write an adapter we need to write a separate notification mechanism). Nonetheless, if we look into the literature of producer-consumer, we already know that producer is capable of providing such a signal (at the time of adding item in the buffer). Thus, considering both the perspectives, for the moment we assume that buffer is capable of such notification.
Based on above discussion, we got following insights on buffer behaviour (we would use it during buffer's TryGet
implementation):
out
the element (with true
as boolean return value)out
an element (and true
as boolean return value)TryGet
call would result in false
boolean return value (out
as null/default of type).For the moment, we can safely assume that if we pass INFINITE timeout to buffer.TryGet
method, then buffer we return us an item as soon as it gets added. This resolves one of our concerns, but, we still need to work on both fixed-size list and variable-size list preparation.
While implementing Awaitable List Adapter, we keep following important points in mind:
buffer.TryGet
ends up waiting for an item. Thus, one is INDIFFERENT to time taken to prepare such a list.1 <= last_chunk_length <= length_of_full_size_list
).1 <= chunk_size <= max_length
)buffer.TryGet
ends up waiting for the first (1st) itemNOTE: We discuss some possible use-cases of these adapters separately (below) in the article.
Based on our constraints/assumptions, we have two (2) parameters to deal with: 1) List size and 2) Timeout period. And we already know if timeout=Infinite
, then we are outputting fixed-size list else variable-sized list. Let's look at the code then:
//NOTE: Some explanation are provided as comments
//generic adapter satisfying TP = T and TC = List<T>, buffer type:<T>
public class AwaitableListAdapter<T> : IDataAdapter<T, List<T>>
{
private readonly int _millisecTimeout;
private readonly int _maxListSize;
//we ask user for timeout value and list size
//if millisecTimeout = Timeout.Infinite
// we will construct fixed size chunks with size = maxListSize
//otherwise,
// we will contruct variable sized chunks with max. size = maxListSize
public AwaitableListAdapter(int maxListSize, int millisecTimeout)
{
_millisecTimeout = millisecTimeout;
_maxListSize = maxListSize;
}
public bool TryGet(IConsumerBuffer<T> buffer, CancellationToken token,
out List<T> consumable)
{
consumable = default(List<T>);
//NOTE: From our discussion, no user wants to consume 0-length list
// so we wait with INFINITE timeout, i.e.
// if item is already in buffer, we promptly receive it
// else we wait until an item is available... so we are good!
if (!buffer.TryGet(Timeout.Infinite, token, out var value)) return false;
//init list WITH first item
consumable = new List<T>(_maxListSize) {value};
//we choose what kind of list is required based on timeout.
return _millisecTimeout == Timeout.Infinite
? TryFillFixedSize(buffer, token, consumable)
: TryFillFixedDurationChunk(buffer, token, consumable);
}
private bool TryFillFixedSize(IConsumerBuffer<T> buffer, CancellationToken token,
List<T> consumable)
{
//We loop until we fill the list
while (consumable.Count < _maxListSize)
{
//we always wait for INFINITE time to be sure of having an item
// except
// we are left with no item and production is over!
if (buffer.TryGet(Timeout.Infinite, token, out var value))
{
consumable.Add(value);
}
else return true;
}
return true;
//our list already has at least 1 item, so we return TRUE!
}
private bool TryFillFixedDurationChunk(IConsumerBuffer<T> buffer, CancellationToken token,
List<T> consumable)
{
var timeRemains = _millisecTimeout;
var sw = Stopwatch.StartNew();
//using stopwatch we can measure elapsed time
while (consumable.Count < _maxListSize)
{
//and we loop until
// 1. chunk is not full
// 2. we receive item with-in remaining time
if (buffer.TryGet(timeRemains, token, out var value))
{
consumable.Add(value);
if (timeRemains != 0)
{
//IMPORTANT:
//we put a lower limit to zero coz:
// 1. of course, we can't wait with -ve time
// 2. but we want to keep looping even if given timeout has over
// and we can still recover items from buffer
// indeed, with timeout=0, we either promptly receive an item
// or buffer returns FALSE.
// this way we can always be able to provide larger chunk
// when possible
// hence the IF does NOT has ELSE with break/return
// but the OUTER IF does has!
timeRemains = (int) Math.Max(0, _millisecTimeout - sw.ElapsedMilliseconds);
}
}
else return true;
}
return true;
//our list already has at least 1 item, so we return TRUE!
}
}
At this point, we already have buffer interface and behavior based implementation requirements. Using below code written snippet we achieve these requirements:
//NOTE: Some explanation are provided as comments
//we implement both interface
public class PpcBuffer<T> : IProducerBuffer<T>, IConsumerBuffer<T>
{
private readonly CancellationToken _token;
private BlockingCollection<T> _collection;
public PpcBuffer(int bufferSize, CancellationToken token)
{
//we say 0 represents unbounded buffer
_collection = bufferSize.Equals(0) ? new BlockingCollection<T>()
: new BlockingCollection<T>(bufferSize);
_token = token;
}
//IProducerBuffer<T> IMPLEMENTATION >>>>>>>>>
public void Add(T item, CancellationToken token)
{
//Add should wait even if buffer is FULL, so we
//simply call TryAdd with INFINITE timeout
TryAdd(item, Timeout.Infinite, token);
}
public bool TryAdd(T item, int millisecTimeout, CancellationToken token)
{
//either blocking collection will add it with in timeout
// or return false... so our requirement is satisfied
//when timeout is INFINITE this method would either
// finish with item being added or in an exception
// 1. when either of cancellation token is canceled
// 2. buffer is closed
// so again we satisfy our requirements.
using (var mergeToken = CancellationTokenSource.CreateLinkedTokenSource(token, _token))
{
return _collection.TryAdd(item, millisecTimeout, mergeToken.Token);
}
}
//IConsumerBuffer<T> IMPLEMENTATION >>>>>>>>>
public bool TryGet(int millisecTimeout, CancellationToken token, out T data)
{
//we do not create merge token, as user should be able to
//extract queued items once pipeline is closed for addition.
return _collection.TryTake(out data, millisecTimeout, token);
}
//shows together both... closed for adding and empty... so we are good.
public bool Finished => _collection.IsCompleted;
//we implement CloseForAdding method to support implementation of
// detached mode >>>>>>>
public void CloseForAdding()
{
_collection.CompleteAdding();
}
}
With such an implementation we are able to cover all the requirements as discussed above. Now, all that remains is the plumbing of these individual artifacts. And, so we do separately for both attached and detached pipeline below.
As we have discussed, attached pipeline mode has following characteristics:
As our interest is to create the form producers.Pipe(consumers)
, we first need to device a raw implementation as fabricating final form would be just a matter of creating an extension method. We will create this method separately. Approach of our raw implementation would revolve around following idea:
NOTE: We have used one of home-made extension methods to span and await on tasks (for both producers/consumers) :
static Task WhenAll(this Func<int, CancellationToken, Task> func, int repeatCount, CancellationToken token = default(CancellationToken))
Following approach implements all above listed steps:
//NOTE: Some explanation are provided as comments
//We hide the implementation inside INTERNAL class to
//expose it through extension method
internal static class PipeImpl<TP, TC>
{
public static Task Execute(CancellationToken token,
int bufferSize,
IDataAdapter<TP, TC> adapter,
IReadOnlyList<IProducer<TP>> producers,
IReadOnlyList<IConsumer<TC>> consumers)
{
//instead of using await, we decided to create a new Task
//so that caller func can await as per its convenience
return Task.Run(async () =>
{
using (var localCts = new CancellationTokenSource())
{
using (var combinedCts = CancellationTokenSource
.CreateLinkedTokenSource(token, localCts.Token))
{
//creating buffer as per required size
using (var ppcBuffer = new PpcBuffer<TP>(bufferSize,
combinedCts.Token))
{
//span consumers
var rc = RunConsumers(consumers, ppcBuffer, adapter,
combinedCts.Token, localCts);
//span producers
var rp = RunProducers(producers, ppcBuffer,
combinedCts.Token, localCts);
//wait until all consumers and producers finish
await Task.WhenAll(rc, rp).ConfigureAwait(false);
}
}
}
});
}
internal static Task RunConsumers(IReadOnlyList<IConsumer<TC>> consumers,
IConsumerBuffer<TP> feed, IDataAdapter<TP, TC> adapter,
CancellationToken token, CancellationTokenSource tokenSrc)
{
//following line span all consumers (RunConsumer method) in the list
//as separate task
return new Func<int, CancellationToken, Task>(async (i, t) =>
await RunConsumer(consumers[i], feed, adapter, t, tokenSrc)
.ConfigureAwait(false))
.WhenAll(consumers.Count, token);
//our home-made WHENALL line waits on all created tasks
// (i.e. waits on all consumer to finish)
}
private static async Task RunConsumer(IConsumer<TC> parallelConsumer,
IConsumerBuffer<TP> feed, IDataAdapter<TP, TC> adapter,
CancellationToken token, CancellationTokenSource tokenSrc)
{
try
{
//this would dispose the consumer once we have nothing left
//to consume
using (parallelConsumer)
{
//init consumers
await parallelConsumer.InitAsync().ConfigureAwait(false);
token.ThrowIfCancellationRequested();
//we loop until adapter is capable to create a consumable
// instance
while (adapter.TryGet(feed, token, out var consumable))
{
//we feed the item to consumer and wait before
// supplying another item.
await parallelConsumer.ConsumeAsync(consumable, token)
.ConfigureAwait(false);
}
}
}
catch
{
//in case producer ends up in error
// we cancel the token so that producer can intercept it
if (!token.IsCancellationRequested) tokenSrc.Cancel();
throw;
}
}
private static Task RunProducers(IReadOnlyList<IProducer<TP>> producers,
PpcBuffer<TP> buffer, CancellationToken token,
CancellationTokenSource tokenSrc)
{
return Task.Run(async () =>
{
try
{
//following line span all consumers (RunProducer method) in the list
//as separate task
await new Func<int, CancellationToken, Task>(async (i, t) =>
await RunProducer(producers[i], buffer, t, tokenSrc)
.ConfigureAwait(false))
.WhenAll(producers.Count, token).ConfigureAwait(false);
//our home-made WHENALL line waits on all created tasks
// (i.e. waits on all producer to finish)
}
finally
{
//>>>>> IMPORTANT: No matter whether producers finishes normally
// or ends-up in error
// we close the buffer
buffer.CloseForAdding();
}
});
}
private static async Task RunProducer(IProducer<TP> parallelProducer,
IProducerBuffer<TP> feed, CancellationToken token,
CancellationTokenSource tokenSrc)
{
try
{
//this would dispose the producer once we have nothing left
//to produce
using (parallelProducer)
{
//initalize producer
await parallelProducer.InitAsync().ConfigureAwait(false);
token.ThrowIfCancellationRequested();
//we provide our buffer to producer
//it will be producer responcibility to populate it
// and return from it once there is nothing left
// to produce.
await parallelProducer.ProduceAsync(feed, token)
.ConfigureAwait(false);
}
}
catch
{
//in case producer ends up in error
// we cancel the token so that consumer can intercept it
if (!token.IsCancellationRequested) tokenSrc.Cancel();
throw;
}
}
}
We have all the ingredients to cook our extension methods and we propose following four (4) such methods to achieve different pipelines as we had discussed above:
<TP> = <TC>
). Normally end-user need to inject IDENTITY adapter to it, but we can implicitly do it inside our method as shown below:
//IMPLEMENTATION
public static Task Pipe<T>(this IReadOnlyList<IProducer<T>> producers,
IReadOnlyList<IConsumer<T>> consumers,
CancellationToken token = default(CancellationToken),
int bufferSize = 256)
{
return PipeImpl<T, T>.Execute(token, bufferSize,
new IdentityAdapter<T>(),
producers, consumers);
}
//=========================
//========= USAGE==========
//=========================
//var producers = new IProducer<T>[]{ producer1, ..., producerN };
//var consumers = new IConsumer<T>[]{ consumer1, ..., consumerM };
//await producers.Pipe(consumers);
//producer1, ..., producerN denotes comma separated N producer instances
//consumer1, ..., consumerM denotes comma separated M consumer instances
<T>
then consumer type is List<T>
and end-user seek fixed sized chunks. Normally, one needs to inject Awaitable List adapter to it, but we can implicitly do it inside our method as shown below:
//IMPLEMENTATION
public static Task Pipe<T>(this IReadOnlyList<IProducer<T>> producers,
IReadOnlyList<IConsumer<List<T>>> consumers,
int listSize,
CancellationToken token = default(CancellationToken),
int bufferSize = 256)
{
//timeout is INFINITE, we will get FIXED-size chunks
return PipeImpl<T, List<T>>.Execute(token, bufferSize,
new AwaitableListAdapter<T>(listSize, Timeout.Infinite),
producers, consumers);
}
//=========================
//========= USAGE==========
//=========================
//var producers = new IProducer<T>[]{ producer1, ..., producerN };
//var consumers = new IConsumer<List<T>>[]{ consumer1, ..., consumerM };
//await producers.Pipe(consumers, some_positive_int_for_chunk_size);
//producer1, ..., producerN denotes comma separated N producer instances
//consumer1, ..., consumerM denotes comma separated M consumer instances
<T>
then consumer type is List<T>
and end-user seek variable sized chunks created using fixed duration. Normally, one needs to inject Awaitable List adapter to it, but we can implicitly do it inside our method as shown below:
//IMPLEMENTATION
public static Task Pipe<T>(this IReadOnlyList<IProducer<T>> producers,
IReadOnlyList<IConsumer<List<T>>> consumers,
int listMaxSize,
int millisecondTimeout,
CancellationToken token = default(CancellationToken),
int bufferSize = 256)
{
//timeout and MAX list size passed to adapter to avail chunks
return PipeImpl<T, List<T>>.Execute(token, bufferSize,
new AwaitableListAdapter<T>(listMaxSize, millisecondTimeout),
producers, consumers);
}
//=========================
//========= USAGE==========
//=========================
//var producers = new IProducer<T>[]{ producer1, ..., producerN };
//var consumers = new IConsumer<List<T>>[]{ consumer1, ..., consumerM };
//await producers.Pipe(consumers, some_positive_int_for_max_chunk_size,
// some_positive_int_for_timeout);
//producer1, ..., producerN denotes comma separated N producer instances
//consumer1, ..., consumerM denotes comma separated M consumer instances
<TP>
and consumer type is <TC>
and IDataAdapter<TP, TC>
implementation is available to end-user.
//IMPLEMENTATION
public static Task Pipe<TP, TC>(this IReadOnlyList<IProducer<TP>> producers,
IReadOnlyList<IConsumer<TC>> consumers,
IDataAdapter<TP, TC> adapter,
CancellationToken token = default(CancellationToken),
int bufferSize = 256)
{
//timeout and MAX list size passed to adapter to avail chunks
return PipeImpl<TP, TC>.Execute(token, bufferSize,
adapter,
producers, consumers);
}
//=========================
//========= USAGE==========
//=========================
//var producers = new IProducer<TP>[]{ producer1, ..., producerN };
//var consumers = new IConsumer<TC>[]{ consumer1, ..., consumerM };
//IDataAdapter<TP, TC> adapter = ...end-user-adapter-creation-call...
//await producers.Pipe(consumers, adapter);
//producer1, ..., producerN denotes comma separated N producer instances
//consumer1, ..., consumerM denotes comma separated M consumer instances
Deatched pipeline differs a bit as we do not have producers instances available to us at pipeline construction time (i.e. while calling PipeImpl<TP,TC>.Execute
) as we do have for attached mode. Due to the absence of these producers we are do not have any mechanism to populate our buffer. Also, we had discussed during our initial discussion, producers for such a pipeline may appear sporadically. Thus, unfortunately, we will NOT be able to achieve our desired producers.Pipe(consumers)
usage form, however, we attempt to achieve a similar simplified usage form based on following information:
As we do NOT have any single point in code to await
on, we need to fabricate a way to keep our pipeline alive for the whole duration so that all produced items (by ephemeral sporadically appearing producers or long-running producers) can be added in it. For all pragmatic reasons, we measure such duration as: "The time duration starting from the moment when such a pipeline is constructed until the moment when the call to CloseForAdding
method is made."
With these assumptions made and intentions declared, we proceed with detached Pipeline interfacing as follow:
//we adopt this interface as this nearly mimic
//all operations of RunProducers method of
//PipeImpl<TP, TC> static class we used for attached mode
//i.e.
// Add and TryAdd method
// and Dispose method
// we have nothing to Init.
public interface IPipeline<T> : IProducerBuffer<T>, IDisposable
{
}
Now, with IPipeline, we will be able to mimic all the producer related operations as we have done before (in RunProducers
method of PipeImpl<TP, TC> static class
). Lets look at the implementation:
//NOTE: Some explanation are provided as comments
internal sealed class PipelineImpl<TP,TC> : IPipeline<TP>
{
private readonly CancellationTokenSource _mergedCts;
private readonly PpcBuffer<TP> _feed;
private readonly Task _consumerTask;
private CancellationTokenSource _localCts;
public Pipeline(IReadOnlyList<IConsumer<TC>> consumers,
IDataAdapter<TP, TC> adapter,
CancellationToken token,
int bufferSize)
{
_localCts = new CancellationTokenSource();
_mergedCts = CancellationTokenSource.CreateLinkedTokenSource(token,
_localCts.Token);
_feed = new PpcBuffer<TP>(bufferSize, _mergedCts.Token);
//in order to span and await on our consumer
// we simply call the existing implementation
// from PipeImpl class
_consumerTask = PipeImpl<TP, TC>.RunConsumers(consumers, _feed,
adapter, token, _localCts);
}
public void Add(TP item, CancellationToken token)
{
TryAdd(item, Timeout.Infinite, token);
}
public bool TryAdd(TP item, int millisecTimeout, CancellationToken token)
{
//passing the item to buffer
return _feed.TryAdd(item, millisecTimeout, token);
}
public void Dispose()
{
if (_localCts == null) return;
try
{
using (_localCts)
{
using (_mergedCts)
{
using (_feed)
{
//FIRST, we cancel our local token
_localCts.Cancel();
//SECOND, we close the feed for addition
_feed.CloseForAdding();
//Then, we wait for remaining items to be
// consumed
_consumerTask.Wait();
}
}
}
}
finally
{
_localCts = null;
}
}
}
Contrary to attached pipeline, where we had a single place in code to await
on the whole pipeline workflow, in detached mode we do NOT have such a luxury. Thus, end-users need to maintain the instance of IPipeline<TP>
somewhere after the construction and explicitly call the Dispose
on it. This, of course, require some attentions, however, before rejecting the usage of this implementation we need to meditate over following thoughts:
WCF
based, ApiController
based etc) data processingAbove, we have already created .Pipe
extension methods. In similar manner, we can obtain following .Pipeline
methods:
<TP> = <TC>
). Normally end-user need to inject IDENTITY adapter to it, but we can implicitly do it inside our method as shown below:
//Implentation
public static IPipeline<T> Pipeline<T>(this IReadOnlyList<IConsumer<T>> consumers,
CancellationToken token = default(CancellationToken),
int bufferSize = 256)
{
return new PipelineImpl<T, T>(consumers,
new IdentityAdapter<T>(),
token, bufferSize);
}
//=========================
//========= USAGE==========
//=========================
//var consumers = new IConsumer<T>[]{ consumer1, ..., consumerM };
//var save_this_instance_somewhere = consumers.Pipeline();
//consumer1, ..., consumerM denotes comma separated M consumer instances
//===================================
//=========Add/TryAdd USAGE==========
//===================================
//elsewhere (in API methods, Event Handlers etc)
//saved_instance.Add(item, token) OR saved_instance.TryAdd(item, timeout, token)
//===================================
//=========Dispose USAGE=============
//===================================
//during App Shutdown Or after network close
//saved_instace.Dispose();
<T>
then consumer type is List<T>
and end-user seek fixed sized chunks. Normally, one needs to inject Awaitable List adapter to it, but we can implicitly do it inside our method as shown below (IMPORTANT: As producers are sporadic, one might like to avoid this adapter completely as unnecessary consumer side delays will be observed if no producer appears for long time... in detached mode, FIXED DURATION chunk is preferred):
//Implentation
public static IPipeline<T> Pipeline<T>(this IReadOnlyList<IConsumer<T>> consumers,
int listSize,
CancellationToken token = default(CancellationToken),
int bufferSize = 256)
{
return new PipelineImpl<T, T>(consumers,
new AwaitableListAdapter<T>(listSize, Timeout.Infinite),
token, bufferSize);
}
//=========================
//========= USAGE==========
//=========================
//var consumers = new IConsumer<T>[]{ consumer1, ..., consumerM };
//var save_this_instance_somewhere = consumers.Pipeline(some_positive_list_size);
//consumer1, ..., consumerM denotes comma separated M consumer instances
//===================================
//=========Add/TryAdd USAGE==========
//===================================
//elsewhere (in API methods, Event Handlers etc)
//saved_instance.Add(item, token) OR saved_instance.TryAdd(item, timeout, token)
//===================================
//=========Dispose USAGE=============
//===================================
//during App Shutdown Or after network close
//saved_instace.Dispose();
<T>
then consumer type is List<T>
and end-user seek variable sized chunks created using fixed duration. Normally, one needs to inject Awaitable List adapter to it, but we can implicitly do it inside our method as shown below:
//Implentation
public static IPipeline<T> Pipeline<T>(this IReadOnlyList<IConsumer<T>> consumers,
int listMaxSize,
int millisecondTimeout,
CancellationToken token = default(CancellationToken),
int bufferSize = 256)
{
return new PipelineImpl<T, T>(consumers,
new AwaitableListAdapter<T>(listMaxSize, millisecondTimeout),
token, bufferSize);
}
//=========================
//========= USAGE==========
//=========================
//var consumers = new IConsumer<T>[]{ consumer1, ..., consumerM };
//var save_this_instance_somewhere = consumers.Pipeline(some_positive_list_size,
some_positive_timeout);
//consumer1, ..., consumerM denotes comma separated M consumer instances
//===================================
//=========Add/TryAdd USAGE==========
//===================================
//elsewhere (in API methods, Event Handlers etc)
//saved_instance.Add(item, token) OR saved_instance.TryAdd(item, timeout, token)
//===================================
//=========Dispose USAGE=============
//===================================
//during App Shutdown Or after network close
//saved_instace.Dispose();
<TP>
and consumer type is <TC>
and IDataAdapter<TP, TC>
implementation is available to end-user.
//Implentation
public static IPipeline<TP> Pipeline<TP, TC>(this IReadOnlyList<IConsumer<TC>> consumers,
IDataAdapter<TP, TC> adapter,
CancellationToken token = default(CancellationToken),
int bufferSize = 256)
{
return new PipelineImpl<TP, TC>(consumers,
adapter,
token, bufferSize);
}
//=========================
//========= USAGE==========
//=========================
//var consumers = new IConsumer<TC>[]{ consumer1, ..., consumerM };
//IDataAdapter<TP, TC> adapter = ...end-user-adapter-creation-call...
//var save_this_instance_somewhere = consumers.Pipeline(adapter);
//consumer1, ..., consumerM denotes comma separated M consumer instances
//===================================
//=========Add/TryAdd USAGE==========
//===================================
//elsewhere (in API methods, Event Handlers etc)
//saved_instance.Add(item, token) OR saved_instance.TryAdd(item, timeout, token)
//===================================
//=========Dispose USAGE=============
//===================================
//during App Shutdown Or after network close
//saved_instace.Dispose();
So far, we have implemented all the initially set requirements. Before we close this discussion, we would like to wrap our features:
| Feature | Implementation |
|---|---|
| Buffer Size | Through method parameter (bufferSize); 0 is unbounded |
| Losslessness | using millisecond Timeout during Add/TryAdd, Timeout.Infinite represents no-loss |
| Interruptibility | Using CancelationToken |
| Concordance | Use of Adapters |
| Attachability |
Attached mode implementation using .Pipe extension methods Detached mode implementation using .Pipeline extension methods |
We have also noticed:
In our original work (Source Code Link, NuGet Package Link), we have further elaborated our implementation as explained below:
<TP>
to <TC>
data transformation can be done solely based on business logic without worrying about calls to buffer.TryGet
(see AwaitableAdapter and AwaitableListAdapter)Thus, it would be simple to inherit from either AwaitableAdapter<TP, TC>
abstract class (if consuming single instances at a time) or AwaitableListAdapter<TP, TC>
abstract class (if consuming data in chunks) instead of implementing IDataAdapter<TP, TC>
interface. Such abstract class
based inheritance would remain business-oriented as we purely write the data transformation logic inside Adapt
method without worrying about buffer handling, and thus, further reducing boiler-plate code.
.Pipe
and .Pipeline
extension methods are also available on Action
(synchronous delegates) and Func
(task returning asynchronous delegates). Thus, avoiding the need to inherit IProducer
and IConsumer
interfaces when Init/Dispose
methods are not warranted. (see PipeExts and PipelineExts)ls -l | grep key | less (3 operations with 2 pipes)
void
consumers, i.e. consumers cannot have return values.IMPORTANT: We suggest you to use v1.4.0 or higher; as it contains some breaking changes compared to previous versions. This library also contains some other interesting extension methods which we might cover in future articles on code-project itself. However, if you are interested in usage of those methods, you can find information here.
.Pipe
implementation, instead of thinking about whole sln in a go, one must organise ones thoughts as follows:
IProducer<TP>
interface to populate the buffer IF Init
/Dispose
methods are requiredAction<IProducerBuffer<TP>, CancellationToken>
Func<IProducerBuffer<TP>, CancellationToken, Task>
IConsumer<TC>
interface to populate the buffer IF Init
/Dispose
methods are requiredAction<TC, CancellationToken>
Func<TC, CancellationToken, Task>
AwaitableAdapter<TP, TC>
or AwaitableListAdapter<TP, TC>
if requirements fits and implement abstract method: abstract TC Adapt(TP produced, CancellationToken token)
IAdapter<TP, TC>
interface.Pipeline
implementation, one must organise ones thoughts as follows:
.Pipe
usageIPipeline<TP>
instance as deemed fit:
Dispose
on the instance as deemed fit:
IPipeline<TP>
instance as required:
ApiController
WCF
endpointsEventHandler
Timer
based callbacks... so on and so forthThis is the v1 of the present idea.
<< Previous Article (Rendezvous with JSON) |


Image Organizer
Sort:
Name
Contributor
Description
Add Image
Save Changes
Fetch Images
Delete Image
{binding Name}
Name:
Description:
Uri:
Contributor:


I am an full-stack website developer.
What do I am mean by full-stack website developer? Being a software engineer at heart, I know it takes more than HTML and CSS to bring the Web to life. I care about the whole stack, looking for creatives ways to solve problems and always keep the customer's needs in mind I also look for new ways to improve U.I's and API performances during and after development.


3 writes to transactionManager
System.Management.Automation (3)
engine\ExecutionContext.cs (3)
72
this.
transactionManager
= new PSTransactionManager();
1188
this.
transactionManager
= null;
1566
transactionManager
= new PSTransactionManager();
5 references to transactionManager
System.Management.Automation (5)
engine\ExecutionContext.cs (5)
67
if (this.
transactionManager
!= null)
69
this.
transactionManager
.Dispose();
1183
if (this.
transactionManager
!= null)
1185
this.
transactionManager
.Dispose();
1273
return
transactionManager
;


|
CloudTV Nano SDK (C++ Northbound API)
4.4
|
Player for content streams i.e. various sources of streaming video. More...
Data Structures | |
| struct | ICallback |
| Callback interface for player status updates. More... | |
| struct | PlayerInfo |
| Struct that contains player information (state and statistics) to be passed back to the stream originator. More... | |
Public Types | |
| enum | PlayerEvent { PLAYER_STARTING, PLAYER_STARTED, PLAYER_STOPPED, PLAYER_BUFFER_UNDERRUN, PLAYER_BUFFER_OVERRUN, PLAYER_RECOVERABLE_ERROR, PLAYER_UNRECOVERABLE_ERROR, PLAYER_DESCRAMBLE_ERROR, PLAYER_DECODE_ERROR, PLAYER_TRANSPORT_STREAM_ID_ERROR, PLAYER_NETWORK_ID_ERROR, PLAYER_PROGRAM_ID_ERROR, PLAYER_PHYSICAL_ERROR } |
| Player event definition. More... | |
Public Member Functions | |
| virtual ResultCode | open_stream (const std::string &uri, const std::map< std::string, std::string > &stream_params, IStream &stream_out, IStream *&stream_in)=0 |
| Called when streaming content should be opened (setup). If the media player will pass the stream through the SDK, the stream_in parameter should be filled in with the return path (i.e. the interface of the stream player). The stream_out parameter contains the SDK object the loaded stream can be sent to. If no routing is supported, stream_in must either be left unchanged or set to 0. More... | |
| virtual void | close_stream ()=0 |
| Called when the library wishes to stop the content. | |
| virtual void | get_player_info (PlayerInfo &info)=0 |
| Obtain player state and statistics. More... | |
| virtual void | register_callback (ICallback *callback)=0 |
| Register a callback interface. More... | |
Static Public Attributes | |
| static const ResultCode | CABLE_TUNING_ERROR |
| There was a tuning error when trying to tune to a channel. | |
| static const ResultCode | CONNECTION_FAILED |
| Connection to a remote host could not be established. | |
Subclass the IMediaPlayer to implement a media player that can resolve a media URI. This is then bound to a particular type of content with Session::register_media_player(). A MediaPlayer is given URIs to content streams, and so must be able both to retrieve the indicated resource and consume its content.
| enum PlayerEvent |
| Enumerator | |
|---|---|
| PLAYER_STARTING |
The player just started. This event should be sent as soon as the start() method of the player is called. |
| PLAYER_STARTED |
The player started. This event must be sent as soon as the first decoded frame is displayed (or as near as possible). Sent in response to a call to start(). |
| PLAYER_STOPPED |
The player stopped. This event must be sent as soon as the last decoded frame was displayed (or as near as possible). Sent in response to a call to stop(), if started. May also be sent upon the call to register_callback(). |
| PLAYER_BUFFER_UNDERRUN |
The player experienced a buffer underrun. Non-fatal. Once the underrun condition has stopped, the player should resume normal, minimal-latency decoding and the PLAYER_STARTED event MUST be sent. |
| PLAYER_BUFFER_OVERRUN |
The player experienced a buffer overrun. Fatal. The player can stop playing. It should expect a successive call to stop(). |
| PLAYER_RECOVERABLE_ERROR |
The player experienced an error that is recoverable. Non-fatal. After recovery, the player should continue normal, minimal-latency decoding and the PLAYER_STARTED event MUST be sent. |
| PLAYER_UNRECOVERABLE_ERROR |
The player experienced an error that is unrecoverable. Fatal. The player can stop playing. It should expect a successive call to stop(). |
| PLAYER_DESCRAMBLE_ERROR |
There was an error descrambling the stream. Fatal. The player can stop playing. It should expect a successive call to stop(). |
| PLAYER_TRANSPORT_STREAM_ID_ERROR |
!< The client failed to decode the stream. Fatal. |
| PLAYER_NETWORK_ID_ERROR |
!< No transport stream with the indicated Transport Stream ID was found. Fatal. |
| PLAYER_PROGRAM_ID_ERROR |
!< No network with the indicated Network ID was found. Fatal. |
| PLAYER_PHYSICAL_ERROR |
!< No program with the indicated Program ID was found. Fatal. !< Unrecoverable error at the physical layer. Fatal. |
|
pure virtual |
| [in] | info | Player state and statistics information to be returned. Fields that can be filled should be set by the player implementation; fields for which information cannot be obtained should be left alone. This way, the caller knows which fields have been set and which not. |
|
pure virtual |
| [in] | uri | URI to open. |
| [in] | stream_params | Stream parameters that the player can use to check if playback is possible or not. In general, the stream parameters should be regarded as a hint, the stream itself is always leading. However, some applications require certain stream parameters to be processed for proper operation. This is application-specific. Valid parameters are documented in the RFB-TV specification (section "Optional stream parameters"), e.g. "video_width", "audio_codec" or "ca_data". RFB-TV 1.3.2 streaming parameters are mapped to the keys/values defined in RFB-TV 2.0.9. |
| [in] | stream_out | The IStream object the loaded stream must be sent to. |
| [out] | stream_in | The IStream object that will receive the processed stream. |
|
pure virtual |
| [in] | callback | This object is to be used to signal playback events to. Passing a null pointer will unregister the callback. |


Aproximadamente 183.000.000 resultados (0,40 segundos)
Reference site about Lorem Ipsum, giving information on its origins, as well as a random Lipsum generator.
Generate Lorem Ipsum placeholder text for use in your graphic, print and web layouts, and discover plugins for your favorite writing, design and blogging tools.
In publishing and graphic design, Lorem ipsum is a placeholder text commonly used to ... De finibus bonorum et malorum (in Latin and English). New York: ...
Lorem ipsum dolor sit amet, consectetuer adipiscing elit. aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis ...


1 IDL Compiler Release Notes
1.1 IC 4.4.2
Fixed Bugs and Malfunctions
-
Correct bugs when path to mib or idl spec files contains UTF-8 characters.
Own Id: OTP-13718 Aux Id: ERL-179
-
Update build scripts to not make assumtions about where env, cp and perl are located.
Own Id: OTP-13800
1.2 IC 4.4.1
Improvements and New Features
-
Internal changes
Own Id: OTP-13551
1.3 IC 4.4
Improvements and New Features
-
Change license text from Erlang Public License to Apache Public License v2
Own Id: OTP-12845
1.4 IC 4.3.6
Fixed Bugs and Malfunctions
-
Fix compiler warnings reported by LLVM
Own Id: OTP-12138
1.5 IC 4.3.5
Fixed Bugs and Malfunctions
-
Added Latin-1 code directive in the generated files to keep old behaviour. Updated IC so it can handle Unicode characters in the path.
Own Id: OTP-11783
1.6 IC 4.3.4
Fixed Bugs and Malfunctions
-
Fix two small silent rules omissions. Thanks to Anthony Ramine.
Own Id: OTP-11351
-
Silence warnings (Thanks to Anthony Ramine)
Own Id: OTP-11517
1.7 IC 4.3.3
Improvements and New Features
-
Header and library files from ic and erl_interface are now installed into usr/{include,lib}. Note that these directories are unversioned, so the latest installed version will be the one in the directory.
Own Id: OTP-11284
1.8 IC 4.3.2
Fixed Bugs and Malfunctions
-
Fixed some compilation warnings on miscellaneous platforms. Thanks to Anthony Ramine.
Own Id: OTP-11086
1.9 IC 4.3.1
Fixed Bugs and Malfunctions
-
Revert the structs erlang_pid, erlang_port and erlang_ref as they were before R16A (without node_org_enc) in order to be backward compatible with user code that accesses the fields of these structs.
Own Id: OTP-10885 Aux Id: seq12256
1.10 IC 4.3
Improvements and New Features
-
Misc build updates
Own Id: OTP-10784
-
Adapt ic for changes in erl_interface and jinterface due to utf8 atom support. This change makes ic dependent on erl_interface-3.7.10 (R16) or later in order to build.
Own Id: OTP-10785
1.11 IC 4.2.31
Fixed Bugs and Malfunctions
-
Fix bug where the ic pre-processor would ignore whitespace quoting.
Own Id: OTP-10109
-
A bug regarding spaces in C function prototypes has been fixed. (Thanks to Richard O'Keefe.)
Own Id: OTP-10138
1.12 IC 4.2.30
Improvements and New Features
-
Add generation of Erlang callback functions to generated Erlang source code to avoid compiler warnings.
Own Id: OTP-9998
1.13 IC 4.2.29
Improvements and New Features
-
Erlang/OTP can now be built using parallel make if you limit the number of jobs, for instance using 'make -j6' or 'make -j10'. 'make -j' does not work at the moment because of some missing dependencies.
Own Id: OTP-9451
1.14 IC 4.2.28
Fixed Bugs and Malfunctions
-
Incorrect use of ets:match changed to ets:match_object.
Own Id: OTP-9630
1.15 IC 4.2.27
Improvements and New Features
-
Reduced compile overhead (Thanks to Haitao Li).
Own Id: OTP-9460
1.16 IC 4.2.26
Improvements and New Features
-
Partial support for recursive structs and unions. Only available for the erl_corba backend and requires that Light IFR is used. I.e. the IC option {light_ifr, true} and that Orber is configured in such a way that Light IFR is activated. Recursive TypeCode is currently not supported.
Own Id: OTP-8868 Aux Id: seq11633
1.17 IC 4.2.25
Improvements and New Features
-
The documentation can now be built and installed without Java.
Own Id: OTP-8639 Aux Id:
1.18 IC 4.2.24
Fixed Bugs and Malfunctions
-
Removed superfluous VT in the documentation.
Own id: OTP-8353 Aux Id:
-
The option c_timeout was not correctly documented.
Own id: OTP-8307 Aux Id: seq11390
-
Removed superfluous backslash in the documentation.
Own id: OTP-8354 Aux Id:
-
The documentation EIX file was not generated.
Own id: OTP-8355 Aux Id:
1.19 IC 4.2.23
Improvements and New Features
-
The documentation is now built with open source tools (xsltproc and fop) that exists on most platforms. One visible change is that the frames are removed.
Own Id: OTP-8201 Aux Id:
1.20 IC 4.2.22
Fixed Bugs and Malfunctions
-
The 64-bit version of libic was not compiled with the -fPIC flag.
Own id: OTP-8088
1.21 IC 4.2.21
Fixed Bugs and Malfunctions
-
The function print_erlang_binary (oe_ei_code_erlang_binary.c) updated to avoid compiler warning.
Own id: OTP-7982
1.22 IC 4.2.20
Improvements and New Features
-
Updated file headers.
Own id: OTP-7837
1.23 IC 4.2.19
Improvements and New Features
-
Documentation source included in open source releases.
Own id: OTP-7595
1.24 IC 4.2.18
Fixed Bugs and Malfunctions
-
Insufficient buffer allocated when passing wide strings using the C backend on a 64-bit architecture.
Own Id: OTP-7313 Aux Id:
1.25 IC 4.2.17
Improvements and New Features
-
Updated file headers.
Own id: OTP-7011
-
IC no longer use the obsolete function file:rawopen/2.
Own id: OTP-7182
1.26 IC 4.2.16
Improvements and New Features
-
Added links to classes inherited from Jinterface in the User's Guide.
Own Id: OTP-6965 Aux Id:
1.27 IC 4.2.15
Fixed Bugs and Malfunctions
-
If an inherited function name begun with a capital letter the generated stub/skeleton oe_tc/1 function was incorrect.
Own Id: OTP-6855 Aux Id:
1.28 IC 4.2.14
Improvements and New Features
-
The documentation source has been converted from SGML to XML.
Own Id: OTP-6754 Aux Id:
1.29 IC 4.2.13
Improvements and New Features
-
Minor Makefile changes.
Own Id: OTP-6701 Aux Id:
1.30 IC 4.2.12
Improvements and New Features
-
Dead code was deleted from the following modules: ic_cclient, ic_code, ic_cserver, ic_erlbe, ic_java_type, ic_noc, ic_plainbe, ic_pp, ic_pragma, icscan, icstruct, ictype, icunion.
1.31 IC 4.2.11
Improvements and New Features
-
Changed code generation to avoid warnings such as unused variables.
Own Id: OTP-5930 Aux Id:
1.32 IC 4.2.10
Fixed Bugs and Malfunctions
-
The FD_SETSIZE limit has been increased to 2048 for VxWorks/PPC603.
Own Id: OTP-5395 Aux Id: seq9751
1.33 IC 4.2.9
Fixed Bugs and Malfunctions
-
In C back-ends, the compiler crashed when generating C code for error reports when a scoped name was used as a type in a union.
Own Id: OTP-5375 Aux Id: seq9740
1.34 IC 4.2.8
Fixed Bugs and Malfunctions
-
In C back-ends, when decoding a sequence of "small" integers, which from Erlang is sent as a string (i.e. each element between 0 and 255), each string element was considered to be of signed character type. Each such element is now correctly treated as an unsigned character type.
Own Id: OTP-5205 Aux Id: seq9241
1.35 IC 4.2.7
Improvements and New Features
-
A new compiler option c_report has been introduced for C back-ends (client and server). If that option is set, encoding/decoding errors will be reported to stderr.
Own Id: OTP-4977
1.36 IC 4.2.6
Improvements and New Features
-
The size of modules, used then registering data in the IFR DB (e.g., oe_MyModule:oe_register()), can be minimized if the compile option light_ifr is used and Orber is configured to use Light IFR. Requires that orber-3.5.1, or later, is used.
Own Id: OTP-5036
Incompatibilities
-
The compile option multiple_be is no longer supported.
Own Id: OTP-5049
1.37 IC 4.2.5
Improvements and New Features
-
Send and receive functions with timeouts have been added to the C back-ends for the standard protocol (i.e. Erlang distribution + gen_server protocol).
Accordingly a new compiler option {c_timeout, {SendTimeout, RecvTimeout}} has been added. Timeouts are specified in milliseconds.
A user that want to implement its own protocols with function timeouts has to implement the following functions.
For C clients the functions int PFX_send_notification(CORBA_Environment *env, unsigned int send_ms), and int PFX_send_request_and_receive_reply(CORBA_Environment *env, unsigned int send_ms, unsigned int recv_ms) have to be additionally implemented, where PFX is the user defined prefix.
For C servers no additional functions have to be implemented, but a clone of the int oe_server_receive_tmo(CORBA_Environment *env, oe_map_t *map, unsigned int send_ms, unsigned int recv_ms) might be handy.
Own Id: OTP-4972
1.38 IC 4.2.4
Improvements and new features
-
The C back-ends has been opened up, so that a user can define his own protocol, differing from the Erlang distribution + gen_server protocol.
For C clients it means to replace the library functions int oe_prepare_notification_encoding(CORBA_Environment *env), int oe_send_notification(CORBA_Environment *env), int oe_prepare_request_encoding(CORBA_Environment *env), int oe_send_request_and_receive_reply(CORBA_Environment *env), and int oe_prepare_reply_decoding(CORBA_Environment *env), with functions of the same signature, but with the prefix "oe" replaced by a user defined prefix. For C servers the functions int oe_prepare_request_decoding(CORBA_Environment *env), and int oe_prepare_reply_encoding(CORBA_Environment *env), are similarly replaced.
The new compiler option {user_protocol, Prefix} has been added.Own Id: OTP-4834
1.39 IC 4.2.3
Fixed Bugs and Malfunctions
-
In generated code for the C server back-end, the naming scope was in error for prototypes in C header files for interfaces inheriting base interfaces.
Own Id: OTP-4881
1.40 IC 4.2.2
Fixed Bugs and Malfunctions
-
IDL long long and unsigned long long could not be used in a struct for the Java backend.
All unsigned integer types for the Java backend had broken marshalling for large values.
Own Id: OTP-4763
1.41 IC 4.2.1
Fixed Bugs and Malfunctions
-
A scoping problem (IC could not find typedefs contained inherited interfaces) in the C-backend solved.
Own Id: OTP-4758
1.42 IC 4.2
Improvements and New Features
-
The CORBA stub/skeleton-files generated by IC have been improved, i.e., depending on the IDL-files, reduced the size of the erl- and beam-files and decreased dependencies off Orber's Interface Repository. It is necessary to re-compile all IDL-files and use COS-applications, including Orber, compiled with IC-4.2.
Own Id: OTP-4576


When It Rains It “Poors”: Poverty and Rainfall in Central Asia
(Job Market Paper)
This paper evaluates the causal relationship between rainfall and measures of poverty in central Asia using panel data.
Using plausibly exogeneous variation in weather, I find that changes in rainfall predicts changes in
the percentage of the population in poverty by between 0.2 and 0.5 standard deviations.
This research implies that natural conditions result in large changes in material well-being.


Jimy Encomendero
School of Electrical and Computer Engineering
Cornell University
Jimy Encomendero is currently a Ph.D. student and researcher working in the School of Electrical and Computer Engineering at Cornell University. Originally from Perú, he received his B.S. degree in Electronics Engineering from Universidad Nacional de Ingeniería and his M.S. degree in Electrical Engineering from the University of Notre Dame.
Jimy’s current research interests focus on the design, epitaxial growth, and fabrication of III-Nitride semiconductor quantum heterostructures. He has contributed to the current understanding of electronic quantum phenomena in polar resonant tunneling heterostructures and their practical device applications. He has demonstrated the first III-Nitride resonant tunneling oscillator, which has been featured in the cover of Applied Physics Letters. He is the author of several publications in scientific journals including Physical Review X, Physical Review Applied, and the Journal of Applied Physics. He has presented his work in multiple conferences including EMC, IWN, ICNS and ICPS. He is also the recipient of the 2018 NIST Student Award.
News
| Oct 5, 2020 | We report the first demonstration of resonant tunneling diodes grown along the Nitrogen-polar direction of the III-nitride wurtzite lattice. Check out our paper published in Applied Physics Letters! [DOI] [PDF]. |
| Mar 19, 2020 | Our paper on doping engineering of III-Nitride RTDs has been published in Physical Review Applied! [DOI] [PDF]. |
| Feb 18, 2020 | We report the first III-Nitride Tunneling Field-Effect Transistor in Applied Physics Letters! [DOI] [PDF]. |
| Aug 1, 2019 | Our book on High-Frequency GaN Electronic Devices has been published! Check out our contributed chapter about the physics of Resonant Tunneling Transport in Polar III-Nitride Heterostructures. |
| Mar 13, 2019 | We have uncovered the physics of electronic resonant tunneling transport in GaN/AlN RTDs! Our paper, published in Physical Review Applied, has been selected as an Editor’s Suggestion! [DOI] [PDF]. |


An example showing rendering via OpenGLES2 in a Native Client module.
Note: the C++ source code of this demo is available in the Native Client SDK (examples/api/graphics_3d).
FPS: 0


--- layout: default title: Case Studies subhead: Unindexed page for links permalink: /case-studies/ ---
{{ study.description }}


Print the version of minikube.
minikube version [flags] [options]
Use "minikube options" for a list of global command-line options (applies to all commands).


slashpackage is a convention proposed by DJB to have a reliable, cross-platform way of accessing files.
(Paul Jarc has a presentation of slashpackage that you may want to view.)
For instance, the execline package follows the slashpackage convention by default: this means that, among others, the execlineb binary can be found at /package/admin/execline/command/execlineb no matter the underlying installation, no matter the platform, no matter the OS - as long as the convention is followed.
This is a key point for system reliability. It is especially important for a script interpreter such as execlineb: people might distribute scripts, starting with a #! line, and if the shebang path is wrong on some machine, the script will break on that machine. Long story short: packages should be able to hardcode paths to other packages' components. The FHS alone does not guarantee this.
This is irrelevant for two reasons:
This is off-topic, but I'll address it here to be done with it.
Yes, of course there are good points in the FHS. The people who designed it in the first place were not idiots. However, this was a long time ago, and the problems that had to be solved then are mostly irrelevant now.
For instance, /usr was created so you could have a minimal system on a local disk, then mount /usr via NFS. Mass storage was a problem at the time, so it was easier and cheaper to only have one copy of /usr on an NFS server. Today, we have lots of cheap mass storage, and bootable CDs; the distinction between /bin and /usr/bin is largely irrelevant.
Same thing with /bin and /sbin. Why are some binaries stored under /sbin ? It's not to prevent the user from accessing them: /sbin is world-readable on practically every system. What is the benefit of having a separate /sbin for administrator programs, over just putting those in /bin, with a 0700 mode if one is paranoid ? This one is left as an exercise for the reader; but in any case, it does not make sense today.
The only filesystem hierarchy a modern system needs is the following:
The FHS addresses this constraints well enough, but due to historical legacy cruft, it does way more than is necessary, and it adds complexity.
The important point, though, is that there is ONE vital thing that the FHS does not do: ensure cross-platform accessibility of packages. I am not blaming the FHS: it was not designed to do that. But the need is there, and denying it only postpones the problems and makes life hard for users. Since the FHS cannot address the need, we need an additional convention. And slashpackage does the job.
I understand your point of view, but disagree with it. Please read this page about cross-platform compatibility.
In my opinion, the right way for a Unix distribution to package software is not to "unify" it and make a "polished" distribution. It's to friggin' package the software, i.e. provide a tarball and automatic installation, configuration and removal scripts.
By doing more than this, and especially by bypassing primary authors' decisions about how files should be accessed, Unix distributions are creating more work. More work for themselves, because they have to maintain subtly different versions of software than what the primary author provides; and more work for the users, because instead of learning "execline", a user now has to learn "Ubuntu execline", "Fedora execline", "OpenBSD execline".
Anyway, I have no objection to following the FHS. None at all. If you're going to package my software, you're free to put the files wherever you want. All I ask is that you do add symbolic links so files can be accessed via /package, and via /command too for executables. If your policy does not allow it, then I ask that you:
Well, I used to not care at all, because I mainly write software for myself, and quality - which includes reliability - is more important to me than widespread use. (Seriously, if widespread use was a good metric for quality, I'd be applying for a job at Microsoft.) After all, I'm rewriting the world, so let me do it my way, right ?
However, I have come to realize that unlike actual software, a convention is only good if it is followed by a non-negligible amount of people. The Unix world is certainly ready for better software, but it is apparently not ready for better filesystem organization.
So, starting in 2013, all skarnet.org releases now feature a way to build and install software on a non-slashpackage installation. I hope this will encourage more people to use it.
This means some responsibility has been transferred from the author to the packagers and the system administrators. If you are forfeiting the slashpackage guarantees, then it is up to you to make sure things work. In particular, you should pay attention to the following points:
But it's your dime.


viznet is designed for visualizing networks composed of nodes and edges, e.g. tensor networks, neural networks and quantum circuits.
It is based on and compatible with matplotlib. The theme brush (for both node and edge) makes the design itself interesting, getting you free from fine tuning the node and wire parameters for hours.
$ pip install viznet
or for the latest version
$ git clone https://github.com/GiggleLiu/viznet.git
$ cd viznet
$ pip install -r requirements.txt
$ python setup.py install
$ cd viznet
$ python apps/nn/bm.py # example on neural network
$ python apps/tn/tebd.py # example on tensor network
$ python apps/qc/ghz.py # example on quantum circuit
you will get something like
| Boltzmann Machine | TEBD |
|---|---|
| quantum circuit |
|---|
The theme for neural network follows from Neural Network Zoo Page,
The theme for quantum circuits follows from ProjectQ.
The first release of viznet (v0.1) was developed by Jin-Guo Liu in the group of Lei Wang at IOP China.
Click here to read the docs!


Iteration
Iteration is the process of repeating nodes in the DOM for each row in a corresponding array. Practically this is equivalent to iteration in a templating system. The difference is that iteration is achieved by cloning nodes. Each time the DOM is rendered it is synchronized with the current contents of the corresponding array. This means that the iteration is dynamically kept in sync with changing values in the array. It is highly optimized so that nodes are re-used when possible rather than re-generating each time. Each iteration may reference either through binding or in conditional expressings a specific row in the array
Iteration can be applied using the <b:iterate> tag.
- on a reference to an array within the model that is to control the iteration.
- with a reference to a property in the model that will be set to the row being iterated. It can be referenced in conditionals, binding or in events (that use the b: namespace)
- index a reference to a property in the model that will be set to the zero-based oridinal of the row being iterated.
- counter a reference to a property in the model that will be set to a one based counter for each row being iterated.
- filter an expression that is evaluated on each iteration to determine whether the row in the array will be processed.
Consider this simple example of iteration:
and note the following:
- <b:iterate> on line 24 is a node that will be iterated on (e.g. cloned) for each element in the array
- on refers the states array on line 10 within the model
- with refers to the state property in the model. This need not be initialized in the model as iterate will do that as soon as it iterates. You can see that the binding references in lines 27 and 28 refer to the state property to bind to the correct row being iterated
- index refers to the ax property in the model which will be set to the ordinal position in the array. It is referenced in line 26 to display the state number
- counter is a counter referenced in line 25 to set an even / odd class to make the display more readable
- fiter an expression that matches the current state name with the search criteria entered by way of the bind on line 21
Here is the working example:
A few important nodes on iteration:
- The b:iterate tag is provided for convenience. Whenever you use a tag in the BINDster namespace, the attributes of the tag are prefixed with the name of the tag. So the real attribute names for the iteration attributes are b:iterate-on, b:iterate-with, b:iterate-index, b:interate-counter, b:iterate-filter. You can apply iteration to any element by using the real attribute names
- When using iterate on table rows be sure you use the <tbody> tag. Browsers are very picky about mixing namespaces and in fact move the <iterate> outside of the table before BINDster gets to process it.
- Iteration of very large datasets can be slow. The example's full source shows the use of the __stats property of the controller which is populated with performance information. Note that this is usually the previous render. The debugger can hugely slow down DOM rendering in BINDster.


--- status: publish layout: page author: display_name: JP email<EMAIL_ADDRESS>tags: [] published: true date: 2013-07-02 09:33:34 -0600 title: Basic Metadata Instructions categories: [] ---
By some accounts, no one in their right mind could claim to enjoy geospatial metadata.
But that doesn't mean its hard. And while it might not be fun there is no doubt that it can be quite useful.
Since most people in Utah use ArcCatalog to create and maintain geospatial metadata, we've tried to make it easy for those contributing data to Utah's State Geographic Information Database (SGID) to provide at least the basic set of metadata that we'd like to see available for all SGID data.
What follows below is a step by step guide to authoring good metadata using the ArcCatalog FGDC Metadata Editor. These are just guidelines that we have adapted and interpreted from the FGDC Metadata Quick Guide to work specifically within ArcCatalog.
Notes:
Basic Metadata Instructions:
To begin editing metadata, open ArcCatalog and add the Metadata Toolbar from the Tools menu-> Customize-> Toolbars and selecting Metadata check box. Highlight the feature class who's metadata you would like to edit and select the Metadata tab. The tools in the Metadata Toolbar are now available, click the Edit metadata tool to begin.
Identification - General Tab:
Item 1: Abstract. Be sure to include:
Item 2: Purpose.
Item 3: Supplemental Information. This is a basically an open-ended comment field in which you can:
Item 4: Access Constraints.
If data are exempt from public records laws such as endangered species, personal health, and intellectual properties, please describe and inform AGRC via email as our normal procedure for SGID data is to offer it to a public audience with no access constraints.
Item 5: Data Set Credit.
Identify others that should be recognized for their contributions to the data set. This includes data development contractors as discussed, below, for Originator.
Identification - Contact Tab:
Item 6: Point_of_Contact.
The individual or organization that is knowledgeable about the data set and should be contacted with questions.
Item 7: Citation Details.
Publication_Date:
The date that the data was published or otherwise made available. Remember format: YYYYMMDD.
Identification - Citation Tab (ESRI FGDC Metadata Editor):
Item 8: Title:
Item 9: Originator:
It is recommended that you indicate the party responsible for the data set. While that is most commonly the organization that developed the data set, in some cases, it is not. For example, if a county planning department hires a contractor to build a street centerline road file, the planning department, not the contractor should be identified as the Originator. The contractor should be fully cited using the Data_Set_Credit (1.11) element, e.g. ‘this data set was developed for the Wayne County Planning Department by Smith Engineering, Inc’.
Item 10: Publication Date:
The date that the data was published or otherwise made available. Remember format YYYYMMDD.
Identification - Time Period Tab:
Item 11: Currentness Reference:
The context for the Time_Period_of_Content. For example: an orthophotograph may have been compiled and delivered in June (publication date) but flown in February (ground condition).
Item 12: Time_Period_of_Content:
The relevant date of the data content. Can be a single date, multiple dates, or a range of dates.
Identification - Status Tab:
Item 13: Progress:
The status of the data set, this field has a fixed domain of: “Complete”, “In Work”, and “Planned”.
Data Quality - General Tab:
Item 14: Completeness Report:
Is there anything I might expect to be in the data set that isn’t? Identification of data omitted from the data set that might normally be expected, as well as the reason for the exclusion. This may include geographic exclusions, ‘data was not available for the South Shores neighborhood’; categorical exclusions ‘municipalities with populations under 1,000 were not included’; and definitions used ‘floating marsh was mapped as land.'
Data Quality - Attribute Accuracy Tab:
Item 15: Attribute Accuracy Report
How sure are you that it IS a pine tree?
Assessments as to how ‘true’ the attribute values may be. May refer to field checks, cross-checks with other documents, statistical analysis of values, and parallel independent measures. It does NOT refer to the positional accuracy of the feature.
Data Quality - Positional Accuracy Tab:
Item 16: Positional Accuracy Report:
Entity Attribute Tab:
Item 17. Attribute/Field Descriptions
Functionally, this is quite possibly the most important section of the metadata. Describe the use of 3 strategies (user should choose 1) for describing attribute values for each field
1. Use Detailed Description --> Attribute --> General tab to create a description for each field describing what the attribute values represent
2. Use Detailed Description --> Attribute --> General tab to create a general field description and then use the Attribute Domain Values tab to enter codes and descriptions for each field.
3. Use Overview Description Tab's Entity and Attribite Overview or Entity and Attribite Detail Citation Link to an external (URL) or imbedded (and hopefully non-proprietary format) document within the metadata
Metadata Reference Tab:
Item 18. Metadata contact
Provide metadata contact name or organization
That's it, we're done. But there's lots more possibilities for documenting your data within the FGDC metadata framework. There are even strategies for feature-by-feature level data documentation (using the ISO standard or just an attribution field/value strategy to store important items in the data table itself.
Good luck and thanks for your attention to metadata.


This `div` is first. This nested `span` is last! This nested `em` is first, but this nested `em` is last! This `b` qualifies! This is the final `div`!


OCTOBER SKY
*** (out of 4) - a good movie
Release Date: February 19, 1999 Starring: Jake Gyllenhaal, Laura Dern, Chris Owen, William Lee Scott, Chad Lindberg, Chris Cooper, Natalie Canerday Directed by: Joe Johnston Distributed by: Universal Pictures MPAA Rating: PG (language, brief teen sensuality and alcohol use, some thematic elements) URL: http://www.execpc.com/~kinnopio/reviews/1999/october.htm
Since 1990, the dramatic picture has undergone a certain change of style. Now, instead of emphasizing feel-good stories replete with characters of noble virtues, the screenplays and direction point toward human struggle and sorrow. This realist tendancy can be inspiring at times, as 1993's SCHINDLER'S LIST shows. Most of today's mainstream drama, however, creates an overabundance of emoting that can turn off members of the audience. That's why pictures like OCTOBER SKY, which are a throwback to the good ol' dramas of yesteryear, are such a welcome relief in contemporary cinema.
The movie is based on the true story of Homer Hickam (Jake Gyllenhaal), a teenager in the 1950's company town of Coalwood, West Virginia. It's a well-known fact that the only boys to make a name for themselves are the ones that get out of the town on football scholarships; the rest grow up to work in the coal mine. The mine, which is run by Homer's father (Chris Cooper), is not the place for Homer, however. After watching the Soviet satellite Sputnik shoot across the October sky in 1957, Homer feels a connection with the outside world -- as he puts it, "Anyone in the world could look up and see the same thing I did." Drafting two of his best friends and the brainiac outcast, Homer begins to construct rockets. With the help of their history teacher, Miss Riley (Laura Dern), and several mine workers, the four are soon bound for great things.
October Sky is a skillfully crafted film, and most of that credit must go to director Joe Johnston. Johnston -- whose career in the film industry includes everything from directing well-known titles like JUMANJI and THE ROCKETEER down to an uncredited role as a Death Star trooper in STAR WARS -- certainly knows what he's doing here. He molds characters which are true to life and also fun to watch into a plot that is straightforward and robust. He does not push any barriers, but instead adopts a conservative style of filmmaking which enhances the film's charm. His finesse brings out elements which might otherwise be lost -- such as the relationships between Homer and various mine workers, or the silent struggle between Homer's mother and father.
The cast is well chosen, although the true bright spots are the supporting roles. Laura Dern (JURASSIC PARK) lends a strong performance in the role of the history teacher, giving the boys the support and guidance they need. Her role is instantly likeable. Chris Cooper (THE HORSE WHISPERER), as Homer's father, is less likeable -- even downright despicable at times -- but his turn here gives gravity to the movie. He's definitely a driving force in Homer's life, and his more humane moments give the piece a certain dynamic that is commendable.
The script is generally cheerful, projecting the boys' endeavors in a lighthearted manner. There are moments when it becomes predictable -- such as the train tracks scene -- and there are moments when it sacrifices orginality for the need to come up with an emotional plot twist. It highlights the essence of youth, however, and for its exceptional job at this it can be forgiven other missteps.
OCTOBER SKY should be a success with audiences. Although it doesn't have the appeal of current teen flicks like VARSITY BLUES or SHE'S ALL THAT, it certainly holds an undeniable quality that older crowds will lean towards. It's a welcome change to the dramatic genre, and one that everyone should see.
all contents © 1999 Craig Roush
-- Craig Roush<EMAIL_ADDRESS>-- Kinnopio's Movie Reviews http://www.execpc.com/~kinnopio
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


[1]
bangkok we always stay on silom road as we find it really convenient for everything .
[2]
pretty huge -/: a big living area with study area in one corner .


@(farmerForm: Form[views.forms.EditFarmerData], isLoggedIn: Boolean, farmer: Farmer)
@import views.html.bootstrap3._
@import views.html.helper._
@Main("Edit Farmer", isLoggedIn, farmer) {
@form(routes.Application.postEditFarmer(), 'class -> "form-horizontal") {
@text(farmerForm("name"),
label = "Name",
placeholder = "Name",
help = "")
@password(farmerForm("password"),
label = "Password",
placeholder = "Password",
help = "")
@text(farmerForm("location"),
label = "Farm Location",
placeholder = "Location",
help = "")
@text(farmerForm("markets"),
label = "Markets",
placeholder = "Markets",
help = "")
@text(farmerForm("phone"),
label = "Phone Number",
placeholder = "Phone",
help = "")
}
}


1-844-363-2663
A revolving fund that pays bond for people charged with crimes in Cook County.
773-309-1198
Criminal defense for activists


@(name: String, pin: String)
|
|
|
|
|
Welcome!
Hey @name and welcome to BroShout.
BroShout is a simple service that aims for you and your friends to have more fun together.
To get started, please enter this one time PIN number in BroShout
@pin
We hope you will enjoy many excellent adventures with your friends.
Again thanks!
The BroTeam
|
|
|
|
|