Data Providers
Data providers are wrappers that load external data, be it images, text, or general tensors, and split it into mini-batches so that the model can consume the data in a uniformed way.
AbstractDataProvider interface
#
MXNet.mx.AbstractDataProvider
— Type.
AbstractDataProvider
The root type for all data provider. A data provider should implement the following interfaces:
As well as the Julia iterator interface (see the Julia manual). Normally this involves defining:
Base.eltype(provider) -> AbstractDataBatch
Base.iterate(provider[, state]) -> (AbstractDataBatch, AbstractDataProvider)
The difference between data and label is that during training stage, both data and label will be feeded into the model, while during prediction stage, only data is loaded. Otherwise, they could be anything, with any names, and of any shapes. The provided data and label names here should match the input names in a target SymbolicNode
.
A data provider should also implement the Julia iteration interface, in order to allow iterating through the data set. The provider will be called in the following way:
for batch in eachbatch(provider)
data = get_data(provider, batch)
end
which will be translated by Julia compiler into
state = Base.start(eachbatch(provider))
while !Base.done(provider, state)
(batch, state) = Base.next(provider, state)
data = get_data(provider, batch)
end
By default, eachbatch
simply returns the provider itself, so the iterator interface is implemented on the provider type itself. But the extra layer of abstraction allows us to implement a data provider easily via a Julia Task
coroutine. See the data provider defined in the char-lstm example for an example of using coroutine to define data providers.
The detailed interface functions for the iterator API is listed below:
Base.eltype(provider) -> AbstractDataBatch
Returns the specific subtype representing a data batch. See AbstractDataBatch
.
-
provider::AbstractDataProvider
: the data provider.Base.start(provider) -> AbstractDataProviderState
This function is always called before iterating into the dataset. It should initialize the iterator, reset the index, and do data shuffling if needed.
-
provider::AbstractDataProvider
: the data provider.Base.done(provider, state) -> Bool
True if there is no more data to iterate in this dataset.
provider::AbstractDataProvider
: the data provider.-
state::AbstractDataProviderState
: the state returned byBase.start
andBase.next
.Base.next(provider) -> (AbstractDataBatch, AbstractDataProviderState)
Returns the current data batch, and the state for the next iteration.
provider::AbstractDataProvider
: the data provider.
Note sometimes you are wrapping an existing data iterator (e.g. the built-in libmxnet data iterator) that is built with a different convention. It might be difficult to adapt to the interfaces stated here. In this case, you can safely assume that
Base.start
will always be called, and called only once before the iteration starts.Base.done
will always be called at the beginning of every iteration and always be called once.- If
Base.done
return true, the iteration will stop, until the next round, again, starting with a call toBase.start
. Base.next
will always be called only once in each iteration. It will always be called after one and only one call toBase.done
; but ifBase.done
returns true,Base.next
will not be called.
With those assumptions, it will be relatively easy to adapt any existing iterator. See the implementation of the built-in MXDataProvider
for example.
Note
Please do not use the one data provider simultaneously in two different places, either in parallel, or in a nested loop. For example, the behavior for the following code is undefined
```julia for batch in data # updating the parameters
# now let's test the performance on the training set
for b2 in data
# ...
end
end ```
#
MXNet.mx.get_batch_size
— Function.
get_batch_size(provider) -> Int
Arguments:
provider::AbstractDataProvider
: the data provider.
Returns the mini-batch size of the provided data. All the provided data should have the same mini-batch size (i.e. the last dimension).
#
MXNet.mx.provide_data
— Function.
provide_data(provider) -> Vector{Tuple{Base.Symbol, Tuple}}
Arguments:
provider::AbstractDataProvider
: the data provider.
Returns a vector of (name, shape) pairs describing the names of the data it provides, and the corresponding shapes.
#
MXNet.mx.provide_label
— Function.
provide_label(provider) -> Vector{Tuple{Base.Symbol, Tuple}}
Arguments:
provider::AbstractDataProvider
: the data provider.
Returns a vector of (name, shape) pairs describing the names of the labels it provides, and the corresponding shapes.
AbstractDataBatch interface
#
MXNet.mx.AbstractDataProviderState
— Type.
AbstractDataProviderState
Base type for data provider states.
#
MXNet.mx.count_samples
— Function.
count_samples(provider, batch) -> Int
Arguments:
batch::AbstractDataBatch
: the data batch object.
Returns the number of samples in this batch. This number should be greater than 0, but less than or equal to the batch size. This is used to indicate at the end of the data set, there might not be enough samples for a whole mini-batch.
#
MXNet.mx.get_data
— Function.
get_data(provider, batch) -> Vector{NDArray}
Arguments:
provider::AbstractDataProvider
: the data provider.batch::AbstractDataBatch
: the data batch object.
Returns a vector of data in this batch, should be in the same order as declared in provide_data() <AbstractDataProvider.provide_data>
.
The last dimension of each NDArray
should always match the batch*size, even when count*samples
returns a value less than the batch size. In this case, the data provider is free to pad the remaining contents with any value.
#
MXNet.mx.get_label
— Function.
get_label(provider, batch) -> Vector{NDArray}
Arguments:
provider::AbstractDataProvider
: the data provider.batch::AbstractDataBatch
: the data batch object.
Returns a vector of labels in this batch. Similar to get_data
.
#
Base.get
— Function.
get(sched)
sched::AbstractMomentumScheduler
: the momentum scheduler.
Returns the current momentum.
#
MXNet.mx.load_data!
— Function.
load_data!(provider, batch, targets)
Arguments:
provider::AbstractDataProvider
: the data provider.batch::AbstractDataBatch
: the data batch object.targets::Vector{Vector{SlicedNDArray}}
: the targets to load data into.
The targets is a list of the same length as number of data provided by this provider. Each element in the list is a list of SlicedNDArray
. This list described a spliting scheme of this data batch into different slices, each slice is specified by a slice-ndarray pair, where slice specify the range of samples in the mini-batch that should be loaded into the corresponding ndarray.
This utility function is used in data parallelization, where a mini-batch is splited and computed on several different devices.
#
MXNet.mx.load_label!
— Function.
load_label!(provider, batch, targets)
provider::AbstractDataProvider provider
: the data provider.batch::AbstractDataBatch batch
: the data batch object.targets::Vector{Vector{SlicedNDArray}}
: the targets to load label into.
The same as load_data!
, except that this is for loading labels.
Implemented providers and other methods
#
MXNet.mx.AbstractDataBatch
— Type.
AbstractDataBatch
Base type for a data mini-batch. It should implement the following interfaces:
The following utility functions will be automatically defined:
#
MXNet.mx.ArrayDataProvider
— Type.
ArrayDataProvider
A convenient tool to iterate NDArray
or Julia Array
.
ArrayDataProvider(data[, label]; batch_size, shuffle, data_padding, label_padding)
Construct a data provider from NDArray
or Julia Arrays.
Arguments:
-
data
: the data, could be- a
NDArray
, or a Julia Array. This is equivalent to:data => data
. - a name-data pair, like
:mydata => array
, where:mydata
is the name of the data - and
array
is anNDArray
or a Julia Array. - a list of name-data pairs.
label
: the same as thedata
parameter. When this argument is omitted, the constructed provider will provide no labels.batch_size::Int
: the batch size, default is 0, which means treating the whole array as a single mini-batch.shuffle::Bool
: turn on if the data should be shuffled at every epoch.data_padding::Real
: when the mini-batch goes beyond the dataset boundary, there might be less samples to include than a mini-batch. This value specify a scalar to pad the contents of all the missing data points.label_padding::Real
: the same asdata_padding
, except for the labels.
- a
TODO: remove data_padding
and label_padding
, and implement rollover that copies the last or first several training samples to feed the padding.
#
MXNet.mx.DataBatch
— Type.
DataBatch
A basic subclass of AbstractDataBatch
, that implement the interface by accessing member fields.
#
MXNet.mx.MXDataProvider
— Type.
MXDataProvider
A data provider that wrap built-in data iterators from libmxnet. See below for a list of built-in data iterators.
#
MXNet.mx.SlicedNDArray
— Type.
SlicedNDArray
A alias type of Tuple{UnitRange{Int},NDArray}
.
#
Base.get
— Method.
get(provider, batch, name) -> NDArray
provider::AbstractDataProvider
: the data provider.batch::AbstractDataBatch
: the data batch object.name::Symbol
: the name of the data to get, should be one of the names provided in eitherprovide_data() <AbstractDataProvider.provide_data>
orprovide_label() <AbstractDataProvider.provide_label>
.
Returns the corresponding data array corresponding to that name.
#
MXNet.mx.CSVIter
— Method.
CSVIter(data_csv, data_shape, label_csv, label_shape, batch_size, round_batch, prefetch_buffer, ctx, dtype)
Can also be called with the alias CSVProvider
. Returns the CSV file iterator.
In this function, the data_shape
parameter is used to set the shape of each line of the input data. If a row in an input file is 1,2,3,4,5,6``and
data_shape` is (3,2), that row will be reshaped, yielding the array [[1,2],[3,4],[5,6]] of shape (3,2).
By default, the CSVIter
has round_batch
parameter set to $True$. So, if batch_size
is 3 and there are 4 total rows in CSV file, 2 more examples are consumed at the first round. If reset
function is called after first round, the call is ignored and remaining examples are returned in the second round.
If one wants all the instances in the second round after calling reset
, make sure to set round_batch
to False.
If $data_csv = 'data/'$ is set, then all the files in this directory will be read.
$reset()$ is expected to be called only after a complete pass of data.
By default, the CSVIter parses all entries in the data file as float32 data type, if dtype
argument is set to be 'int32' or 'int64' then CSVIter will parse all entries in the file as int32 or int64 data type accordingly.
Examples::
// Contents of CSV file $data/data.csv$. 1,2,3 2,3,4 3,4,5 4,5,6
// Creates a CSVIter
with batch_size
=2 and default round_batch
=True. CSVIter = mx.io.CSVIter(datacsv = 'data/data.csv', datashape = (3,), batch_size = 2)
// Two batches read from the above iterator are as follows: [[ 1. 2. 3.] [ 2. 3. 4.]] [[ 3. 4. 5.] [ 4. 5. 6.]]
// Creates a CSVIter
with default round_batch
set to True. CSVIter = mx.io.CSVIter(datacsv = 'data/data.csv', datashape = (3,), batch_size = 3)
// Two batches read from the above iterator in the first pass are as follows: [[1. 2. 3.] [2. 3. 4.] [3. 4. 5.]]
[[4. 5. 6.] [1. 2. 3.] [2. 3. 4.]]
// Now, reset
method is called. CSVIter.reset()
// Batch read from the above iterator in the second pass is as follows: [[ 3. 4. 5.] [ 4. 5. 6.] [ 1. 2. 3.]]
// Creates a CSVIter
with round_batch
=False. CSVIter = mx.io.CSVIter(datacsv = 'data/data.csv', datashape = (3,), batchsize = 3, roundbatch=False)
// Contents of two batches read from the above iterator in both passes, after calling // reset
method before second pass, is as follows: [[1. 2. 3.] [2. 3. 4.] [3. 4. 5.]]
[[4. 5. 6.] [2. 3. 4.] [3. 4. 5.]]
// Creates a 'CSVIter' with dtype
='int32' CSVIter = mx.io.CSVIter(datacsv = 'data/data.csv', datashape = (3,), batchsize = 3, roundbatch=False, dtype='int32')
// Contents of two batches read from the above iterator in both passes, after calling // reset
method before second pass, is as follows: [[1 2 3] [2 3 4] [3 4 5]]
[[4 5 6] [2 3 4] [3 4 5]]
Defined in src/io/iter_csv.cc:L308
Arguments:
data_name::Symbol
: keyword argument, default:data
. The name of the data.label_name::Symbol
: keyword argument, default:softmax_label
. The name of the label. Could benothing
if no label is presented in this dataset.data_csv::string, required
: The input CSV file or a directory path.data_shape::Shape(tuple), required
: The shape of one example.label_csv::string, optional, default='NULL'
: The input CSV file or a directory path. If NULL, all labels will be returned as 0.label_shape::Shape(tuple), optional, default=[1]
: The shape of one label.batch_size::int (non-negative), required
: Batch size.round_batch::boolean, optional, default=1
: Whether to use round robin to handle overflow batch or not.prefetch_buffer::long (non-negative), optional, default=4
: Maximum number of batches to prefetch.ctx::{'cpu', 'gpu'},optional, default='gpu'
: Context data loader optimized for.dtype::{None, 'float16', 'float32', 'float64', 'int32', 'int64', 'int8', 'uint8'},optional, default='None'
: Output data type. $None$ means no change.
Returns the constructed MXDataProvider
.
#
MXNet.mx.ImageDetRecordIter
— Method.
ImageDetRecordIter(path_imglist, path_imgrec, aug_seq, label_width, data_shape, preprocess_threads, verbose, num_parts, part_index, shuffle_chunk_size, shuffle_chunk_seed, label_pad_width, label_pad_value, shuffle, seed, verbose, batch_size, round_batch, prefetch_buffer, ctx, dtype, resize, rand_crop_prob, min_crop_scales, max_crop_scales, min_crop_aspect_ratios, max_crop_aspect_ratios, min_crop_overlaps, max_crop_overlaps, min_crop_sample_coverages, max_crop_sample_coverages, min_crop_object_coverages, max_crop_object_coverages, num_crop_sampler, crop_emit_mode, emit_overlap_thresh, max_crop_trials, rand_pad_prob, max_pad_scale, max_random_hue, random_hue_prob, max_random_saturation, random_saturation_prob, max_random_illumination, random_illumination_prob, max_random_contrast, random_contrast_prob, rand_mirror_prob, fill_value, inter_method, data_shape, resize_mode, seed, mean_img, mean_r, mean_g, mean_b, mean_a, std_r, std_g, std_b, std_a, scale, verbose)
Can also be called with the alias ImageDetRecordProvider
. Create iterator for image detection dataset packed in recordio.
Arguments:
data_name::Symbol
: keyword argument, default:data
. The name of the data.label_name::Symbol
: keyword argument, default:softmax_label
. The name of the label. Could benothing
if no label is presented in this dataset.path_imglist::string, optional, default=''
: Dataset Param: Path to image list.path_imgrec::string, optional, default='./data/imgrec.rec'
: Dataset Param: Path to image record file.aug_seq::string, optional, default='det_aug_default'
: Augmentation Param: the augmenter names to represent sequence of augmenters to be applied, seperated by comma. Additional keyword parameters will be seen by these augmenters. Make sure you don't use normal augmenters for detection tasks.label_width::int, optional, default='-1'
: Dataset Param: How many labels for an image, -1 for variable label size.data_shape::Shape(tuple), required
: Dataset Param: Shape of each instance generated by the DataIter.preprocess_threads::int, optional, default='4'
: Backend Param: Number of thread to do preprocessing.verbose::boolean, optional, default=1
: Auxiliary Param: Whether to output parser information.num_parts::int, optional, default='1'
: partition the data into multiple partspart_index::int, optional, default='0'
: the index of the part will readshuffle_chunk_size::long (non-negative), optional, default=0
: the size(MB) of the shuffle chunk, used with shuffle=True, it can enable global shufflingshuffle_chunk_seed::int, optional, default='0'
: the seed for chunk shufflinglabel_pad_width::int, optional, default='0'
: pad output label width if set larger than 0, -1 for auto estimatelabel_pad_value::float, optional, default=-1
: label padding value if enabledshuffle::boolean, optional, default=0
: Augmentation Param: Whether to shuffle data.seed::int, optional, default='0'
: Augmentation Param: Random Seed.batch_size::int (non-negative), required
: Batch size.round_batch::boolean, optional, default=1
: Whether to use round robin to handle overflow batch or not.prefetch_buffer::long (non-negative), optional, default=4
: Maximum number of batches to prefetch.ctx::{'cpu', 'gpu'},optional, default='gpu'
: Context data loader optimized for.dtype::{None, 'float16', 'float32', 'float64', 'int32', 'int64', 'int8', 'uint8'},optional, default='None'
: Output data type. $None$ means no change.resize::int, optional, default='-1'
: Augmentation Param: scale shorter edge to size before applying other augmentations, -1 to disable.rand_crop_prob::float, optional, default=0
: Augmentation Param: Probability of random cropping, <= 0 to disablemin_crop_scales::tuple of <float>, optional, default=[0]
: Augmentation Param: Min crop scales.max_crop_scales::tuple of <float>, optional, default=[1]
: Augmentation Param: Max crop scales.min_crop_aspect_ratios::tuple of <float>, optional, default=[1]
: Augmentation Param: Min crop aspect ratios.max_crop_aspect_ratios::tuple of <float>, optional, default=[1]
: Augmentation Param: Max crop aspect ratios.min_crop_overlaps::tuple of <float>, optional, default=[0]
: Augmentation Param: Minimum crop IOU between crop_box and ground-truths.max_crop_overlaps::tuple of <float>, optional, default=[1]
: Augmentation Param: Maximum crop IOU between crop_box and ground-truth.min_crop_sample_coverages::tuple of <float>, optional, default=[0]
: Augmentation Param: Minimum ratio of intersect/crop_area between crop box and ground-truths.max_crop_sample_coverages::tuple of <float>, optional, default=[1]
: Augmentation Param: Maximum ratio of intersect/crop_area between crop box and ground-truths.min_crop_object_coverages::tuple of <float>, optional, default=[0]
: Augmentation Param: Minimum ratio of intersect/gt_area between crop box and ground-truths.max_crop_object_coverages::tuple of <float>, optional, default=[1]
: Augmentation Param: Maximum ratio of intersect/gt_area between crop box and ground-truths.num_crop_sampler::int, optional, default='1'
: Augmentation Param: Number of crop samplers.crop_emit_mode::{'center', 'overlap'},optional, default='center'
: Augmentation Param: Emition mode for invalid ground-truths after crop. center: emit if centroid of object is out of crop region; overlap: emit if overlap is less than emitoverlapthresh.emit_overlap_thresh::float, optional, default=0.300000012
: Augmentation Param: Emit overlap thresh for emit mode overlap only.max_crop_trials::Shape(tuple), optional, default=[25]
: Augmentation Param: Skip cropping if fail crop trail count exceeds this number.rand_pad_prob::float, optional, default=0
: Augmentation Param: Probability for random padding.max_pad_scale::float, optional, default=1
: Augmentation Param: Maximum padding scale.max_random_hue::int, optional, default='0'
: Augmentation Param: Maximum random value of H channel in HSL color space.random_hue_prob::float, optional, default=0
: Augmentation Param: Probability to apply random hue.max_random_saturation::int, optional, default='0'
: Augmentation Param: Maximum random value of S channel in HSL color space.random_saturation_prob::float, optional, default=0
: Augmentation Param: Probability to apply random saturation.max_random_illumination::int, optional, default='0'
: Augmentation Param: Maximum random value of L channel in HSL color space.random_illumination_prob::float, optional, default=0
: Augmentation Param: Probability to apply random illumination.max_random_contrast::float, optional, default=0
: Augmentation Param: Maximum random value of delta contrast.random_contrast_prob::float, optional, default=0
: Augmentation Param: Probability to apply random contrast.rand_mirror_prob::float, optional, default=0
: Augmentation Param: Probability to apply horizontal flip aka. mirror.fill_value::int, optional, default='127'
: Augmentation Param: Filled color value while padding.inter_method::int, optional, default='1'
: Augmentation Param: 0-NN 1-bilinear 2-cubic 3-area 4-lanczos4 9-auto 10-rand.resize_mode::{'fit', 'force', 'shrink'},optional, default='force'
: Augmentation Param: How image data fit in datashape. force: force reshape to datashape regardless of aspect ratio; shrink: ensure each side fit in datashape, preserve aspect ratio; fit: fit image to datashape, preserve ratio, will upscale if applicable.mean_img::string, optional, default=''
: Augmentation Param: Mean Image to be subtracted.mean_r::float, optional, default=0
: Augmentation Param: Mean value on R channel.mean_g::float, optional, default=0
: Augmentation Param: Mean value on G channel.mean_b::float, optional, default=0
: Augmentation Param: Mean value on B channel.mean_a::float, optional, default=0
: Augmentation Param: Mean value on Alpha channel.std_r::float, optional, default=0
: Augmentation Param: Standard deviation on R channel.std_g::float, optional, default=0
: Augmentation Param: Standard deviation on G channel.std_b::float, optional, default=0
: Augmentation Param: Standard deviation on B channel.std_a::float, optional, default=0
: Augmentation Param: Standard deviation on Alpha channel.scale::float, optional, default=1
: Augmentation Param: Scale in color space.
Returns the constructed MXDataProvider
.
#
MXNet.mx.ImageRecordInt8Iter
— Method.
ImageRecordInt8Iter(path_imglist, path_imgrec, path_imgidx, aug_seq, label_width, data_shape, preprocess_threads, verbose, num_parts, part_index, device_id, shuffle_chunk_size, shuffle_chunk_seed, seed_aug, shuffle, seed, verbose, batch_size, round_batch, prefetch_buffer, ctx, dtype, resize, rand_crop, random_resized_crop, max_rotate_angle, max_aspect_ratio, min_aspect_ratio, max_shear_ratio, max_crop_size, min_crop_size, max_random_scale, min_random_scale, max_random_area, min_random_area, max_img_size, min_img_size, brightness, contrast, saturation, pca_noise, random_h, random_s, random_l, rotate, fill_value, data_shape, inter_method, pad)
Can also be called with the alias ImageRecordInt8Provider
. Iterating on image RecordIO files
.. note:: $ImageRecordInt8Iter$ is deprecated. Use ImageRecordIter(dtype='int8') instead.
This iterator is identical to $ImageRecordIter$ except for using $int8$ as the data type instead of $float$.
Defined in src/io/iterimagerecordio_2.cc:L934
Arguments:
data_name::Symbol
: keyword argument, default:data
. The name of the data.label_name::Symbol
: keyword argument, default:softmax_label
. The name of the label. Could benothing
if no label is presented in this dataset.path_imglist::string, optional, default=''
: Path to the image list (.lst) file. Generally created with tools/im2rec.py. Format (Tab separated):. path_imgrec::string, optional, default=''
: Path to the image RecordIO (.rec) file or a directory path. Created with tools/im2rec.py.path_imgidx::string, optional, default=''
: Path to the image RecordIO index (.idx) file. Created with tools/im2rec.py.aug_seq::string, optional, default='aug_default'
: The augmenter names to represent sequence of augmenters to be applied, seperated by comma. Additional keyword parameters will be seen by these augmenters.label_width::int, optional, default='1'
: The number of labels per image.data_shape::Shape(tuple), required
: The shape of one output image in (channels, height, width) format.preprocess_threads::int, optional, default='4'
: The number of threads to do preprocessing.verbose::boolean, optional, default=1
: If or not output verbose information.num_parts::int, optional, default='1'
: Virtually partition the data into these many parts.part_index::int, optional, default='0'
: The i-th virtual partition to be read.device_id::int, optional, default='0'
: The device id used to create context for internal NDArray. Setting deviceid to -1 will create Context::CPU(0). Setting deviceid to valid positive device id will create Context::CPUPinned(device_id). Default is 0.shuffle_chunk_size::long (non-negative), optional, default=0
: The data shuffle buffer size in MB. Only valid if shuffle is true.shuffle_chunk_seed::int, optional, default='0'
: The random seed for shufflingseed_aug::int or None, optional, default='None'
: Random seed for augmentations.shuffle::boolean, optional, default=0
: Whether to shuffle data randomly or not.seed::int, optional, default='0'
: The random seed.batch_size::int (non-negative), required
: Batch size.round_batch::boolean, optional, default=1
: Whether to use round robin to handle overflow batch or not.prefetch_buffer::long (non-negative), optional, default=4
: Maximum number of batches to prefetch.ctx::{'cpu', 'gpu'},optional, default='gpu'
: Context data loader optimized for.dtype::{None, 'float16', 'float32', 'float64', 'int32', 'int64', 'int8', 'uint8'},optional, default='None'
: Output data type. $None$ means no change.resize::int, optional, default='-1'
: Down scale the shorter edge to a new size before applying other augmentations.rand_crop::boolean, optional, default=0
: If or not randomly crop the imagerandom_resized_crop::boolean, optional, default=0
: If or not perform random resized cropping on the image, as a standard preprocessing for resnet training on ImageNet data.max_rotate_angle::int, optional, default='0'
: Rotate by a random degree in $[-v, v]$max_aspect_ratio::float, optional, default=0
: Change the aspect (namely width/height) to a random value. If minaspectratio is None then the aspect ratio ins sampled from [1 - maxaspectratio, 1 + maxaspectratio], else it is in $[min_aspect_ratio, max_aspect_ratio]$min_aspect_ratio::float or None, optional, default=None
: Change the aspect (namely width/height) to a random value in $[min_aspect_ratio, max_aspect_ratio]$max_shear_ratio::float, optional, default=0
: Apply a shear transformation (namely $(x,y)->(x+my,y)$) with $m$ randomly chose from $[-max_shear_ratio, max_shear_ratio]$max_crop_size::int, optional, default='-1'
: Crop both width and height into a random size in $[min_crop_size, max_crop_size].$Ignored if $random_resized_crop$ is True.min_crop_size::int, optional, default='-1'
: Crop both width and height into a random size in $[min_crop_size, max_crop_size].$Ignored if $random_resized_crop$ is True.max_random_scale::float, optional, default=1
: Resize into $[widths, heights]$ with $s$ randomly chosen from $[min_random_scale, max_random_scale]$. Ignored if $random_resized_crop$ is True.min_random_scale::float, optional, default=1
: Resize into $[widths, heights]$ with $s$ randomly chosen from $[min_random_scale, max_random_scale]$Ignored if $random_resized_crop$ is True.max_random_area::float, optional, default=1
: Change the area (namely width * height) to a random value in $[min_random_area, max_random_area]$. Ignored if $random_resized_crop$ is False.min_random_area::float, optional, default=1
: Change the area (namely width * height) to a random value in $[min_random_area, max_random_area]$. Ignored if $random_resized_crop$ is False.max_img_size::float, optional, default=1e+10
: Set the maximal width and height after all resize and rotate argumentation are appliedmin_img_size::float, optional, default=0
: Set the minimal width and height after all resize and rotate argumentation are appliedbrightness::float, optional, default=0
: Add a random value in $[-brightness, brightness]$ to the brightness of image.contrast::float, optional, default=0
: Add a random value in $[-contrast, contrast]$ to the contrast of image.saturation::float, optional, default=0
: Add a random value in $[-saturation, saturation]$ to the saturation of image.pca_noise::float, optional, default=0
: Add PCA based noise to the image.random_h::int, optional, default='0'
: Add a random value in $[-random_h, random_h]$ to the H channel in HSL color space.random_s::int, optional, default='0'
: Add a random value in $[-random_s, random_s]$ to the S channel in HSL color space.random_l::int, optional, default='0'
: Add a random value in $[-random_l, random_l]$ to the L channel in HSL color space.rotate::int, optional, default='-1'
: Rotate by an angle. If set, it overwrites the $max_rotate_angle$ option.fill_value::int, optional, default='255'
: Set the padding pixels value to $fill_value$.inter_method::int, optional, default='1'
: The interpolation method: 0-NN 1-bilinear 2-cubic 3-area 4-lanczos4 9-auto 10-rand.pad::int, optional, default='0'
: Change size from $[width, height]$ into $[pad + width + pad, pad + height + pad]$ by padding pixes
Returns the constructed MXDataProvider
.
#
MXNet.mx.ImageRecordIter
— Method.
ImageRecordIter(path_imglist, path_imgrec, path_imgidx, aug_seq, label_width, data_shape, preprocess_threads, verbose, num_parts, part_index, device_id, shuffle_chunk_size, shuffle_chunk_seed, seed_aug, shuffle, seed, verbose, batch_size, round_batch, prefetch_buffer, ctx, dtype, resize, rand_crop, random_resized_crop, max_rotate_angle, max_aspect_ratio, min_aspect_ratio, max_shear_ratio, max_crop_size, min_crop_size, max_random_scale, min_random_scale, max_random_area, min_random_area, max_img_size, min_img_size, brightness, contrast, saturation, pca_noise, random_h, random_s, random_l, rotate, fill_value, data_shape, inter_method, pad, seed, mirror, rand_mirror, mean_img, mean_r, mean_g, mean_b, mean_a, std_r, std_g, std_b, std_a, scale, max_random_contrast, max_random_illumination, verbose)
Can also be called with the alias ImageRecordProvider
. Iterates on image RecordIO files
Reads batches of images from .rec RecordIO files. One can use $im2rec.py$ tool (in tools/) to pack raw image files into RecordIO files. This iterator is less flexible to customization but is fast and has lot of language bindings. To iterate over raw images directly use $ImageIter$ instead (in Python).
Example::
dataiter = mx.io.ImageRecordIter( pathimgrec="./sample.rec", # The target record file. datashape=(3, 227, 227), # Output data shape; 227x227 region will be cropped from the original image. batchsize=4, # Number of items per batch. resize=256 # Resize the shorter edge to 256 before cropping. # You can specify more augmentation options. Use help(mx.io.ImageRecordIter) to see all the options. )
You can now use the data_iter to access batches of images.
batch = dataiter.next() # first batch. images = batch.data[0] # This will contain 4 (=batchsize) images each of 3x227x227.
process the images
... data_iter.reset() # To restart the iterator from the beginning.
Defined in src/io/iterimagerecordio_2.cc:L897
Arguments:
data_name::Symbol
: keyword argument, default:data
. The name of the data.label_name::Symbol
: keyword argument, default:softmax_label
. The name of the label. Could benothing
if no label is presented in this dataset.path_imglist::string, optional, default=''
: Path to the image list (.lst) file. Generally created with tools/im2rec.py. Format (Tab separated):. path_imgrec::string, optional, default=''
: Path to the image RecordIO (.rec) file or a directory path. Created with tools/im2rec.py.path_imgidx::string, optional, default=''
: Path to the image RecordIO index (.idx) file. Created with tools/im2rec.py.aug_seq::string, optional, default='aug_default'
: The augmenter names to represent sequence of augmenters to be applied, seperated by comma. Additional keyword parameters will be seen by these augmenters.label_width::int, optional, default='1'
: The number of labels per image.data_shape::Shape(tuple), required
: The shape of one output image in (channels, height, width) format.preprocess_threads::int, optional, default='4'
: The number of threads to do preprocessing.verbose::boolean, optional, default=1
: If or not output verbose information.num_parts::int, optional, default='1'
: Virtually partition the data into these many parts.part_index::int, optional, default='0'
: The i-th virtual partition to be read.device_id::int, optional, default='0'
: The device id used to create context for internal NDArray. Setting deviceid to -1 will create Context::CPU(0). Setting deviceid to valid positive device id will create Context::CPUPinned(device_id). Default is 0.shuffle_chunk_size::long (non-negative), optional, default=0
: The data shuffle buffer size in MB. Only valid if shuffle is true.shuffle_chunk_seed::int, optional, default='0'
: The random seed for shufflingseed_aug::int or None, optional, default='None'
: Random seed for augmentations.shuffle::boolean, optional, default=0
: Whether to shuffle data randomly or not.seed::int, optional, default='0'
: The random seed.batch_size::int (non-negative), required
: Batch size.round_batch::boolean, optional, default=1
: Whether to use round robin to handle overflow batch or not.prefetch_buffer::long (non-negative), optional, default=4
: Maximum number of batches to prefetch.ctx::{'cpu', 'gpu'},optional, default='gpu'
: Context data loader optimized for.dtype::{None, 'float16', 'float32', 'float64', 'int32', 'int64', 'int8', 'uint8'},optional, default='None'
: Output data type. $None$ means no change.resize::int, optional, default='-1'
: Down scale the shorter edge to a new size before applying other augmentations.rand_crop::boolean, optional, default=0
: If or not randomly crop the imagerandom_resized_crop::boolean, optional, default=0
: If or not perform random resized cropping on the image, as a standard preprocessing for resnet training on ImageNet data.max_rotate_angle::int, optional, default='0'
: Rotate by a random degree in $[-v, v]$max_aspect_ratio::float, optional, default=0
: Change the aspect (namely width/height) to a random value. If minaspectratio is None then the aspect ratio ins sampled from [1 - maxaspectratio, 1 + maxaspectratio], else it is in $[min_aspect_ratio, max_aspect_ratio]$min_aspect_ratio::float or None, optional, default=None
: Change the aspect (namely width/height) to a random value in $[min_aspect_ratio, max_aspect_ratio]$max_shear_ratio::float, optional, default=0
: Apply a shear transformation (namely $(x,y)->(x+my,y)$) with $m$ randomly chose from $[-max_shear_ratio, max_shear_ratio]$max_crop_size::int, optional, default='-1'
: Crop both width and height into a random size in $[min_crop_size, max_crop_size].$Ignored if $random_resized_crop$ is True.min_crop_size::int, optional, default='-1'
: Crop both width and height into a random size in $[min_crop_size, max_crop_size].$Ignored if $random_resized_crop$ is True.max_random_scale::float, optional, default=1
: Resize into $[widths, heights]$ with $s$ randomly chosen from $[min_random_scale, max_random_scale]$. Ignored if $random_resized_crop$ is True.min_random_scale::float, optional, default=1
: Resize into $[widths, heights]$ with $s$ randomly chosen from $[min_random_scale, max_random_scale]$Ignored if $random_resized_crop$ is True.max_random_area::float, optional, default=1
: Change the area (namely width * height) to a random value in $[min_random_area, max_random_area]$. Ignored if $random_resized_crop$ is False.min_random_area::float, optional, default=1
: Change the area (namely width * height) to a random value in $[min_random_area, max_random_area]$. Ignored if $random_resized_crop$ is False.max_img_size::float, optional, default=1e+10
: Set the maximal width and height after all resize and rotate argumentation are appliedmin_img_size::float, optional, default=0
: Set the minimal width and height after all resize and rotate argumentation are appliedbrightness::float, optional, default=0
: Add a random value in $[-brightness, brightness]$ to the brightness of image.contrast::float, optional, default=0
: Add a random value in $[-contrast, contrast]$ to the contrast of image.saturation::float, optional, default=0
: Add a random value in $[-saturation, saturation]$ to the saturation of image.pca_noise::float, optional, default=0
: Add PCA based noise to the image.random_h::int, optional, default='0'
: Add a random value in $[-random_h, random_h]$ to the H channel in HSL color space.random_s::int, optional, default='0'
: Add a random value in $[-random_s, random_s]$ to the S channel in HSL color space.random_l::int, optional, default='0'
: Add a random value in $[-random_l, random_l]$ to the L channel in HSL color space.rotate::int, optional, default='-1'
: Rotate by an angle. If set, it overwrites the $max_rotate_angle$ option.fill_value::int, optional, default='255'
: Set the padding pixels value to $fill_value$.inter_method::int, optional, default='1'
: The interpolation method: 0-NN 1-bilinear 2-cubic 3-area 4-lanczos4 9-auto 10-rand.pad::int, optional, default='0'
: Change size from $[width, height]$ into $[pad + width + pad, pad + height + pad]$ by padding pixesmirror::boolean, optional, default=0
: Whether to mirror the image or not. If true, images are flipped along the horizontal axis.rand_mirror::boolean, optional, default=0
: Whether to randomly mirror images or not. If true, 50% of the images will be randomly mirrored (flipped along the horizontal axis)mean_img::string, optional, default=''
: Filename of the mean image.mean_r::float, optional, default=0
: The mean value to be subtracted on the R channelmean_g::float, optional, default=0
: The mean value to be subtracted on the G channelmean_b::float, optional, default=0
: The mean value to be subtracted on the B channelmean_a::float, optional, default=0
: The mean value to be subtracted on the alpha channelstd_r::float, optional, default=1
: Augmentation Param: Standard deviation on R channel.std_g::float, optional, default=1
: Augmentation Param: Standard deviation on G channel.std_b::float, optional, default=1
: Augmentation Param: Standard deviation on B channel.std_a::float, optional, default=1
: Augmentation Param: Standard deviation on Alpha channel.scale::float, optional, default=1
: Multiply the image with a scale value.max_random_contrast::float, optional, default=0
: Change the contrast with a value randomly chosen from $[-max_random_contrast, max_random_contrast]$max_random_illumination::float, optional, default=0
: Change the illumination with a value randomly chosen from $[-max_random_illumination, max_random_illumination]$
Returns the constructed MXDataProvider
.
#
MXNet.mx.ImageRecordIter_v1
— Method.
ImageRecordIter_v1(path_imglist, path_imgrec, path_imgidx, aug_seq, label_width, data_shape, preprocess_threads, verbose, num_parts, part_index, device_id, shuffle_chunk_size, shuffle_chunk_seed, seed_aug, shuffle, seed, verbose, batch_size, round_batch, prefetch_buffer, ctx, dtype, resize, rand_crop, random_resized_crop, max_rotate_angle, max_aspect_ratio, min_aspect_ratio, max_shear_ratio, max_crop_size, min_crop_size, max_random_scale, min_random_scale, max_random_area, min_random_area, max_img_size, min_img_size, brightness, contrast, saturation, pca_noise, random_h, random_s, random_l, rotate, fill_value, data_shape, inter_method, pad, seed, mirror, rand_mirror, mean_img, mean_r, mean_g, mean_b, mean_a, std_r, std_g, std_b, std_a, scale, max_random_contrast, max_random_illumination, verbose)
Iterating on image RecordIO files
.. note::
$ImageRecordIter_v1$ is deprecated. Use $ImageRecordIter$ instead.
Read images batches from RecordIO files with a rich of data augmentation options.
One can use $tools/im2rec.py$ to pack individual image files into RecordIO files.
Defined in src/io/iterimagerecordio.cc:L352
Arguments:
data_name::Symbol
: keyword argument, default:data
. The name of the data.label_name::Symbol
: keyword argument, default:softmax_label
. The name of the label. Could benothing
if no label is presented in this dataset.path_imglist::string, optional, default=''
: Path to the image list (.lst) file. Generally created with tools/im2rec.py. Format (Tab separated):. path_imgrec::string, optional, default=''
: Path to the image RecordIO (.rec) file or a directory path. Created with tools/im2rec.py.path_imgidx::string, optional, default=''
: Path to the image RecordIO index (.idx) file. Created with tools/im2rec.py.aug_seq::string, optional, default='aug_default'
: The augmenter names to represent sequence of augmenters to be applied, seperated by comma. Additional keyword parameters will be seen by these augmenters.label_width::int, optional, default='1'
: The number of labels per image.data_shape::Shape(tuple), required
: The shape of one output image in (channels, height, width) format.preprocess_threads::int, optional, default='4'
: The number of threads to do preprocessing.verbose::boolean, optional, default=1
: If or not output verbose information.num_parts::int, optional, default='1'
: Virtually partition the data into these many parts.part_index::int, optional, default='0'
: The i-th virtual partition to be read.device_id::int, optional, default='0'
: The device id used to create context for internal NDArray. Setting deviceid to -1 will create Context::CPU(0). Setting deviceid to valid positive device id will create Context::CPUPinned(device_id). Default is 0.shuffle_chunk_size::long (non-negative), optional, default=0
: The data shuffle buffer size in MB. Only valid if shuffle is true.shuffle_chunk_seed::int, optional, default='0'
: The random seed for shufflingseed_aug::int or None, optional, default='None'
: Random seed for augmentations.shuffle::boolean, optional, default=0
: Whether to shuffle data randomly or not.seed::int, optional, default='0'
: The random seed.batch_size::int (non-negative), required
: Batch size.round_batch::boolean, optional, default=1
: Whether to use round robin to handle overflow batch or not.prefetch_buffer::long (non-negative), optional, default=4
: Maximum number of batches to prefetch.ctx::{'cpu', 'gpu'},optional, default='gpu'
: Context data loader optimized for.dtype::{None, 'float16', 'float32', 'float64', 'int32', 'int64', 'int8', 'uint8'},optional, default='None'
: Output data type. $None$ means no change.resize::int, optional, default='-1'
: Down scale the shorter edge to a new size before applying other augmentations.rand_crop::boolean, optional, default=0
: If or not randomly crop the imagerandom_resized_crop::boolean, optional, default=0
: If or not perform random resized cropping on the image, as a standard preprocessing for resnet training on ImageNet data.max_rotate_angle::int, optional, default='0'
: Rotate by a random degree in $[-v, v]$max_aspect_ratio::float, optional, default=0
: Change the aspect (namely width/height) to a random value. If minaspectratio is None then the aspect ratio ins sampled from [1 - maxaspectratio, 1 + maxaspectratio], else it is in $[min_aspect_ratio, max_aspect_ratio]$min_aspect_ratio::float or None, optional, default=None
: Change the aspect (namely width/height) to a random value in $[min_aspect_ratio, max_aspect_ratio]$max_shear_ratio::float, optional, default=0
: Apply a shear transformation (namely $(x,y)->(x+my,y)$) with $m$ randomly chose from $[-max_shear_ratio, max_shear_ratio]$max_crop_size::int, optional, default='-1'
: Crop both width and height into a random size in $[min_crop_size, max_crop_size].$Ignored if $random_resized_crop$ is True.min_crop_size::int, optional, default='-1'
: Crop both width and height into a random size in $[min_crop_size, max_crop_size].$Ignored if $random_resized_crop$ is True.max_random_scale::float, optional, default=1
: Resize into $[widths, heights]$ with $s$ randomly chosen from $[min_random_scale, max_random_scale]$. Ignored if $random_resized_crop$ is True.min_random_scale::float, optional, default=1
: Resize into $[widths, heights]$ with $s$ randomly chosen from $[min_random_scale, max_random_scale]$Ignored if $random_resized_crop$ is True.max_random_area::float, optional, default=1
: Change the area (namely width * height) to a random value in $[min_random_area, max_random_area]$. Ignored if $random_resized_crop$ is False.min_random_area::float, optional, default=1
: Change the area (namely width * height) to a random value in $[min_random_area, max_random_area]$. Ignored if $random_resized_crop$ is False.max_img_size::float, optional, default=1e+10
: Set the maximal width and height after all resize and rotate argumentation are appliedmin_img_size::float, optional, default=0
: Set the minimal width and height after all resize and rotate argumentation are appliedbrightness::float, optional, default=0
: Add a random value in $[-brightness, brightness]$ to the brightness of image.contrast::float, optional, default=0
: Add a random value in $[-contrast, contrast]$ to the contrast of image.saturation::float, optional, default=0
: Add a random value in $[-saturation, saturation]$ to the saturation of image.pca_noise::float, optional, default=0
: Add PCA based noise to the image.random_h::int, optional, default='0'
: Add a random value in $[-random_h, random_h]$ to the H channel in HSL color space.random_s::int, optional, default='0'
: Add a random value in $[-random_s, random_s]$ to the S channel in HSL color space.random_l::int, optional, default='0'
: Add a random value in $[-random_l, random_l]$ to the L channel in HSL color space.rotate::int, optional, default='-1'
: Rotate by an angle. If set, it overwrites the $max_rotate_angle$ option.fill_value::int, optional, default='255'
: Set the padding pixels value to $fill_value$.inter_method::int, optional, default='1'
: The interpolation method: 0-NN 1-bilinear 2-cubic 3-area 4-lanczos4 9-auto 10-rand.pad::int, optional, default='0'
: Change size from $[width, height]$ into $[pad + width + pad, pad + height + pad]$ by padding pixesmirror::boolean, optional, default=0
: Whether to mirror the image or not. If true, images are flipped along the horizontal axis.rand_mirror::boolean, optional, default=0
: Whether to randomly mirror images or not. If true, 50% of the images will be randomly mirrored (flipped along the horizontal axis)mean_img::string, optional, default=''
: Filename of the mean image.mean_r::float, optional, default=0
: The mean value to be subtracted on the R channelmean_g::float, optional, default=0
: The mean value to be subtracted on the G channelmean_b::float, optional, default=0
: The mean value to be subtracted on the B channelmean_a::float, optional, default=0
: The mean value to be subtracted on the alpha channelstd_r::float, optional, default=1
: Augmentation Param: Standard deviation on R channel.std_g::float, optional, default=1
: Augmentation Param: Standard deviation on G channel.std_b::float, optional, default=1
: Augmentation Param: Standard deviation on B channel.std_a::float, optional, default=1
: Augmentation Param: Standard deviation on Alpha channel.scale::float, optional, default=1
: Multiply the image with a scale value.max_random_contrast::float, optional, default=0
: Change the contrast with a value randomly chosen from $[-max_random_contrast, max_random_contrast]$max_random_illumination::float, optional, default=0
: Change the illumination with a value randomly chosen from $[-max_random_illumination, max_random_illumination]$
Returns the constructed MXDataProvider
.
#
MXNet.mx.ImageRecordUInt8Iter
— Method.
ImageRecordUInt8Iter(path_imglist, path_imgrec, path_imgidx, aug_seq, label_width, data_shape, preprocess_threads, verbose, num_parts, part_index, device_id, shuffle_chunk_size, shuffle_chunk_seed, seed_aug, shuffle, seed, verbose, batch_size, round_batch, prefetch_buffer, ctx, dtype, resize, rand_crop, random_resized_crop, max_rotate_angle, max_aspect_ratio, min_aspect_ratio, max_shear_ratio, max_crop_size, min_crop_size, max_random_scale, min_random_scale, max_random_area, min_random_area, max_img_size, min_img_size, brightness, contrast, saturation, pca_noise, random_h, random_s, random_l, rotate, fill_value, data_shape, inter_method, pad)
Can also be called with the alias ImageRecordUInt8Provider
. Iterating on image RecordIO files
.. note:: ImageRecordUInt8Iter is deprecated. Use ImageRecordIter(dtype='uint8') instead.
This iterator is identical to $ImageRecordIter$ except for using $uint8$ as the data type instead of $float$.
Defined in src/io/iterimagerecordio_2.cc:L916
Arguments:
data_name::Symbol
: keyword argument, default:data
. The name of the data.label_name::Symbol
: keyword argument, default:softmax_label
. The name of the label. Could benothing
if no label is presented in this dataset.path_imglist::string, optional, default=''
: Path to the image list (.lst) file. Generally created with tools/im2rec.py. Format (Tab separated):. path_imgrec::string, optional, default=''
: Path to the image RecordIO (.rec) file or a directory path. Created with tools/im2rec.py.path_imgidx::string, optional, default=''
: Path to the image RecordIO index (.idx) file. Created with tools/im2rec.py.aug_seq::string, optional, default='aug_default'
: The augmenter names to represent sequence of augmenters to be applied, seperated by comma. Additional keyword parameters will be seen by these augmenters.label_width::int, optional, default='1'
: The number of labels per image.data_shape::Shape(tuple), required
: The shape of one output image in (channels, height, width) format.preprocess_threads::int, optional, default='4'
: The number of threads to do preprocessing.verbose::boolean, optional, default=1
: If or not output verbose information.num_parts::int, optional, default='1'
: Virtually partition the data into these many parts.part_index::int, optional, default='0'
: The i-th virtual partition to be read.device_id::int, optional, default='0'
: The device id used to create context for internal NDArray. Setting deviceid to -1 will create Context::CPU(0). Setting deviceid to valid positive device id will create Context::CPUPinned(device_id). Default is 0.shuffle_chunk_size::long (non-negative), optional, default=0
: The data shuffle buffer size in MB. Only valid if shuffle is true.shuffle_chunk_seed::int, optional, default='0'
: The random seed for shufflingseed_aug::int or None, optional, default='None'
: Random seed for augmentations.shuffle::boolean, optional, default=0
: Whether to shuffle data randomly or not.seed::int, optional, default='0'
: The random seed.batch_size::int (non-negative), required
: Batch size.round_batch::boolean, optional, default=1
: Whether to use round robin to handle overflow batch or not.prefetch_buffer::long (non-negative), optional, default=4
: Maximum number of batches to prefetch.ctx::{'cpu', 'gpu'},optional, default='gpu'
: Context data loader optimized for.dtype::{None, 'float16', 'float32', 'float64', 'int32', 'int64', 'int8', 'uint8'},optional, default='None'
: Output data type. $None$ means no change.resize::int, optional, default='-1'
: Down scale the shorter edge to a new size before applying other augmentations.rand_crop::boolean, optional, default=0
: If or not randomly crop the imagerandom_resized_crop::boolean, optional, default=0
: If or not perform random resized cropping on the image, as a standard preprocessing for resnet training on ImageNet data.max_rotate_angle::int, optional, default='0'
: Rotate by a random degree in $[-v, v]$max_aspect_ratio::float, optional, default=0
: Change the aspect (namely width/height) to a random value. If minaspectratio is None then the aspect ratio ins sampled from [1 - maxaspectratio, 1 + maxaspectratio], else it is in $[min_aspect_ratio, max_aspect_ratio]$min_aspect_ratio::float or None, optional, default=None
: Change the aspect (namely width/height) to a random value in $[min_aspect_ratio, max_aspect_ratio]$max_shear_ratio::float, optional, default=0
: Apply a shear transformation (namely $(x,y)->(x+my,y)$) with $m$ randomly chose from $[-max_shear_ratio, max_shear_ratio]$max_crop_size::int, optional, default='-1'
: Crop both width and height into a random size in $[min_crop_size, max_crop_size].$Ignored if $random_resized_crop$ is True.min_crop_size::int, optional, default='-1'
: Crop both width and height into a random size in $[min_crop_size, max_crop_size].$Ignored if $random_resized_crop$ is True.max_random_scale::float, optional, default=1
: Resize into $[widths, heights]$ with $s$ randomly chosen from $[min_random_scale, max_random_scale]$. Ignored if $random_resized_crop$ is True.min_random_scale::float, optional, default=1
: Resize into $[widths, heights]$ with $s$ randomly chosen from $[min_random_scale, max_random_scale]$Ignored if $random_resized_crop$ is True.max_random_area::float, optional, default=1
: Change the area (namely width * height) to a random value in $[min_random_area, max_random_area]$. Ignored if $random_resized_crop$ is False.min_random_area::float, optional, default=1
: Change the area (namely width * height) to a random value in $[min_random_area, max_random_area]$. Ignored if $random_resized_crop$ is False.max_img_size::float, optional, default=1e+10
: Set the maximal width and height after all resize and rotate argumentation are appliedmin_img_size::float, optional, default=0
: Set the minimal width and height after all resize and rotate argumentation are appliedbrightness::float, optional, default=0
: Add a random value in $[-brightness, brightness]$ to the brightness of image.contrast::float, optional, default=0
: Add a random value in $[-contrast, contrast]$ to the contrast of image.saturation::float, optional, default=0
: Add a random value in $[-saturation, saturation]$ to the saturation of image.pca_noise::float, optional, default=0
: Add PCA based noise to the image.random_h::int, optional, default='0'
: Add a random value in $[-random_h, random_h]$ to the H channel in HSL color space.random_s::int, optional, default='0'
: Add a random value in $[-random_s, random_s]$ to the S channel in HSL color space.random_l::int, optional, default='0'
: Add a random value in $[-random_l, random_l]$ to the L channel in HSL color space.rotate::int, optional, default='-1'
: Rotate by an angle. If set, it overwrites the $max_rotate_angle$ option.fill_value::int, optional, default='255'
: Set the padding pixels value to $fill_value$.inter_method::int, optional, default='1'
: The interpolation method: 0-NN 1-bilinear 2-cubic 3-area 4-lanczos4 9-auto 10-rand.pad::int, optional, default='0'
: Change size from $[width, height]$ into $[pad + width + pad, pad + height + pad]$ by padding pixes
Returns the constructed MXDataProvider
.
#
MXNet.mx.ImageRecordUInt8Iter_v1
— Method.
ImageRecordUInt8Iter_v1(path_imglist, path_imgrec, path_imgidx, aug_seq, label_width, data_shape, preprocess_threads, verbose, num_parts, part_index, device_id, shuffle_chunk_size, shuffle_chunk_seed, seed_aug, shuffle, seed, verbose, batch_size, round_batch, prefetch_buffer, ctx, dtype, resize, rand_crop, random_resized_crop, max_rotate_angle, max_aspect_ratio, min_aspect_ratio, max_shear_ratio, max_crop_size, min_crop_size, max_random_scale, min_random_scale, max_random_area, min_random_area, max_img_size, min_img_size, brightness, contrast, saturation, pca_noise, random_h, random_s, random_l, rotate, fill_value, data_shape, inter_method, pad)
Iterating on image RecordIO files
.. note::
$ImageRecordUInt8Iter_v1$ is deprecated. Use $ImageRecordUInt8Iter$ instead.
This iterator is identical to $ImageRecordIter$ except for using $uint8$ as the data type instead of $float$.
Defined in src/io/iterimagerecordio.cc:L377
Arguments:
data_name::Symbol
: keyword argument, default:data
. The name of the data.label_name::Symbol
: keyword argument, default:softmax_label
. The name of the label. Could benothing
if no label is presented in this dataset.path_imglist::string, optional, default=''
: Path to the image list (.lst) file. Generally created with tools/im2rec.py. Format (Tab separated):. path_imgrec::string, optional, default=''
: Path to the image RecordIO (.rec) file or a directory path. Created with tools/im2rec.py.path_imgidx::string, optional, default=''
: Path to the image RecordIO index (.idx) file. Created with tools/im2rec.py.aug_seq::string, optional, default='aug_default'
: The augmenter names to represent sequence of augmenters to be applied, seperated by comma. Additional keyword parameters will be seen by these augmenters.label_width::int, optional, default='1'
: The number of labels per image.data_shape::Shape(tuple), required
: The shape of one output image in (channels, height, width) format.preprocess_threads::int, optional, default='4'
: The number of threads to do preprocessing.verbose::boolean, optional, default=1
: If or not output verbose information.num_parts::int, optional, default='1'
: Virtually partition the data into these many parts.part_index::int, optional, default='0'
: The i-th virtual partition to be read.device_id::int, optional, default='0'
: The device id used to create context for internal NDArray. Setting deviceid to -1 will create Context::CPU(0). Setting deviceid to valid positive device id will create Context::CPUPinned(device_id). Default is 0.shuffle_chunk_size::long (non-negative), optional, default=0
: The data shuffle buffer size in MB. Only valid if shuffle is true.shuffle_chunk_seed::int, optional, default='0'
: The random seed for shufflingseed_aug::int or None, optional, default='None'
: Random seed for augmentations.shuffle::boolean, optional, default=0
: Whether to shuffle data randomly or not.seed::int, optional, default='0'
: The random seed.batch_size::int (non-negative), required
: Batch size.round_batch::boolean, optional, default=1
: Whether to use round robin to handle overflow batch or not.prefetch_buffer::long (non-negative), optional, default=4
: Maximum number of batches to prefetch.ctx::{'cpu', 'gpu'},optional, default='gpu'
: Context data loader optimized for.dtype::{None, 'float16', 'float32', 'float64', 'int32', 'int64', 'int8', 'uint8'},optional, default='None'
: Output data type. $None$ means no change.resize::int, optional, default='-1'
: Down scale the shorter edge to a new size before applying other augmentations.rand_crop::boolean, optional, default=0
: If or not randomly crop the imagerandom_resized_crop::boolean, optional, default=0
: If or not perform random resized cropping on the image, as a standard preprocessing for resnet training on ImageNet data.max_rotate_angle::int, optional, default='0'
: Rotate by a random degree in $[-v, v]$max_aspect_ratio::float, optional, default=0
: Change the aspect (namely width/height) to a random value. If minaspectratio is None then the aspect ratio ins sampled from [1 - maxaspectratio, 1 + maxaspectratio], else it is in $[min_aspect_ratio, max_aspect_ratio]$min_aspect_ratio::float or None, optional, default=None
: Change the aspect (namely width/height) to a random value in $[min_aspect_ratio, max_aspect_ratio]$max_shear_ratio::float, optional, default=0
: Apply a shear transformation (namely $(x,y)->(x+my,y)$) with $m$ randomly chose from $[-max_shear_ratio, max_shear_ratio]$max_crop_size::int, optional, default='-1'
: Crop both width and height into a random size in $[min_crop_size, max_crop_size].$Ignored if $random_resized_crop$ is True.min_crop_size::int, optional, default='-1'
: Crop both width and height into a random size in $[min_crop_size, max_crop_size].$Ignored if $random_resized_crop$ is True.max_random_scale::float, optional, default=1
: Resize into $[widths, heights]$ with $s$ randomly chosen from $[min_random_scale, max_random_scale]$. Ignored if $random_resized_crop$ is True.min_random_scale::float, optional, default=1
: Resize into $[widths, heights]$ with $s$ randomly chosen from $[min_random_scale, max_random_scale]$Ignored if $random_resized_crop$ is True.max_random_area::float, optional, default=1
: Change the area (namely width * height) to a random value in $[min_random_area, max_random_area]$. Ignored if $random_resized_crop$ is False.min_random_area::float, optional, default=1
: Change the area (namely width * height) to a random value in $[min_random_area, max_random_area]$. Ignored if $random_resized_crop$ is False.max_img_size::float, optional, default=1e+10
: Set the maximal width and height after all resize and rotate argumentation are appliedmin_img_size::float, optional, default=0
: Set the minimal width and height after all resize and rotate argumentation are appliedbrightness::float, optional, default=0
: Add a random value in $[-brightness, brightness]$ to the brightness of image.contrast::float, optional, default=0
: Add a random value in $[-contrast, contrast]$ to the contrast of image.saturation::float, optional, default=0
: Add a random value in $[-saturation, saturation]$ to the saturation of image.pca_noise::float, optional, default=0
: Add PCA based noise to the image.random_h::int, optional, default='0'
: Add a random value in $[-random_h, random_h]$ to the H channel in HSL color space.random_s::int, optional, default='0'
: Add a random value in $[-random_s, random_s]$ to the S channel in HSL color space.random_l::int, optional, default='0'
: Add a random value in $[-random_l, random_l]$ to the L channel in HSL color space.rotate::int, optional, default='-1'
: Rotate by an angle. If set, it overwrites the $max_rotate_angle$ option.fill_value::int, optional, default='255'
: Set the padding pixels value to $fill_value$.inter_method::int, optional, default='1'
: The interpolation method: 0-NN 1-bilinear 2-cubic 3-area 4-lanczos4 9-auto 10-rand.pad::int, optional, default='0'
: Change size from $[width, height]$ into $[pad + width + pad, pad + height + pad]$ by padding pixes
Returns the constructed MXDataProvider
.
#
MXNet.mx.LibSVMIter
— Method.
LibSVMIter(data_libsvm, data_shape, label_libsvm, label_shape, num_parts, part_index, batch_size, round_batch, prefetch_buffer, ctx, dtype)
Can also be called with the alias LibSVMProvider
. Returns the LibSVM iterator which returns data with csr
storage type. This iterator is experimental and should be used with care.
The input data is stored in a format similar to LibSVM file format, except that the indices are expected to be zero-based instead of one-based, and the column indices for each row are expected to be sorted in ascending order. Details of the LibSVM format are available here. <https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/>
_
The data_shape
parameter is used to set the shape of each line of the data. The dimension of both data_shape
and label_shape
are expected to be 1.
The data_libsvm
parameter is used to set the path input LibSVM file. When it is set to a directory, all the files in the directory will be read.
When label_libsvm
is set to $NULL$, both data and label are read from the file specified by data_libsvm
. In this case, the data is stored in csr
storage type, while the label is a 1D dense array.
The LibSVMIter
only support round_batch
parameter set to $True$. Therefore, if batch_size
is 3 and there are 4 total rows in libsvm file, 2 more examples are consumed at the first round.
When num_parts
and part_index
are provided, the data is split into num_parts
partitions, and the iterator only reads the part_index
-th partition. However, the partitions are not guaranteed to be even.
$reset()$ is expected to be called only after a complete pass of data.
Example::
Contents of libsvm file $data.t$.
1.0 0:0.5 2:1.2 -2.0 -3.0 0:0.6 1:2.4 2:1.2 4 2:-1.2
Creates a LibSVMIter
with batch_size
=3.
dataiter = mx.io.LibSVMIter(datalibsvm = 'data.t', datashape = (3,), batchsize = 3)
The data of the first batch is stored in csr storage type
batch = data_iter.next() csr = batch.data[0]
csr.asnumpy()
[[ 0.5 0. 1.2 ] [ 0. 0. 0. ] [ 0.6 2.4 1.2]]
The label of first batch
label = batch.label[0] label
[ 1. -2. -3.]
secondbatch = dataiter.next()
The data of the second batch
second_batch.data[0].asnumpy()
[[ 0. 0. -1.2 ] [ 0.5 0. 1.2 ] [ 0. 0. 0. ]]
The label of the second batch
second_batch.label[0].asnumpy()
[ 4. 1. -2.]
data_iter.reset()
To restart the iterator for the second pass of the data
When label_libsvm
is set to the path to another LibSVM file, data is read from data_libsvm
and label from label_libsvm
. In this case, both data and label are stored in the csr format. If the label column in the data_libsvm
file is ignored.
Example::
Contents of libsvm file $label.t$
1.0 -2.0 0:0.125 -3.0 2:1.2 4 1:1.0 2:-1.2
Creates a LibSVMIter
with specified label file
dataiter = mx.io.LibSVMIter(datalibsvm = 'data.t', data_shape = (3,),
label_libsvm = 'label.t', label_shape = (3,), batch_size = 3)
Both data and label are in csr storage type
batch = dataiter.next() csrdata = batch.data[0]
csr_data.asnumpy()
[[ 0.5 0. 1.2 ] [ 0. 0. 0. ] [ 0.6 2.4 1.2 ]]
csr_label = batch.label[0]
csr_label.asnumpy()
[[ 0. 0. 0. ] [ 0.125 0. 0. ] [ 0. 0. 1.2 ]]
Defined in src/io/iter_libsvm.cc:L298
Arguments:
data_name::Symbol
: keyword argument, default:data
. The name of the data.label_name::Symbol
: keyword argument, default:softmax_label
. The name of the label. Could benothing
if no label is presented in this dataset.data_libsvm::string, required
: The input zero-base indexed LibSVM data file or a directory path.data_shape::Shape(tuple), required
: The shape of one example.label_libsvm::string, optional, default='NULL'
: The input LibSVM label file or a directory path. If NULL, all labels will be read from $data_libsvm$.label_shape::Shape(tuple), optional, default=[1]
: The shape of one label.num_parts::int, optional, default='1'
: partition the data into multiple partspart_index::int, optional, default='0'
: the index of the part will readbatch_size::int (non-negative), required
: Batch size.round_batch::boolean, optional, default=1
: Whether to use round robin to handle overflow batch or not.prefetch_buffer::long (non-negative), optional, default=4
: Maximum number of batches to prefetch.ctx::{'cpu', 'gpu'},optional, default='gpu'
: Context data loader optimized for.dtype::{None, 'float16', 'float32', 'float64', 'int32', 'int64', 'int8', 'uint8'},optional, default='None'
: Output data type. $None$ means no change.
Returns the constructed MXDataProvider
.
#
MXNet.mx.MNISTIter
— Method.
MNISTIter(image, label, batch_size, shuffle, flat, seed, silent, num_parts, part_index, prefetch_buffer, ctx, dtype)
Can also be called with the alias MNISTProvider
. Iterating on the MNIST dataset.
One can download the dataset from http://yann.lecun.com/exdb/mnist/
Defined in src/io/iter_mnist.cc:L265
Arguments:
data_name::Symbol
: keyword argument, default:data
. The name of the data.label_name::Symbol
: keyword argument, default:softmax_label
. The name of the label. Could benothing
if no label is presented in this dataset.image::string, optional, default='./train-images-idx3-ubyte'
: Dataset Param: Mnist image path.label::string, optional, default='./train-labels-idx1-ubyte'
: Dataset Param: Mnist label path.batch_size::int, optional, default='128'
: Batch Param: Batch Size.shuffle::boolean, optional, default=1
: Augmentation Param: Whether to shuffle data.flat::boolean, optional, default=0
: Augmentation Param: Whether to flat the data into 1D.seed::int, optional, default='0'
: Augmentation Param: Random Seed.silent::boolean, optional, default=0
: Auxiliary Param: Whether to print out data info.num_parts::int, optional, default='1'
: partition the data into multiple partspart_index::int, optional, default='0'
: the index of the part will readprefetch_buffer::long (non-negative), optional, default=4
: Maximum number of batches to prefetch.ctx::{'cpu', 'gpu'},optional, default='gpu'
: Context data loader optimized for.dtype::{None, 'float16', 'float32', 'float64', 'int32', 'int64', 'int8', 'uint8'},optional, default='None'
: Output data type. $None$ means no change.
Returns the constructed MXDataProvider
.
#
MXNet.mx.eachbatch
— Method.
eachbatch(provider::AbstractDataProvider)
Allows you to perform operations on data every epoch. This is especially useful when you need to perform real-time augmentation of the data.
Arguments:
provider
: an instance of the custom DataProvider type. You must return this
instance after modifying its fields.
#
MXNet.mx.from_json
— Method.
from_json(repr :: AbstractString, ::Type{SymbolicNode})
Load a SymbolicNode
from a JSON string representation.
#
MXNet.mx.is_shared
— Method.
is_shared(j_arr, arr)
Test whether j_arr
is sharing data with arr
.
Arguments:
j_arr::Array
: the Julia Array.arr::NDArray
: theNDArray
.
#
MXNet.mx.load
— Method.
load(filename, ::Type{NDArray})
Load NDArrays from binary file.
Arguments:
filename::String
: the path of the file to load. It could be S3 or HDFS address.
Returns either Dict{Symbol, NDArray}
or Vector{NDArray}
.
filename
can point to s3
or hdfs
resources if the libmxnet
is built with the corresponding components enabled. Examples:
s3://my-bucket/path/my-s3-ndarray
hdfs://my-bucket/path/my-hdfs-ndarray
/path-to/my-local-ndarray
#
MXNet.mx.load
— Method.
load(filename :: AbstractString, ::Type{SymbolicNode})
Load a SymbolicNode
from a JSON file.
#
MXNet.mx.load_data!
— Method.
load_data!(provider, batch, targets)
Arguments:
provider::AbstractDataProvider
: the data provider.batch::AbstractDataBatch
: the data batch object.targets::Vector{Vector{SlicedNDArray}}
: the targets to load data into.
The targets is a list of the same length as number of data provided by this provider. Each element in the list is a list of SlicedNDArray
. This list described a spliting scheme of this data batch into different slices, each slice is specified by a slice-ndarray pair, where slice specify the range of samples in the mini-batch that should be loaded into the corresponding ndarray.
This utility function is used in data parallelization, where a mini-batch is splited and computed on several different devices.
#
MXNet.mx.load_label!
— Method.
load_label!(provider, batch, targets)
provider::AbstractDataProvider provider
: the data provider.batch::AbstractDataBatch batch
: the data batch object.targets::Vector{Vector{SlicedNDArray}}
: the targets to load label into.
The same as load_data!
, except that this is for loading labels.
#
MXNet.mx.save
— Method.
save(filename :: AbstractString, node :: SymbolicNode)
Save a SymbolicNode
to a JSON file.
#
MXNet.mx.save
— Method.
save(filename::AbstractString, data)
Save NDarrays to binary file. Filename could be S3 or HDFS address, if libmxnet
is built with corresponding support (see load
).
filename::String
: path to the binary file to write to.data
: data to save to file. Data can be aNDArray
, aVector
ofNDArray
, or aDict{Symbol}
containsNDArray
s.
#
MXNet.mx.to_json
— Method.
to_json(s::SymbolicNode)
Convert a SymbolicNode
into a JSON string.
#
MXNet.mx.try_get_shared
— Method.
try_get_shared(arr; sync=:nop)
Try to create a Julia array by sharing the data with the underlying NDArray
.
Arguments:
arr::NDArray
: the array to be shared.
Note
The returned array does not guarantee to share data with the underlying NDArray
. In particular, data sharing is possible only when the NDArray
lives on CPU.
sync::Symbol
::nop
,:write
,:read
On CPU, invoke_wait_to_read
if:read
; invoke_wait_to_write
if:write
.


Right from the beginning of our sales process, we are striving to get to know your business. Even though we have experience in many industries, you are not a faceless statistic. Your pains are unique because they are yours. HillSouth will study your technology situation and engineer sound solutions that will drive real results for your business throughout our partnership.
TELL US ABOUT YOUTrue performance comes in the form of a long-term relationship, where solutions are provided, nurtured and monitored for continued success. We break each of the years of your contract into their natural quarters. The first quarter is the on-boarding process where we implement your Stabilization Project followed by a Deep Dive Assessment where we strive to learn the absolute most about your network. The following three are monitoring quarters each separated by mini assessments and another Deep Dive Assessment.
WANT TO GET STARTED?Not knowing is painful, especially when it is affecting your business. HillSouth is on your side, so we use email, online chat and phone calls as methods for you to submit your trouble tickets. However, our best method for ticket submission is our ConnectWise Customer Portal, where you can submit your ticket, see all of the notes that the technicians add, and be notified when the ticket is marked as complete. Stop worrying about your trouble tickets and worry about the things that matter, your customers.
SOLVE YOUR PROBLEMSGreat customer service come from knowing two things: 1) Single points of failure are how trouble tickets get lost and ultimately ignored and 2) Put your best assets, the employees that the customers know first, as your most trusted line of defense. That is where the HillSouth HelpDesk comes in. Our Tier 1 Support solves about 80% of all trouble tickets within 30 - 60 minutes. Our Tier 2 Support has a guaranteed response time of 12 hours and acts as the face of the company. Our Tier 3 Support builds and services the most complex of problems.
LET US HELP YOU TODAYThat's a great question, and it gets answered by giving us a call or sending us an email. We'll come in and take a look at your unique situation and work towards designing, building and delivering you a sound solution backed by our guarantee: to never stop thinking.


A quick drive to sunrise point will surely make your morning. ★ ★ ★ ★ ☆ 4.0 stars
Grab dinner and a movie with your loved one, they will love it. ★ ★ ★ ★ ★ 5.0 stars
Go out for a special meal in ASTON V8 and enjoy your night. ★ ★ ★ ★ ☆ 4.0 stars
Head outstation with the entire family, have some family love. ★ ★ ★ ☆ ☆ 3.0 stars
Experience the power of ferrari at most nominal cost. ★ ★ ★ ★ ★ 5.0 stars
Have your meeting sorted at best-in-class mercedes slr. ★ ★ ★ ★ ☆ 4.0 stars


Welcome to iReporter
3 Under Investigation
| Date Created | Issue Type | Location | Topic | Description | Status |
|---|---|---|---|---|---|
| 24th Nov, 2018 | Red Flag | Polling Unit | NextLevel Daura | Last 4 years has been a sham | Under Investigation |
| 24th Nov, 2018 | Red Flag | Polling Unit | NextLevel Daura | Last 4 years has been a sham | Under Investigation |
| 24th Nov, 2018 | Red Flag | Polling Unit | NextLevel Daura | Last 4 years has been a sham | Under Investigation |
| 24th Nov, 2018 | Red Flag | Polling Unit | NextLevel Daura | Last 4 years has been a sham | Under Investigation |
| 24th Nov, 2018 | Red Flag | Polling Unit | NextLevel Daura | Last 4 years has been a sham | Under Investigation |
| 24th Nov, 2018 | Red Flag | Polling Unit | NextLevel Daura | Last 4 years has been a sham | Under Investigation |
| 24th Nov, 2016 | Intervention | Iyana Ipaja | Fix our light problem | We don't have electricity for more than 2 weeks | Resolved |


val DEFAULT_ALGORITHM_NAME: DigestAlgorithmName
The DEFAULT_ALGORITHM_NAME instance will be parametrized and initialized at runtime.
It would be probably useful to assume an override priority order.


General Definitions
Suspensive Primitives
There are two types of primitives. Those that may cause a task to be preempted, these are called suspending primitives, and those that return to the caller without preemption, called non- suspending primitives. The suspending primitives are,
| PRIMITIVE | SUSPENDING CONDITION |
|---|---|
| ENTER_CR | Critical region is not available. |
| RECV | No message is pending to task. |
| WAIT | No event is pending to task for requested event condition. |
The above suspending primitives have complementing primitives which, when called from another task or interrupt service routine, cause the suspended task to resume. A task resumes when the suspending condition is satisfied. The complementing primitives in corresponding order are EXIT_CR, SEND, and SIGNAL.
Suspending primitives may not be called from an interrupt service routine because an interrupt may not be suspended by the kernel. Suspension supports real-time applications at the task level.
While tasks may be prioritized, primitives do not have a priority attribute. For example, there is no priority assigned to messages for the SEND and RECV primitives. All prioritization is considered with respect to tasks.
Common Primitive Attributes
This section describes the kernel interface conventions. These conventions have been defined to give a consistent interface, which makes application software easier to implement and maintain.
Primitive Access
The primitive interface to the kernel is invoked as an assembly or C-language subroutine call. Macros, defined in include files IF_K_M.H and IF_K_M.INC implement the actual access to the kernel.
All kernel primitives return a function status. This is the status of the execution of the algorithm for the primitive; either SUCCESS or FAILURE or some specialized meaning. No data associated with the primitive are ever returned as a status. Data are passed and returned as parameters.
Literals Used with Primitives
Include files provide a common, portable interface to the kernel. Include files IF_L.H, IF_L.INC, IF_K_L.H, and IF_K_L.INC contain the literals used with the kernel primitives described in this section. These include files may be changed to adapt the kernel to the application. Those literals that may be changed are indicated with an asterisk (*).
These literals provide a method to reference kernel attributes symbolically. This higher level of abstraction makes the interface more maintainable and portable.
| GENERAL | LITERAL MEANING |
|---|---|
| TRUE | General literal for TRUE conditional. |
| FALSE | General literal for FALSE conditional. |
| DO_FOREVER | This is used within a task to define the start of the main task execution loop. |
| DO_NOTHING | A statement place-holder, which generates no code. |
| INV_ADDR | This literal is used to set pointer variables to an invalid address, which may be used for fault detection. |
| NOT_USED | This is a parameter literal to indicate a parameter is not used or has no significance. |
| SUCCESS | This value is returned by primitives to indicate no errors occurred. |
| FAILURE | If an error occurred during primitive execution, a FAILURE status is returned. |
| TIMEOUT | This literal is returned by primitives to indicate a timeout condition occurred while waiting for the requested operation. |
| NO_TOUT | If no timeout is desired, this literal is used for the time-specification parameter. |
Task Attributes
| LITERAL | MEANING |
|---|---|
| MAX_TSK(*) | For task management operations, this defines the maximum number of task instances, including multiple-instance tasks, that may be referenced. The range is 0 to 63. |
| MAX_TNAME | For task management operations, this defines the limit for task names. The range is 0 to 63. |
| INV_TNAME | Invalid task name indication. |
| INV_TID | Invalid task identifier. |
| STKSIZ_L STKSIZ_M STKSIZ_S |
|
| MEMSIZ_L MEMSIZ_M MEMSIZ_S |
|
| PRI_HIGH PRI_NORM PRI_NONE |
|
| BANK_0 BANK_1 BANK_2 BANK_3 |
|
Queue Management
| LITERAL | MEANING |
|---|---|
| MAX_QUE | For queue management operations, this defines the maximum number of queues that may be referenced. The range is 0 to 255. |
| INV_QNAME | Invalid queue name reference. |
| INV_QID | Invalid queue identifier. |
| Q_FULL | Queue is full indication. |
| Q_EMPTY | Queue is empty indication. |
Semaphore Management
| LITERAL | MEANING |
|---|---|
| MAX_SEM | For semaphore related operations, this defines the maximum number of semaphores that may be referenced. The range is 0 to 255. |
| INV_SEMNAME | Invalid semaphore name reference. |
| INV_SEMID | Invalid semaphore identifier. |
Event Management
| LITERAL | MEANING |
|---|---|
| EVT_OR | Conditionally wait for any event specified. |
| EVT_AND | Conditionally wait for all events specified. |
| EVT_0 EVT_1 EVT_2 EVT_3 EVT_4 EVT_5 EVT_6 EVT_7 EVT_8 EVT_9 EVT_10 EVT_11 EVT_12 EVT_13 EVT_14 EVT_15 | Specific event identifiers; event number 0 to event number 15, the maximum number of events. These symbolic names may be replaced by names meaningful to the application. |
Timer Management
| LITERAL | MEANING |
|---|---|
| MS_100 | (*) Time intervals: 100 milliseconds. |
| MS_500 | 500 milliseconds. |
| SEC_1 | One second. |
| SEC_10 | Ten seconds. |
| MIN_1 | One minute. |
| MIN_10 | Ten minutes. |
| HR_1 | One hour. |
Note
The timer management literal values are for a real-time clock interrupt rate of 32.77 milliseconds. If this rate is changed, these literals must also be changed. Other time reference symbols may be added, within the limitation of the maximum interval that can be represented in 16 bits.
Fault Management
When used as the error location parameter with the LOG_FATAL and LOG_WARN primitives these literals in the bit significance shown below.
| bit | meaning |
|---|---|
| 15 | reserved |
| 14‑12 | layer identifier |
| 11‑10 | subsystem identifier |
| 9‑8 | level identifier |
| 7‑4 | procedure identifier |
| 3‑0 | fault identifier |
Here is an example of how the literals may be used with the fault management primitives. The example shows the announcement of the third fault in layer 3, subsystem 0, level 2, and procedure 2.
LOG_FATAL (LY_3 + SS_0 +LV_L + P2 + 3, NOT_USED)
The error location is logged in the Fault Analysis Area.


Vegetables and legumes/beans CATEGORY
Grain Food CATEGORY
Fruits CATEGORY
Milk, chees, eggs and alternatives CATEGORY
Lean meats and poultry, fish... CATEGORY
Method of preparation...
Description ...
Share Recipe
Edit Recipe


IROS Advisory Committee |
|
Honorary Founding Chair
|
| Fumio Harashima, University of Tokyo, Tokyo Metropolitan University, Tokyo Denki University, Nara University, Japan |
|
|
Founding Chair
|
| Toshio Fukuda, Nagoya University, Meijo University, Beijing Institute of Technology (1988 GC) |
|
|
Founding Vice Chair
|
| Hideki Hashimoto, Chuo University, Japan (1988 VC) |
| Shin'ichi Yuta, Shibaura Institute of Technology, Japan (1989 GC) |
|
IROS Steering Committee |
|
Past Chair
|
| Tzyh-Jong Tarn, Washington University, USA (2001 GC) |
|
|
Chair
|
| C.S. George Lee, Purdue University, USA (1998 PC, 2003 GC) |
|
|
Awards Chair
|
| Shigeki Sugano, Waseda University, Japan (2013 GC) |
|
|
Secretary
|
| Yasushi Nakauchi, Tsukuba University, Japan |
|
|
Members
|
| Satoshi Tadokoro, President of IEEE RAS |
| Kamal Al-Haddad, President of IEEE IES |
| Atsuo Takanishi, President of RSJ |
| Akira Maeda, President of SICE |
| Hirofumi Tashiro, Executive Director of NTF |
| Masakatsu Fujie (1990 GC) |
| Fumio Miyazaki (1991 GC) |
| Ren C. Luo (1992 GC, 2010 GC) |
| Volker Graefe (1994 GC) |
| Katsushi Ikeuchi (1995 GC) |
| Minoru Asada (1996 GC) |
| Tamio Arai (1996 PC) |
| Christian Laugier (1997 GC, 2002 PC, 2008 PC) |
| Hyung Suck Cho (1999 GC) |
| Roland Siegwart (2002 GC) |
| Junku Yuh (2003 PC) |
| Kazuhiro Kosuge (2004 GC) |
| Hajime Asama (2004 PC, 2010 PC) |
| Max Meng (2005 GC) |
| Hong Zhang (2005 PC, 2017 GC) |
| Yunhui Liu (2006 GC) |
| Ning Xi (2006 PC) |
| Thomas C. Henderson (2007 PC) |
| Raja Chatila (2008 Co-GC) |
| Jean-Pierre Merlet (2008 Co-GC) |
| William R. Hamel (2009 PC) |
| Oussama Khatib (2011 GC) |
| Gaurav Sukhatme (2011 PC) |
| Anibal T. de Almeida (2012 GC) |
| Urbano Nunes (2012 Co-GC) |
| Eugenio Guglielmelli (2012 PC) |
| Makoto Kaneko (2013 PC) |
| Kevin Lynch (2014 GC) |
| Lynne E. Parker (2014 PC) |
| Jianwei Zhang (2015 GC) |
| Alois Knoll (2015 PC) |
| Il Hong Suh (2016 GC) |
| Dong-Soo Kwon (2016 PC) |
|
National Advisory Committee |
|
Members
|
| Wook Hyoun Kwon (Seoul Nat'l U) |
| Z. Zenn Bien (KAIST) |
| Hyung Suck Cho (KAIST) |
| Sukhan Lee (Sungkyunkwan U) |
| Myung Jin Chung (Gwangju Health U) |
| Jun Ho Oh (KAIST) |
| Jin Bae Park (Yonsei U) |
| Beom Hee Lee (Seoul Nat'l U) |
| Kyung Chul Shin (Yujin Robot) |
| Kyungwon Chung (KIRIA) |
| Sang-rok Oh (KIST) |
| Mun Sang Kim (KIST) |
| Jangmyung Lee (Pusan Nat'l U) |
| Wan Kyun Chung (POSTECH) |
| Junku Yuh (KIST) |
|


The ::com::sun::star::container::XNameContainer
and ::com::sun::star::container::XNameReplace
interfaces can be used to manage the members that this group contains.
It is possible to insert, remove, and replace members.
The method
::com::sun::star::container::XNameReplace::replaceByName can be used to rename the
specified member. To do so, the new name has to be passed as second
argument. An implementation should support the following data types:
- a non-empty
string
specifying the new name of the
group member.
- an object supporting the
::com::sun::star::container::XNamed interface, for example an instance of
DataPilotFieldGroupItem received from another group.
The method
::com::sun::star::container::XNameContainer::insertByName can be used to insert a new
member into this field group. An implementation should ignore the
second argument and insert the specified member name.
The method
::com::sun::star::container::XNameContainer::removeByName can be used to remove an
existing member name.


--- layout: default ---
CS 573. DATA VISUALIZATION
Computer Science
September 7 - November 9, 2017
* last assignment due November 16th
Instructor:
Curran Kelleher
<EMAIL_ADDRESS>Required Material:
Reference Material (optional, but awesome)::
Course Description:
This course is all about data visualization, the art and science of turning data into readable graphics. We’ll explore how to design and create data visualizations based on data available and tasks to be achieved. This process includes data modeling, data processing (such as aggregation and filtering), mapping data attributes to graphical attributes, and strategic visual encoding based on known properties of visual perception as well as the task(s) at hand. Students will also learn to evaluate the effectiveness of visualization designs, and think critically about each design decision, such as choice of color and choice of visual encoding. Students will create their own data visualizations, and learn to use Open Source data visualization tools, especially D3.js. Students will also read papers from the current and past visualization literature and create video presentations of their findings.
Prerequisite Courses:
A graduate or undergraduate course in computer graphics.
Learning Outcomes:
By the completion of this course, learners will be able to:
Communication:
Course Approach:
Course Content:
|
Class |
Material to be Covered |
Homework Assignments/Projects |
|
1 |
Overview of Data Visualization |
Reading: Chapter. 1 “What’s Vis, and Why Do It?” Assignment (Homework): |
|
2 |
The Shapes of Data |
Reading: Chapter 2 “What: Data Abstraction” Assignment (Project): |
|
3 |
Marks and Channels |
Reading: Chapter 5 “Marks and Channels” Assignment (Homework): |
|
4 |
Rules of Thumb |
Reading: Chapter 6 “Rules of Thumb” Assignment (Project): |
|
5 |
Common Visualization Idioms |
Reading: Chapter 7 “Arrange Tables” Assignment (Project): |
|
6 |
Using Color in Visualization |
Reading: Assignment (Project): |
|
7 |
Interaction Techniques |
Reading: Assignment (Project): |
|
8 |
Multiple Linked Views |
Reading: Assignment (Project): |
|
9 |
Data Reduction |
Reading: Assignment (Project) |
|
10 |
Focus + Context |
Reading: Assignment (Project): |
Note: Weekly assignments may be subject to change, and will be given in detail week by week.
Course Requirements:
|
|
Percentage |
|
Assignments |
50% |
|
Project |
30% |
|
Quizzes |
10% |
|
Participation |
10% |
Course incompletes may be granted if the major part of the course is completed; however, no additional credit can be given for missed class discussions or teamwork beyond the end of the course (November 9, 2017). In addition, in the case of an incomplete, the student is responsible for handing in the final work within the WPI required timeframe of one (1) year. After this time, an incomplete grade changes to a failing (F) grade
We will have weekly assignments, due by the end of the week (Wednesday 11:59PM). Submission of the assignments will usually be inside the Canvas online system.
Many assignments will involve submission of a link to your work in an external system such as GitHub or bl.ocks.org. Feedback can be expected one week after submission.
Late work will be accepted, with a 10% grade penalty for each day that it is late.
Resubmissions based on initial grading and feedback will be re-graded if submitted before the deadline. You may re-submit as many times as you like before the deadline. After the deadline for an assignment, it will only be graded once and the grade will not be updated after subsequent resubmissions. Please submit a few days ahead of the deadline so you can get feedback, resubmit based on feedback, and maximize your grade.
Our Slack team will function as participatory space for discussion, like a classroom. Students are encouraged and expected to actively participate in discussions there. Sometimes, assignments will include a discussion participation dimension. This may involve sharing your work with the class, posting your response to a specific question, or responding to the posts of others (peer feedback). Participation in code reviews via GitHub may also be required. Each assignment will include specific grading criteria with regard to participation.
Technical Requirements:
Library Access:
As a student at WPI, you have access to a variety of resources through the library. Use the link here to access databases, e-journals, and/or e-books. You will be required to log in with your WPI username and password to access materials.
POLICIES
Academic Integrity:
You are expected to be familiar with the Student Guide to Academic Integrity at WPI that is downloadable from here. Consequences for violating the Academic Honest Policy range from earning a zero on the assignment, failing the course, or being suspended or expelled from WPI.
Common examples of violations include:
Academic Accommodations:
We at WPI strive to create an inclusive environment where all students are valued members of the class community. If you need course adaptations or accommodations because of a disability, or if you have medical information to share with us that may impact your performance or participation in this course, please make an appointment with us as soon as possible. If you have approved accommodations, please request your accommodation letters online through the Office of Disability Services student portal. If you have not already done so, students with disabilities who need to utilize accommodations for this course are encouraged to contact the Office of Disability Services as soon as possible to ensure that such accommodations are implemented in a timely fashion.
Email –<EMAIL_ADDRESS>Phone –<PHONE_NUMBER>
On Campus – Daniels Hall, First Floor 124
Grading Policy:
Final course grades are based on a student’s performance as follows:
|
Letter Grade |
Percentage |
|
A |
90 - 100 |
|
B |
80 - 89 |
|
C |
70 - 79 |
|
D |
60 - 69 |
|
F |
< 60 |
Course incompletes may be granted if the major part of the course is completed; however, no additional credit can be given for missed class discussions or teamwork beyond the end of the course (November 9, 2017). In addition, in the case of an incomplete, the student is responsible for handing in the final work within the WPI required timeframe of one (1) year. After this time, an incomplete grade changes to a failing (F) grade.


I am a student of Iyengar Yoga and web development. This website is an ongoing project, serving three functions:
I am heavily influenced by Iyengar Yoga because I have benefitted greatly from the method's systematic approach to learning yoga. Iyengar yoga is about blah blah blah.
Aside from my technical reasons to build this site, I have started this project to further my own understanding of yoga. This site is not meant to be anything more than that.
Audio taken from http://www.yogagroup.org/patanjal.html
Yogena cittasya padena vacam
Malam sarirasyaca vaidyakena
Yopakarottam prvaram muninam
Patanjalim pranjaliranato’smi
Abahu purusakaram
Sankha cakrasi dharinam
Sahasra sirasam svetam
Pranamami Patanjalim
Let us bow before the noblest of sages, Patanjali, who gave yoga for serenity and sanctity of mind, grammar for clarity and purity of speech, and medicine for perfection of health.
Let us prostrate before Patanjali, an incarnation of Adisesa, whose upper body has a human form, whose arms hold a conch and a disc, and who is crowned by a thousand-headed cobra.


PUBLI©ITY mission is to develop multicultural public spaces in the Chicago Area, encouraging: social justice, diversity, equality, education, art stimulation and establishment of dynamic green spaces.
Well designed public space must allows the users to interact with the environment and the people around them regar- less of age, income level, cultural background, or social status.
Type: maker
Artists working in collaboration with community members, to figuratively, literally, and critically reimagine, the spaces around us.
Type: educator
A device that will encourage people to interact with each other in a more meaningful way.
Type: maker
Using performance art interventions and participatory design tools to invite community conversations and inclusive workshops for the exploration and re-imagining of public space.
Type: thinker
Encouraging the conversation about the reinvention of underutilized public space through a sustainability, technology and culture.
Type: maker
Foster agency and embrace democratic design through welcoming, hands-on access to design processes, available directly to the inhabitants of public spaces.
Type: maker
The moment of serendipity; harmoniously curated sensory experiences to evoke new emotions.
Type: maker
Interventions addressing community concerns through curiosity, play and interactions within the community.
Type: maker
A film to share real stories and ideas, as visionaries with a keen interest in urban planning.
Type: thinker
Inspire curiosity, playfulness and unique experiences, giving visitors a more intimate relationship with public spaces.
Type: maker
Design is meant to be a public space, translating languages among disciplines.
Type: thinker
Connecting visitors with natural systems and propose methods for harnessing natural resources.
Type: maker
Democratic participatory design for public access and education for sustainability and green technology.
Type: maker


Rome is a sprawling, cosmopolitan city with nearly 3,000 years of globally influential art, architecture and culture on display.
Ancient ruins such as the Forum and the Colosseum evoke the power of the former Roman Empire.
Barcelona, the cosmopolitan capital of Spain's Catalonia region, is known for its art and architecture.
The fantastical Sagrada Família church and other modernist landmarks designed by Antoni Gaudí dot the city.
San Francisco, officially City and County of San Francisco and colloquially known as SF, San Fran or "The City", is the cultural, commercial, and financial center of Northern California.


Welcome to my starter kit
This is a super, stupidly simple version of the starter kit.
Ideally, this would exist in a <my-app />
tag, but here we are.
Forms
Lists
@mrjimbusiness
This is a super, stupidly simple version of the starter kit.
Ideally, this would exist in a <my-app />
tag, but here we are.


More and more, I find myself designing with code, and I don't intend to stop. It's an amazing feeling when you're able to craft a desisn by typing up some html and css, especially when you can do it quickly. Some things, however, are lost in this design process. One of which is visual guides, grids, and layout tools that you find in design & prototyping applications such as photoshop and illustrator.
Ghost Grid was created as a way to visualize complex css grid systems for designers who work in the browser.
There have been other attempts at bridging this gap which I have taken heavy inspiration from. Most notably is the Golden Grid System and its 'Golden Gridlet' overlay. My goal with Ghost Grid was to simply create an easily customizable, dependency free version of what Joni Korpi did with GGS that had a few more features.


{% if data['error'] %}
> Error: {{data['error']}}
{% endif %}
We have {{data['usercount']}} users!
This is your new username:
{{data['username']}}
Use your own if you already have one.
Enter a valid username
Save this username to recover your data anywhere
Start feeding


Coming soon
Sign up with your IOST address and get 10% mining bonus at launch.
Get Bonus
Join us on Telegram
You've successfully registered.
You've already registered for the bonus.
Copyright © Royal Palace 2019
Copyright © Royal Palace 2019


Main
Light
MPC
iTunes
PC
Weather
Light Control
Sofa light
N/A
N/A
TV light
N/A
Recliner light
N/A
Living room light
N/A
Chair light
N/A
All lights
ON
OFF
Media Player Control
No video playing
Play
Stop
Close
MPC Select Movie
Back ..
Folder Downloads
Folder E:\Watch
No Folder Selected
iTunes Control
No music playing
Play
Stop
Vol.Up
Vol.Down
Open
Last MQTT message:
no message received


Please enter the name you would like your processed outputs to be called.
Please enter the video time where you would like to save a frame.
3D Endoscope Hardware
Warning: You are about to being a endoscopy session.
Please be sure of the following:
1) You are using a USB3 FPGA board with a 2-sensor endoscope.
2) The USB3 board is powered on, and connected to this computer with a USB and HDMI cable.
Please enter the name you would like to save your endoscope videos as:


I am an assistant professor in the Department of Mathematical Sciences at NJIT. I teach applied mathematics courses (linear algebra, partial differential equations, vector analysis, etc.) and conduct research in scientific computing and numerical analysis. My primary research interests are in fast and high-order accurate numerical methods for the PDEs of mathematical physics and optimization algorithms for inverse problems.
askham [at] njit [dot] edu


--- layout: post title: "@Disclosedtech: Edsurge and NewSchool Venture Fund" date: 2017-02-17 12:35:00 +0000 tags: image: https://s3.amazonaws.com/hackedu/2017-02-17-pigeon.jpg imagecredits: https://www.flickr.com/photos/soualte/4366080164 ---
"New Funding Challenge Aims to Support Special Education Edtech Tools," Edsurge wrote on February 16, 2017. This article fails to disclose that the funding challenge is run by one of Edsurge's investors NewSchools Venture Fund.
For more information about this and other disclosure alerts, read this blog post.


Employee Data Management
words and things
Current Employees
Employee Name
Role
Start Date
Months Worked
Monthly Rate ($)
Total Billed ($)
Add Employee
Employee Name
Role
Start Date
Monthly Rate
Submit