In this project, you'll classify images from the CIFAR-10 dataset. The dataset consists of airplanes, dogs, cats, and other objects. You'll preprocess the images, then train a convolutional neural network on all the samples. The images need to be normalized and the labels need to be one-hot encoded. You'll get to apply what you learned and build a convolutional, max pooling, dropout, and fully connected layers. At the end, you'll get to see your neural network's predictions on the sample images.
Run the following cell to download the CIFAR-10 dataset for python.
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
from urllib.request import urlretrieve
from os.path import isfile, isdir
from tqdm import tqdm
import problem_unittests as tests
import tarfile
cifar10_dataset_folder_path = 'cifar-10-batches-py'
class DLProgress(tqdm):
last_block = 0
def hook(self, block_num=1, block_size=1, total_size=None):
self.total = total_size
self.update((block_num - self.last_block) * block_size)
self.last_block = block_num
if not isfile('cifar-10-python.tar.gz'):
with DLProgress(unit='B', unit_scale=True, miniters=1, desc='CIFAR-10 Dataset') as pbar:
urlretrieve(
'https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz',
'cifar-10-python.tar.gz',
pbar.hook)
if not isdir(cifar10_dataset_folder_path):
with tarfile.open('cifar-10-python.tar.gz') as tar:
tar.extractall()
tar.close()
tests.test_folder_path(cifar10_dataset_folder_path)
The dataset is broken into batches to prevent your machine from running out of memory. The CIFAR-10 dataset consists of 5 batches, named data_batch_1
, data_batch_2
, etc.. Each batch contains the labels and images that are one of the following:
Understanding a dataset is part of making predictions on the data. Play around with the code cell below by changing the batch_id
and sample_id
. The batch_id
is the id for a batch (1-5). The sample_id
is the id for a image and label pair in the batch.
Ask yourself "What are all possible labels?", "What is the range of values for the image data?", "Are the labels in order or random?". Answers to questions like these will help you preprocess the data and end up with better predictions.
%matplotlib inline
%config InlineBackend.figure_format = 'retina'
import helper
import numpy as np
# Explore the dataset
batch_id = 1
sample_id = 5
helper.display_stats(cifar10_dataset_folder_path, batch_id, sample_id)
def normalize(x):
"""
Normalize a list of sample image data in the range of 0 to 1
: x: List of image data. The image shape is (32, 32, 3)
: return: Numpy array of normalize data
"""
# TODO: Implement Function
min_data = np.min(x)
max_data = np.max(x)
normalize_data = (x - min_data) / (max_data - min_data)
return normalize_data
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_normalize(normalize)
Just like the previous code cell, you'll be implementing a function for preprocessing. This time, you'll implement the one_hot_encode
function. The input, x
, are a list of labels. Implement the function to return the list of labels as One-Hot encoded Numpy array. The possible values for labels are 0 to 9. The one-hot encoding function should return the same encoding for each value between each call to one_hot_encode
. Make sure to save the map of encodings outside the function.
Hint: Don't reinvent the wheel.
def one_hot_encode(x):
"""
One hot encode a list of sample labels. Return a one-hot encoded vector for each label.
: x: List of sample Labels
: return: Numpy array of one-hot encoded labels
"""
# TODO: Implement Function
num = len(x)
label = np.zeros((num, 10))
label[np.arange(num), x] = 1
return label
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_one_hot_encode(one_hot_encode)
As you saw from exploring the data above, the order of the samples are randomized. It doesn't hurt to randomize it again, but you don't need to for this dataset.
Running the code cell below will preprocess all the CIFAR-10 data and save it to file. The code below also uses 10% of the training data for validation.
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# Preprocess Training, Validation, and Testing Data
helper.preprocess_and_save_data(cifar10_dataset_folder_path, normalize, one_hot_encode)
This is your first checkpoint. If you ever decide to come back to this notebook or have to restart the notebook, you can start from here. The preprocessed data has been saved to disk.
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import pickle
import problem_unittests as tests
import helper
# Load the Preprocessed Validation data
valid_features, valid_labels = pickle.load(open('preprocess_validation.p', mode='rb'))
For the neural network, you'll build each layer into a function. Most of the code you've seen has been outside of functions. To test your code more thoroughly, we require that you put each layer in a function. This allows us to give you better feedback and test for simple mistakes using our unittests before you submit your project.
If you're finding it hard to dedicate enough time for this course a week, we've provided a small shortcut to this part of the project. In the next couple of problems, you'll have the option to use TensorFlow Layers or TensorFlow Layers (contrib) to build each layer, except "Convolutional & Max Pooling" layer. TF Layers is similar to Keras's and TFLearn's abstraction to layers, so it's easy to pickup.
If you would like to get the most of this course, try to solve all the problems without TF Layers. Let's begin!
The neural network needs to read the image data, one-hot encoded labels, and dropout keep probability. Implement the following functions
neural_net_image_input
image_shape
with batch size set to None
.name
parameter in the TF Placeholder.neural_net_label_input
n_classes
with batch size set to None
.name
parameter in the TF Placeholder.neural_net_keep_prob_input
name
parameter in the TF Placeholder.These names will be used at the end of the project to load your saved model.
Note: None
for shapes in TensorFlow allow for a dynamic size.
import tensorflow as tf
def neural_net_image_input(image_shape):
"""
Return a Tensor for a bach of image input
: image_shape: Shape of the images
: return: Tensor for image input.
"""
# TODO: Implement Function
width, height, depth = image_shape
return tf.placeholder(tf.float32, shape=(None, width, height, depth), name = "x")
def neural_net_label_input(n_classes):
"""
Return a Tensor for a batch of label input
: n_classes: Number of classes
: return: Tensor for label input.
"""
# TODO: Implement Function
return tf.placeholder(tf.float32, shape=(None, n_classes), name = "y")
def neural_net_keep_prob_input():
"""
Return a Tensor for keep probability
: return: Tensor for keep probability.
"""
# TODO: Implement Function
return tf.placeholder(tf.float32, name = "keep_prob")
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tf.reset_default_graph()
tests.test_nn_image_inputs(neural_net_image_input)
tests.test_nn_label_inputs(neural_net_label_input)
tests.test_nn_keep_prob_inputs(neural_net_keep_prob_input)
Convolution layers have a lot of success with images. For this code cell, you should implement the function conv2d_maxpool
to apply convolution then max pooling:
conv_ksize
, conv_num_outputs
and the shape of x_tensor
.x_tensor
using weight and conv_strides
.pool_ksize
and pool_strides
.Note: You can't use TensorFlow Layers or TensorFlow Layers (contrib) for this layer. You're free to use any TensorFlow package for all the other layers.
def conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides):
"""
Apply convolution then max pooling to x_tensor
:param x_tensor: TensorFlow Tensor
:param conv_num_outputs: Number of outputs for the convolutional layer
:param conv_strides: Stride 2-D Tuple for convolution
:param pool_ksize: kernal size 2-D Tuple for pool
:param pool_strides: Stride 2-D Tuple for pool
: return: A tensor that represents convolution and max pooling of x_tensor
"""
# TODO: Implement Function
num, width, height, depth = x_tensor.get_shape().as_list()
conv_width, conv_height = conv_ksize
conv_strides_width, conv_strides_height = conv_strides
pool_width, pool_height = pool_ksize
pool_strides_width, pool_strides_height = pool_strides
weight = tf.Variable(tf.truncated_normal([conv_width, conv_height, depth, conv_num_outputs], mean=0, stddev=0.1))
bias = tf.Variable(tf.zeros(conv_num_outputs))
x = tf.nn.conv2d(x_tensor, weight, strides = [1, conv_strides_width, conv_strides_height, 1], padding = "SAME")
x = tf.nn.bias_add(x, bias)
x = tf.nn.relu(x)
x = tf.nn.max_pool(x, ksize = [1, pool_width, pool_height, 1], strides = [1, pool_strides_width, pool_strides_height, 1], padding = "SAME")
return x
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_con_pool(conv2d_maxpool)
Implement the flatten
function to change the dimension of x_tensor
from a 4-D tensor to a 2-D tensor. The output should be the shape (Batch Size, Flattened Image Size). You can use TensorFlow Layers or TensorFlow Layers (contrib) for this layer.
def flatten(x_tensor):
"""
Flatten x_tensor to (Batch Size, Flattened Image Size)
: x_tensor: A tensor of size (Batch Size, ...), where ... are the image dimensions.
: return: A tensor of size (Batch Size, Flattened Image Size).
 You can use TensorFlow Layers or TensorFlow Layers (contrib) for this layer.
def fully_conn(x_tensor, num_outputs):
"""
Apply a fully connected layer to x_tensor using weight and bias
: x_tensor: A 2-D tensor where the first dimension is batch size.
: num_outputs: The number of output that the new tensor should be.
: return: A 2-D tensor where the second dimension is num_outputs.
 You can use TensorFlow Layers or TensorFlow Layers (contrib) for this layer.
Note: Activation, softmax, or cross entropy shouldn't be applied to this.
def output(x_tensor, num_outputs):
"""
Apply a output layer to x_tensor using weight and bias
: x_tensor: A 2-D tensor where the first dimension is batch size.
: num_outputs: The number of output that the new tensor should be.
: return: A 2-D tensor where the second dimension is num_outputs.
 The function takes in a batch of images, x
, and outputs logits. Use the layers you created above to create this model:
keep_prob
. def conv_net(x, keep_prob):
"""
Create a convolutional neural network model
: x: Placeholder tensor that holds image data.
: keep_prob: Placeholder tensor that hold dropout keep probability.
: return: Tensor that represents logits
"""
# TODO: Apply 1, 2, or 3 Convolution and Max Pool layers
# Play around with different number of outputs, kernel size and stride
# Function Definition from Above:
# conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)
conv1 = conv2d_maxpool(x, 64, [3, 3], [1, 1], [2, 2], [2, 2])
conv2 = conv2d_maxpool(conv1, 128, [2, 2], [1, 1], [2, 2], [2, 2])
conv3 = conv2d_maxpool(conv2, 256, [2, 2], [1, 1], [2, 2], [2, 2])
# TODO: Apply a Flatten Layer
# Function Definition from Above:
# flatten(x_tensor)
fc1 = flatten(conv3)
# TODO: Apply 1, 2, or 3 Fully Connected Layers
# Play around with different number of outputs
# Function Definition from Above:
# fully_conn(x_tensor, num_outputs)
fc1 = fully_conn(fc1, 4096)
fc2 = tf.nn.dropout(fc1, keep_prob)
fc2 = fully_conn(fc2, 4096)
fc3 = tf.nn.dropout(fc2, keep_prob)
fc3 = fully_conn(fc3, 4096)
fc3 = tf.nn.dropout(fc3, keep_prob)
# TODO: Apply an Output Layer
# Set this to the number of classes
# Function Definition from Above:
# output(x_tensor, num_outputs)
out = output(fc3, 10)
# TODO: return output
return out
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
##############################
## Build the Neural Network ##
##############################
# Remove previous weights, bias, inputs, etc..
tf.reset_default_graph()
# Inputs
x = neural_net_image_input((32, 32, 3))
y = neural_net_label_input(10)
keep_prob = neural_net_keep_prob_input()
# Model
logits = conv_net(x, keep_prob)
# Name logits Tensor, so that is can be loaded from disk after training
logits = tf.identity(logits, name='logits')
# Loss and Optimizer
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y))
optimizer = tf.train.AdamOptimizer().minimize(cost)
# Accuracy
correct_pred = tf.equal(tf.argmax(logits, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32), name='accuracy')
tests.test_conv_net(conv_net)
Implement the function train_neural_network
to do a single optimization. The optimization should use optimizer
to optimize in session
with a feed_dict
of the following:
x
for image inputy
for labelskeep_prob
for keep probability for dropoutThis function will be called for each batch, so tf.global_variables_initializer()
has already been called.
Note: Nothing needs to be returned. This function is only optimizing the neural network.
def train_neural_network(session, optimizer, keep_probability, feature_batch, label_batch):
"""
Optimize the session on a batch of images and labels
: session: Current TensorFlow session
: optimizer: TensorFlow optimizer function
: keep_probability: keep probability
: feature_batch: Batch of Numpy image data
: label_batch: Batch of Numpy label data
"""
# TODO: Implement Function
session.run(optimizer, feed_dict = {x: feature_batch, y: label_batch, keep_prob: keep_probability})
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_train_nn(train_neural_network)
Implement the function print_stats
to print loss and validation accuracy. Use the global variables valid_features
and valid_labels
to calculate validation accuracy. Use a keep probability of 1.0
to calculate the loss and validation accuracy.
def print_stats(session, feature_batch, label_batch, cost, accuracy):
"""
Print information about loss and validation accuracy
: session: Current TensorFlow session
: feature_batch: Batch of Numpy image data
: label_batch: Batch of Numpy label data
: cost: TensorFlow cost function
: accuracy: TensorFlow accuracy function
"""
# TODO: Implement Function
loss = sess.run(cost, feed_dict={x: feature_batch, y: label_batch, keep_prob: 1.})
valid_acc = sess.run(accuracy, feed_dict = {x: valid_features, y: valid_labels, keep_prob: 1.})
print('Loss: {:>10.4f} Validation Accuracy: {:.6f}'.format(loss, valid_acc))
Tune the following parameters:
epochs
to the number of iterations until the network stops learning or start overfittingbatch_size
to the highest number that your machine has memory for. Most people set them to common sizes of memory:keep_probability
to the probability of keeping a node using dropout# TODO: Tune Parameters
epochs = 10
batch_size = 256
keep_probability = 0.5
Instead of training the neural network on all the CIFAR-10 batches of data, let's use a single batch. This should save time while you iterate on the model to get a better accuracy. Once the final validation accuracy is 50% or greater, run the model on all the data in the next section.
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
print('Checking the Training on a Single Batch...')
with tf.Session() as sess:
# Initializing the variables
sess.run(tf.global_variables_initializer())
# Training cycle
for epoch in range(epochs):
batch_i = 1
for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):
train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)
print('Epoch {:>2}, CIFAR-10 Batch {}: '.format(epoch + 1, batch_i), end='')
print_stats(sess, batch_features, batch_labels, cost, accuracy)
Now that you got a good accuracy with a single CIFAR-10 batch, try it with all five batches.
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
save_model_path = './image_classification'
print('Training...')
with tf.Session() as sess:
# Initializing the variables
sess.run(tf.global_variables_initializer())
# Training cycle
for epoch in range(epochs):
# Loop over all batches
n_batches = 5
for batch_i in range(1, n_batches + 1):
for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):
train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)
print('Epoch {:>2}, CIFAR-10 Batch {}: '.format(epoch + 1, batch_i), end='')
print_stats(sess, batch_features, batch_labels, cost, accuracy)
# Save Model
saver = tf.train.Saver()
save_path = saver.save(sess, save_model_path)
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
%matplotlib inline
%config InlineBackend.figure_format = 'retina'
import tensorflow as tf
import pickle
import helper
import random
# Set batch size if not already set
try:
if batch_size:
pass
except NameError:
batch_size = 64
save_model_path = './image_classification'
n_samples = 4
top_n_predictions = 3
def test_model():
"""
Test the saved model against the test dataset
"""
test_features, test_labels = pickle.load(open('preprocess_training.p', mode='rb'))
loaded_graph = tf.Graph()
with tf.Session(graph=loaded_graph) as sess:
# Load model
loader = tf.train.import_meta_graph(save_model_path + '.meta')
loader.restore(sess, save_model_path)
# Get Tensors from loaded model
loaded_x = loaded_graph.get_tensor_by_name('x:0')
loaded_y = loaded_graph.get_tensor_by_name('y:0')
loaded_keep_prob = loaded_graph.get_tensor_by_name('keep_prob:0')
loaded_logits = loaded_graph.get_tensor_by_name('logits:0')
loaded_acc = loaded_graph.get_tensor_by_name('accuracy:0')
# Get accuracy in batches for memory limitations
test_batch_acc_total = 0
test_batch_count = 0
for train_feature_batch, train_label_batch in helper.batch_features_labels(test_features, test_labels, batch_size):
test_batch_acc_total += sess.run(
loaded_acc,
feed_dict={loaded_x: train_feature_batch, loaded_y: train_label_batch, loaded_keep_prob: 1.0})
test_batch_count += 1
print('Testing Accuracy: {}\n'.format(test_batch_acc_total/test_batch_count))
# Print Random Samples
random_test_features, random_test_labels = tuple(zip(*random.sample(list(zip(test_features, test_labels)), n_samples)))
random_test_predictions = sess.run(
tf.nn.top_k(tf.nn.softmax(loaded_logits), top_n_predictions),
feed_dict={loaded_x: random_test_features, loaded_y: random_test_labels, loaded_keep_prob: 1.0})
helper.display_image_predictions(random_test_features, random_test_labels, random_test_predictions)
test_model()
You might be wondering why you can't get an accuracy any higher. First things first, 50% isn't bad for a simple CNN. Pure guessing would get you 10% accuracy. However, you might notice people are getting scores well above 70%. That's because we haven't taught you all there is to know about neural networks. We still need to cover a few more techniques.
When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as "dlnd_image_classification.ipynb" and save it as a HTML file under "File" -> "Download as". Include the "helper.py" and "problem_unittests.py" files in your submission.


ADP
: adposition
Definition
ADP
for Japanese covers postpositional particles. It corresponds to
particle (case)
/ 助詞-格助詞 and particle (binding)
/ 助詞-係助詞
in UniDic definition.
Note that some particles (助詞) in Japanese are classified to other Universal PoS classes, such as と (CCONJ), て (SCONJ) and ね (PART).
Examples
- が nominative case postpositional particle (
particle (case)
/ 助詞-格助詞) - を accusative case postpositional particle (
particle (case)
/ 助詞-格助詞) - は a binding particle mainly used as a topic marker (
particle (binding)
/ 助詞-係助詞)
Treebank Statistics (UD_Japanese)
There are 63 ADP
lemmas (0%), 79 ADP
types (0%) and 38273 ADP
tokens (22%).
Out of 14 observed tags, the rank of ADP
is: 8 in number of lemmas, 8 in number of types and 2 in number of tokens.
The 10 most frequent ADP
lemmas: の, は, を, に, が, と, で, も, から, や
The 10 most frequent ADP
types: の, は, を, に, が, と, で, も, から, や
The 10 most frequent ambiguous lemmas: の (ADP 7759, PART 1040), に (ADP 5055, PART 32), が (ADP 3888, SCONJ 700), と (ADP 2949, SCONJ 487), で (ADP 2582, CCONJ 8), から (ADP 927, SCONJ 53), など (ADP 524, ADV 4), より (ADP 98, ADV 1), くらい (ADP 21, NOUN 3), ずつ (ADP 10, NOUN 2)
The 10 most frequent ambiguous types: の (ADP 7759, PART 1039, AUX 67), に (ADP 5055, AUX 562, PART 32, SCONJ 16), が (ADP 3887, SCONJ 700), と (ADP 2937, SCONJ 487), で (ADP 2582, AUX 443, SCONJ 135, CCONJ 8, VERB 6), から (ADP 927, SCONJ 53), など (ADP 517, ADV 4), へ (ADP 230, VERB 3), より (ADP 98, VERB 18, ADV 1), くらい (ADP 21, NOUN 3)
- の
- に
- が
- と
- で
- ADP 2582: 一方 B 国 で は 輸出 により 小麦 の 量 が 減る ので 、 小麦 の 価格 は 上がる 。
- AUX 443: 富士通 は 、 産業機械 だけ で なく 、 社会 インフラ や 物流 業界 など に対して M 2 M サービス を 提供 する 。
- SCONJ 135: 新宿 の 高速バス を 利用 する とき は 必ず 足 を 運ん で しまう 。
- CCONJ 8: 1 シート 15 枚 撮り で 、 5 インチ や 8 インチ の フロッピーディスク の ような スリーブ に 収納 さ れ て いる 。
- VERB 6: ちなみに 納豆 パック の おまけ で つい て る タレ は 、 市販 の 白 だ し で 代用 できる 。
- から
- など
- へ
- より
- くらい
Morphology
The form / lemma ratio of ADP
is 1.253968 (the average of all parts of speech is 1.059217).
The 1st highest number of forms (3) was observed with the lemma “と”: 、と, と, なんて.
The 2nd highest number of forms (3) was observed with the lemma “など”: 、など, など, 等.
The 3rd highest number of forms (2) was observed with the lemma “が”: 、が, が.
ADP
does not occur with any features.
Relations
ADP
nodes are attached to their parents using 4 different relations: ja-dep/case (37599; 98% instances), ja-dep/cc (585; 2% instances), ja-dep/fixed (74; 0% instances), ja-dep/dep (15; 0% instances)
Parents of ADP
nodes belong to 8 different parts of speech: NOUN (28561; 75% instances), PROPN (4069; 11% instances), VERB (2170; 6% instances), NUM (1997; 5% instances), PRON (974; 3% instances), ADJ (417; 1% instances), ADV (83; 0% instances), CCONJ (2; 0% instances)
38260 (100%) ADP
nodes are leaves.
7 (0%) ADP
nodes have one child.
5 (0%) ADP
nodes have two children.
1 (0%) ADP
nodes have three or more children.
The highest child degree of a ADP
node is 5.
Children of ADP
nodes are attached using 4 different relations: ja-dep/punct (13; 59% instances), ja-dep/compound (5; 23% instances), ja-dep/acl (3; 14% instances), ja-dep/nummod (1; 5% instances)
Children of ADP
nodes belong to 5 different parts of speech: PUNCT (13; 59% instances), SYM (5; 23% instances), ADJ (2; 9% instances), NUM (1; 5% instances), VERB (1; 5% instances)
Treebank Statistics (UD_Japanese-KTC)
There are 2 ADP
lemmas (0%), 1 ADP
types (6%) and 40132 ADP
tokens (21%).
Out of 16 observed tags, the rank of ADP
is: 14 in number of lemmas, 2 in number of types and 2 in number of tokens.

The 1st highest number of forms (1) was observed with the lemma “_”: _.
The 2nd highest number of forms (1) was observed with the lemma “の”: _.
ADP
does not occur with any features.
Relations
ADP
nodes are attached to their parents using 4 different relations: ja-dep/case (38196; 95% instances), ja-dep/dep (1765; 4% instances), ja-dep/mark (166; 0% instances), ja-dep/nmod (5; 0% instances)
Parents of ADP
nodes belong to 11 different parts of speech: NOUN (34462; 86% instances), VERB (2357; 6% instances), PROPN (1792; 4% instances), PRON (601; 1% instances), ADJ (363; 1% instances), ADV (231; 1% instances), NUM (145; 0% instances), ADP (90; 0% instances), CONJ (82; 0% instances), INTJ (8; 0% instances), AUX (1; 0% instances)
38944 (97%) ADP
nodes are leaves.
533 (1%) ADP
nodes have one child.
569 (1%) ADP
nodes have two children.
86 (0%) ADP
nodes have three or more children.
The highest child degree of a ADP
node is 3.
Children of ADP
nodes are attached using 2 different relations: ja-dep/mwe (1839; 95% instances), ja-dep/dep (90; 5% instances)
Children of ADP
nodes belong to 5 different parts of speech: VERB (1183; 61% instances), SCONJ (625; 32% instances), ADP (90; 5% instances), AUX (30; 2% instances), PART (1; 0% instances)
ADP in other languages: [am] [ar] [bg] [bxr] [ca] [ckb] [cop] [cs] [cu] [da] [de] [el] [en] [es] [et] [eu] [fa] [fi] [fo] [fr] [ga] [gl] [got] [grc] [he] [hi] [hr] [hu] [id] [it] [ja] [kk] [kmr] [ko] [la] [lv] [mr] [nl] [no] [pl] [pt] [ro] [ru] [sa] [sk] [sla] [sl] [so] [sr] [sv] [swl] [ta] [tr] [ug] [uk] [u] [urj] [ur] [vi] [yue] [zh]


This page can be used to test Google Pay API with PayPay as a payment method. No payment will be processed.
Price: USD $1.00
Open full screen
Close full screen
Based on the Google Pay tutorial.


Business Intelligence Enhancements
The final broad category of DB2 V7 enhancements is to better enable creation, management, and access of DB2 data warehouses built on the mainframe. DB2 V7 supports a new management tool called the Data Warehouse Manager.
The DB2 Data Warehouse Manager makes it easier to use DB2 for data warehousing and business intelligence applications. It is integrated with DB2 Control Center. The predominant capabilities provided by DB2 Data Warehouse Manager include
The ability to control and govern data warehouse queries Help for data cleansing, generating key columns, generating period tables, and inverting and pivoting tables Statistical transformers for business intelligence operations, such as subtotals, rollups, cubes, moving averages, regression, and so on Data replication to allow heterogeneous data movement between data warehouse sources and targets
The DB2 Data Warehouse Manager will make it significantly easier for technicians to deploy useful data warehouses using DB2 as the data store.
|


Tutors246
Home
About
Services
Contact
Sign Up
First, Last Name*:
Email address*:
Work Phone:
Home Phone:
Cell Phone:
This web form is protected from SPAM by
Tutors246.com
Enter web form code*:
reload image
* - required fields.


GIN (Graphic INput) modes set the primary graphic command functions which are entered using the numeric keypad and/or numbered keys.
To perform graphics operations, the system must be in a GIN MODE.
In each GIN mode, the numeric keys perform a function unique to that mode.


Fujitec’s New Installation (NI) group is segmented into teams within individual markets.
Installation/project management specialists from local market . National & global support
Fujitec’s core commitment is providing specialized customer solutions crafted with safety, quality, reliability and comfort.
Ensuring desired results are consistent with specified products
Flexible scheduling to allow for “other trade” or unforeseen delays
Continual management of customer expectations before, during and after project lifecycle
Our ability to react and adjust when architects or design teams create change
Materials ordered per the building’s schedule
Continuously managed and refined by NI
NI manages each project with “one week goals” and “two week look aheads”
Helps resolve issues and monitor progress
Fujitec “Sense of Urgency” or “SOU” checks are reviewed on a daily basis
Helps overcome field and project issues for timely delivery
Dedicated “Construction Maintenance” teams assigned to each install
Teams perform preventative maintenance prior to turnover


The DASH Dev Team is excited to announce our newest project - DASH:
Recall! This mode is a reimagining and rebalancing of vanilla map with
the goal of offering up even more routing possibilities in a variety of
seeds and further diversity in how seeds can be completed within a
competitive racing situation. The following changes are introduced
along with the existing DASH modifications:
-
Rebalanced Beam System
(New Damage Table)
- Current Charge Damage added to HUD under X-Ray
- 4 Individual Charge Upgrades added to Major Item pool
| DASH Combo |
Beam Damage |
Starter Charge |
Charge x1 |
Charge x2 |
Charge x3 |
Charge x4 |
| Power |
20 |
20 |
40 |
60 |
80 |
100 |
| Ice |
30 |
30 |
60 |
90 |
120 |
150 |
| Spazer |
40 |
40 |
80 |
120 |
160 |
200 |
| Wave |
50 |
50 |
100 |
150 |
200 |
250 |
| Ice + Spazer |
50 |
50 |
100 |
150 |
200 |
250 |
| Ice + Wave |
60 |
60 |
120 |
180 |
240 |
300 |
| Wave + Spazer |
80 |
80 |
180 |
250 |
360 |
450 |
| IWS (New Route) |
100 |
100 |
200 |
300 |
400 |
500 |
| Plasma |
100 |
100 |
200 |
300 |
400 |
500 |
-
36 Item Locations
(Map Balance)
- Item Pool: +3 Charge Upgrades, +3 New Items, -2 ETanks, -2 Reserves
- Kraid Etank removed from Major Item Location pool
- Sky Missile added to Major Item Location pool
- Watering Hole Missile added to Major Item Location pool
- Mickey Mouse Missile added to Major Item Location pool
-
Rebalanced WS Reserve Location
(Map Balance)
-
All speed blocks removed from the bowling room (both at the top for scouting the location and the singular block that blocks access to the item)
- WS Reserve Item no longer in a capsule
-
Rebalanced Forgotten Highway
(Map Balance)
- Gray door to Cac Alley is now a blue door (recent change based on player feedback)
-
WS etank location can now be collected before defeating Phantoon if you enter the WS location from the Forgotten Highway side. The door from WS shaft to Sponge Bath remains locked until Phantoon is defeated.
-
Orange door at Forgotten Highway elevator is now blue on both sides
-
Rebalanced Plasma Location
(Map Balance)
- Plasma Gray door is now a blue door
- Replaced pink pirates with green pirates
-
Added platform in Plasma room that
allows exit without “can fly”
-
Rebalanced Spring Ball Location
(Map Balance)
-
Removed all ceiling blocks from the pants room before Shaktool (grapple block and the adjacent blocks)
- Removed the sand from the Shaktool
- Green Gates Opened
(Map Balance)
- Removed green gate from Crab Tunnel room in Maridia
- Removed green gate from Crocomire's Lair (Above Grapple)
-
Pressure Valve (New Item)
-
Works like Gravity Suit in rooms with water other than East
Maridia
-
Note that there is now a North and South Maridia zone that the Pressure Valve DOES work in (See
New Maridia Logic Map with Color Coding)
- Does NOT work in lava or acid anywhere
-
Heat Shield (New Item)
-
Provides 100% reduction to heat damage in Upper Norfair
heated rooms
-
Provides 50% reduction to heat damage in Lower Norfair
heated rooms
- Does NOT stack heat reduction with Gravity Suit
-
Double Jump Item
(New Item)
-
Works like Space Jump but gives only one extra spin jump
-
Rebalanced Gravity Suit
(Item Balance)
-
Gravity Suit provides a flat 25% reduction to heat damage in both Upper Norfair and Lower Norfair heated rooms (down from 50%).
- Does NOT stack with Heat Shield Item
- MB2 does vanilla body damage with only Gravity Suit equipped
-
Common Map Updates
(Map Balance)
-
WS save station is now usable at all times
-
The door to enter Croc's lair is now blue on both sides
-
Ending Sequence Criteria
(Challenge)
-
Less than 45 minutes = Best ending
-
Less than 1 hour = Fair ending
-
1 hour or more = Worst ending
The DASH (Diversity and Selective Haste) Super Metroid Randomizer seeks
to provide more options for runners to complete seeds by limiting the
gating potential of certain items and item locations. DASH is based on
the
Super Metroid Item Randomizer
developed by Total and featured in the 2017 Super Metroid Randomizer
Tournament.
Item Utility Changes
|
- Bombs opens Wrecked Ship Reserve location
- Spazer opens Waterway and Botwoon hallway
-
Samus begins with Starter Charge Beam that does the same
damage as uncharged shots
-
Pseudo Screws and Special Beam Attacks do full damage
with Starter Charge Beam
|
Suit Changes
|
- Gravity Suit provides 50% heat damage reduction
-
Varia and Gravity each add 50% damage reduction to enemy
attacks and environmental damage
-
Varia and Gravity stack to reach 75% reduction to enemy
attacks and environmental damage
|
Item Placement Changes
|
|
|
Miscellaneous
|
- Saving the animals behaves like the vanilla game
|
The table above describes changes featured in DASH that are different
from the
Super Metroid Item Randomizer. A complete breakdown of all updates can be found
here.
There will be a Super Metroid randomizer tournament for cash prizes
featured at the SG Live 2020 event. Due to the current pandemic, this
event is now online, and will be restreamed on SpeedGaming. This
tournament will feature 16 players who qualify for the tournament, as
well as potentially a few other players who purchase special tickets to
support SG Live 2020. An updated version of the DASH randomizer will be
used in this tournament.
To sign up for the tournament, click here:
https://speedgaming.challonge.com/sgl20smr
To see information on the changes to the DASH randomizer that have been
implemented for this tournament specifically, click here:
https://dashrando.github.io/sgl2020
There will be a special DASH Randomizer one-day tournament on January
18, 2020 that will be featured on the
SpeedGaming Twitch
channel. This tournament will feature 16 players who qualify for the
tournament with the top 8 finishers winning cash prizes. The tournament
will begin at 12:00 p.m. ET and will continue throughout the full day
until it is completed. For more information and a complete set of
rules, click here:
https://dashrando.github.io/tournament
On Saturday, November 16, a DASH race between
Zoast and
Rossu was featured during
the DreamHack Atlanta event. Twitch channel
NoResetSpeedrun
hosted a charity marathon at the event, and
Kipp provided commentary on
the run as well as providing basic information about DASH. You can
watch the VOD of the race here:
https://www.twitch.tv/videos/510229319
A beta version of the DASH randomizer was featured in a Crowd Control
race during SG Live 2019. The VOD features extensive discussion before
the race and can be watched here:
https://www.twitch.tv/videos/500670759


Pant, Gill released for SMAT
Delhi has one win from its two matches while Punjab has won its only match against Jharkhand so far.
by Sports ReporterRishabh Pant and Shubman Gill have been released by the Indian team management from the Test squad to allow them to play for Delhi and Punjab respectively in the Syed Mushtaq Ali tournament here in Surat from Sunday.
Delhi has two matches remaining in the Super League phase, against Haryana on Sunday and Rajasthan on Wednesday while Punjab has matches against Karantaka (Sunday), Tamil Nadu (Monday) and Mumbai (Wednesday).
Delhi has one win from its two matches while Punjab has won its only match against Jharkhand so far. Apart from Pant, Delhi will also have the services of pacer Navdeep Saini.
Shikhar Dhawan will miss Sunday's match as well after he suffered an injury to his knee in the first match against Maharashtra.


Class RTPSWriter, manages the sending of data to the readers. More...
#include <RTPSWriter.h>
Public Member Functions | |
| template<typename T > | |
| CacheChange_t * | new_change (T &data, ChangeKind_t changeKind, InstanceHandle_t handle=c_InstanceHandle_Unknown) |
| Create a new change based with the provided changeKind. More... | |
| RTPS_DllAPI CacheChange_t * | new_change (const std::function< uint32_t()> &dataCdrSerializedSize, ChangeKind_t changeKind, InstanceHandle_t handle=c_InstanceHandle_Unknown) |
| virtual RTPS_DllAPI bool | matched_reader_add (RemoteReaderAttributes &ratt)=0 |
| Add a matched reader. More... | |
| virtual RTPS_DllAPI bool | matched_reader_remove (RemoteReaderAttributes &ratt)=0 |
| Remove a matched reader. More... | |
| virtual RTPS_DllAPI bool | matched_reader_is_matched (RemoteReaderAttributes &ratt)=0 |
| Tells us if a specific Reader is matched against this writer. More... | |
| virtual RTPS_DllAPI bool | is_acked_by_all (CacheChange_t *) |
| Check if a specific change has been acknowledged by all Readers. More... | |
| virtual RTPS_DllAPI bool | wait_for_all_acked (const Duration_t &) |
| virtual RTPS_DllAPI void | updateAttributes (WriterAttributes &att)=0 |
| Update the Attributes of the Writer. More... | |
| virtual RTPS_DllAPI void | send_any_unsent_changes ()=0 |
| This method triggers the send operation for unsent changes. More... | |
| RTPS_DllAPI SequenceNumber_t | get_seq_num_min () |
| Get Min Seq Num in History. More... | |
| RTPS_DllAPI SequenceNumber_t | get_seq_num_max () |
| Get Max Seq Num in History. More... | |
| RTPS_DllAPI uint32_t | getTypeMaxSerialized () |
| Get maximum size of the serialized type. More... | |
| uint32_t | getMaxDataSize () |
| uint32_t | calculateMaxDataSize (uint32_t length) |
| RTPS_DllAPI WriterListener * | getListener () |
| Get listener. More... | |
| RTPS_DllAPI bool | getLivelinessAsserted () |
| Get the asserted liveliness. More... | |
| RTPS_DllAPI void | setLivelinessAsserted (bool l) |
| Get the asserted liveliness. More... | |
| RTPS_DllAPI bool | isAsync () |
| Get the publication mode. More... | |
| RTPS_DllAPI bool | remove_older_changes (unsigned int max=0) |
| Remove an specified max number of changes. More... | |
| virtual bool | clean_history (unsigned int max=0)=0 |
| virtual void | add_flow_controller (std::unique_ptr< FlowController > controller)=0 |
| RTPSParticipantImpl * | getRTPSParticipant () const |
| Get RTPS participant. More... | |
| Public Member Functions inherited from Endpoint | |
| RTPS_DllAPI const GUID_t & | getGuid () const |
| Get associated GUID. More... | |
| RTPS_DllAPI std::recursive_mutex * | getMutex () const |
| Get mutex. More... | |
| RTPS_DllAPI EndpointAttributes * | getAttributes () |
| Get associated attributes. More... | |
Protected Member Functions | |
| RTPSWriter (RTPSParticipantImpl *, GUID_t &guid, WriterAttributes &att, WriterHistory *hist, WriterListener *listen=nullptr) | |
| virtual | ~RTPSWriter () |
| void | update_cached_info_nts (std::vector< GUID_t > &&allRemoteReaders, std::vector< LocatorList_t > &allLocatorLists) |
| void | init_header () |
| Initialize the header of hte CDRMessages. More... | |
| virtual void | unsent_change_added_to_history (CacheChange_t *change)=0 |
| Add a change to the unsent list. More... | |
| virtual bool | change_removed_by_history (CacheChange_t *a_change)=0 |
| Indicate the writer that a change has been removed by the history due to some HistoryQos requirement. More... | |
| Protected Member Functions inherited from Endpoint | |
| Endpoint (RTPSParticipantImpl *pimpl, GUID_t &guid, EndpointAttributes &att) | |
| virtual | ~Endpoint () |
Protected Attributes | |
| bool | m_pushMode |
| Is the data sent directly or announced by HB and THEN send to the ones who ask for it?. More... | |
| RTPSMessageGroup_t | m_cdrmessages |
| Group created to send messages more efficiently. More... | |
| bool | m_livelinessAsserted |
| INdicates if the liveliness has been asserted. More... | |
| WriterHistory * | mp_history |
| WriterHistory. More... | |
| WriterListener * | mp_listener |
| Listener. More... | |
| bool | is_async_ |
| LocatorList_t | mAllShrinkedLocatorList |
| std::vector< GUID_t > | mAllRemoteReaders |
| Protected Attributes inherited from Endpoint | |
| RTPSParticipantImpl * | mp_RTPSParticipant |
| Pointer to the RTPSParticipant containing this endpoint. More... | |
| const GUID_t | m_guid |
| Endpoint GUID. More... | |
| EndpointAttributes | m_att |
| Endpoint Attributes. More... | |
| std::recursive_mutex * | mp_mutex |
| Endpoint Mutex. More... | |
Friends | |
| class | WriterHistory |
| class | RTPSParticipantImpl |
| class | RTPSMessageGroup |
Class RTPSWriter, manages the sending of data to the readers.
Is always associated with a HistoryCache.
|
protected |
|
protectedvirtual |
|
pure virtual |
| uint32_t calculateMaxDataSize | ( | uint32_t | length | ) |
|
protectedpure virtual |
Indicate the writer that a change has been removed by the history due to some HistoryQos requirement.
| a_change | Pointer to the change that is going to be removed. |
|
pure virtual |
| RTPS_DllAPI SequenceNumber_t get_seq_num_max | ( | ) |
Get Max Seq Num in History.
| RTPS_DllAPI SequenceNumber_t get_seq_num_min | ( | ) |
Get Min Seq Num in History.
|
inline |
Get listener.
|
inline |
Get the asserted liveliness.
| uint32_t getMaxDataSize | ( | ) |
|
inline |
Get RTPS participant.
| RTPS_DllAPI uint32_t getTypeMaxSerialized | ( | ) |
Get maximum size of the serialized type.
|
protected |
Initialize the header of hte CDRMessages.
|
inlinevirtual |
Check if a specific change has been acknowledged by all Readers.
Is only useful in reliable Writer. In BE Writers always returns true;
|
inline |
Get the publication mode.
|
pure virtual |
Add a matched reader.
| ratt | Pointer to the ReaderProxyData object added. |
|
pure virtual |
Tells us if a specific Reader is matched against this writer.
| ratt | Pointer to the ReaderProxyData object |
|
pure virtual |
Remove a matched reader.
| ratt | Pointer to the object to remove. |
|
inline |
Create a new change based with the provided changeKind.
| changeKind | The type of change. |
| handle | InstanceHandle to assign. |
| RTPS_DllAPI CacheChange_t* new_change | ( | const std::function< uint32_t()> & | dataCdrSerializedSize, |
| ChangeKind_t | changeKind, | ||
| InstanceHandle_t | handle = c_InstanceHandle_Unknown |
||
| ) |
| RTPS_DllAPI bool remove_older_changes | ( | unsigned int | max = 0 | ) |
Remove an specified max number of changes.
|
pure virtual |
This method triggers the send operation for unsent changes.
|
inline |
Get the asserted liveliness.
|
protectedpure virtual |
Add a change to the unsent list.
| change | Pointer to the change to add. |
|
protected |
|
pure virtual |
Update the Attributes of the Writer.
| att | New attributes |
|
inlinevirtual |
|
friend |
|
friend |
|
friend |
|
protected |
|
protected |
Group created to send messages more efficiently.
|
protected |
INdicates if the liveliness has been asserted.
|
protected |
Is the data sent directly or announced by HB and THEN send to the ones who ask for it?.
|
protected |
|
protected |
|
protected |
|
protected |
Listener.


INTRODUCING
JOHN OSIKO
mAsCuLiNe
dArK.
rEcHeRcHe.
bOlD.
eLeGaNt.
sImPlE.
Photographer
Creative designer
Creative writer
UI | UX developer
Currently am a front-end engineer making magical bedtimes at Moringa School.
LEARN MORE


Enter your DOB:
Day
This is your age:
This is your age in Mars:
This is your age in Jupiter:
This is your age in Venus:
This is your age in Mercury:


Build a spatial description using statements of the form:ROI is relationship to colour.
ROI is
No description


Practice makes man perfect. We at coding studio strongly believe this quote and practice coding 24x7
Just like we do to our codes, we test our skills periodically.After all, self valuation is the best way to progress!
What's the use of skill if it isn't implemented. We at coding.Studio(); work on real time projects in order to solve the problems around us!
Projects Developed by Team coding.Studio();
A knowledge sharing platform exclusively built for the students of VBIT
An effecient webapp which creates a schedule based on student preference for exam preperation
A marketplace built for vbit students exclusively to buy/sell accessories such as drafters and books.
Subscribe to our newsletter for updates about coding.Studio(); like recruitments, projects and upcoming events.


Proverbs 10:1
The proverbs of Solomon.


Rake 0.7.2 Released
Version 0.7.2 supplies a bug fix and a few minor enhancements. In
particular, the new version fixes an incompatibility with the soon to be
released Ruby 1.8.6. We strongly recommend upgrading to Rake 0.7.2 in order
to be compatible with the new version of Ruby.
Changes
Bug Fixes in 0.7.2
There are quite a number of bug fixes in the new 0.7.2 version of Rake:
- Removed dependency on internal fu_xxx functions from FileUtils.
- Error messages are now send to stderr rather than stdout (from Payton
Quackenbush).
- Better error handling on invalid command line arguments (from Payton
Quackenbush).
- Fixed some bugs where the application object was going to the global
appliation instead of using its own data.
- Fixed the method name leak from FileUtils (bug found by Glenn
Vanderburg).
- Added test for noop, bad_option and verbose flags to sh command.
- Added a description to the gem task in GemPackageTask.
- Fixed a bug when rules have multiple prerequisites (patch by Joel
VanderWerf)
- Added the handful of RakeFileUtils to the private
method as well.
New Features in 0.7.2
The following new features are available in Rake version 0.7.2:
- Added square and curly bracket patterns to FileList#include (Tilman
Sauerbeck).
- FileLists can now pass a block to FileList#exclude to exclude files based
on calculated values.
- Added plain filename support to rule dependents (suggested by Nobu Nakada).
- Added pathmap support to rule dependents. In other words, if a pathmap
format (beginning with a ’%’) is given as a Rake rule
dependent, then the name of the depend will be the name of the target with
the pathmap format applied.
- Added a ‘tasks’ method to a namespace to get a list of tasks
associated with the namespace.
- Added tar_command and zip_command options to the Package task.
- The clean task will no longer delete ‘core’ if it is a
directory.
Internal Rake Improvements
The following changes will are mainly internal improvements and
refactorings and have little effect on the end user. But they may be of
interest to the general public.
- Added rcov task and updated unit testing for better code coverage.
- Added a ‘shame’ task to the Rakefile.
- Added rake_extension to handle detection of extension collisions.
- Added a protected ‘require "rubygems"’ to
test/test_application to unbreak cruisecontrol.rb.
- Removed rake_dup. Now we just simply rescue a bad dup.
- Refactored the FileList reject logic to remove duplication.
- Removed if FILE at the end of the rake.rb file.
What is Rake
Rake is a build tool similar to the make program in many ways. But instead
of cryptic make recipes, Rake uses standard Ruby code to declare tasks and
dependencies. You have the full power of a modern scripting language built
right into your build tool.
Availability
The easiest way to get and install rake is via RubyGems …
gem install rake (you may need root/admin privileges)
Otherwise, you can get it from the more traditional places:
Thanks
As usual, it was input from users that drove a alot of these changes. The
following people either contributed patches, made suggestions or made
otherwise helpful comments. Thanks to …
- Payton Quackenbush — For several error handling improvements.
- Glenn Vanderburg — For finding and fixing the method name leak from
FileUtils.
- Joel VanderWerf — for finding and fixing a bug in the handling of
multiple prerequisites.
- Tilman Sauerbeck — For some enhancing FileList to support more
advanced file globbing.
- Nobu Nakada — For suggesting plain file name support to rule
dependents.


Insult Me!
This showcases an implementation of a silly but fun Node.js application built using the Circuit Breaker microservices design pattern. There are three services involved, in addition to the front end which you are currently viewing.


The baby who is crying can receive routine care.
Keep warm
Position the baby skin-to-skin with the mother. The warmth from the mother’s body is one of the best ways to keep a baby warm. Cover the baby with a warm, dry cloth and a cap or other head covering. Otherwise, cover the baby with part of the mother’s clothing. Skin-to-skin contact helps a newborn baby breathe well and initiate breastfeeding. Postpone bathing and weighing and keep the area warm.
Drying can be done on the mother’s abdomen. Place a clean cloth on mother’s abdomen before birth. Position the baby on the cloth and dry thoroughly. Remove the wet cloth, place the baby skin-to-skin with the mother, and cover with a dry cloth.
Check breathing
to assess the baby’s breathing. Listen to the sounds of breathing and look at or feel the movement of the chest.
Check that the baby is breathing quietly and easily or crying. Make sure that the neck
is slightly extended and air can pass freely through the baby’s nose. Be sure that mother and baby are not alone during the first
hours after birth.
Practice key skills
Practice in pairs
- Position the baby skin-to-skin and cover head and body
- Check breathing
Check yourself
Mark the box beside the best answer
Point out the action step Keep warm, check breathing in routine care (green zone).
To improve care in your facility
‐ What are the reasons that some babies do not receive skin-to-
skin care after birth?
‐ Who checks the baby’s breathing and helps mother initiate
breastfeeding?
What to monitor
‐ Do all babies receive skin-to-skin care at birth?
‐ Do all babies initiate breastfeeding in the first hour after birth?


vignettes/taxonomic_resolution_using_lcvplants.Rmd
taxonomic_resolution_using_lcvplants.Rmd
When comparing or merging any list of plant species, for instance for
compiling regional species lists, merging occurrence lists with trait
data or phylogenies, the taxonomic names must be matched to avoid
artificial inflation due to synonyms, data loss or erroneous matches due
to homonyms. The lcvplants
package facilitates this process
by automatizing large-scale taxonomic harmonization of plant names by
fuzzy matching and synonymy resolution against the Leipzig Catalogue of
Vascular Plants as taxonomic backbone.
The Leipzig Catalogue of Vascular Plants (LCVP) is a novel global taxonomic backbone, updating The Plant List, comprising more than 1,300,000 names and 350,000 accepted taxa names. We described the LCVP in detail in the related scientific publication (Freiberg et al, 2020).
You can install lcvplants
from github using the devtools
package You may need to install devtools first, so if you do not have it
installed, run the following code.
install.packages("devtools")
To use lcvplants
you also need the data of the LCVP
package, which you can install in the same way.
devtools::install_github("idiv-biodiversity/lcvplants")
devtools::install_github("idiv-biodiversity/LCVP")
The basic function in lcvplants
is
lcvp_search
. It allows users to input a list of species
name that will be matched to the names in the LCVP data. It returns a
data.frame indicating the full name matched — including the authority
name and removing possible orthographic errors —, the taxonomic status,
the accepted name, family and order.
lcvp_search("Hibiscus vitifolius")
| Search | global.Id | Input.Genus | Input.Epitheton | Rank | Input.Subspecies.Epitheton | Input.Authors | Status | globalId.of.Output.Taxon | Output.Taxon | Family | Order | Literature | Comments |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Hibiscus vitifolius | 604882 | Hibiscus | vitifolius | species | nil | L. | accepted | 604882 | Hibiscus vitifolius L. | Malvaceae | Malvales |
The lcvp_search
algorithm will first try to exactly
match the binomial names provided by the user. If no match is found, it
will try to find the closest name given the maximal transformation cost
defined in the argument max_distance
(see ?base::agrep for
more details). When the argument show_correct = TRUE
a
column is added to the final result indicating whether the binomial name
was exactly matched (TRUE), or if it is misspelled (FALSE).
lcvp_search("Hibiscus vitifoliuse",
max_distance = 0.1,
show_correct = TRUE)
| Search | global.Id | Input.Genus | Input.Epitheton | Rank | Input.Subspecies.Epitheton | Input.Authors | Status | globalId.of.Output.Taxon | Output.Taxon | Family | Order | Literature | Comments | Correct |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Hibiscus vitifoliuse | 604882 | Hibiscus | vitifolius | species | nil | L. | accepted | 604882 | Hibiscus vitifolius L. | Malvaceae | Malvales | FALSE |
If more than one name is fuzzy matched (more than one closest match
found), only the accepted or the first name will be returned. The
function lcvp_fuzzy_search
can be used to return all the
closest results when argument keep_closest = TRUE
, or all
names within the max_distance
defined when
keep_closest = FALSE
.
lcvp_fuzzy_search("Hibiscus vitifoliuse",
max_distance = 0.1,
keep_closest = TRUE)
| global.Id | Input.Genus | Input.Epitheton | Rank | Input.Subspecies.Epitheton | Input.Authors | Status | globalId.of.Output.Taxon | Output.Taxon | Family | Order | Literature | Comments | Name.Distance |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 604882 | Hibiscus | vitifolius | species | nil | L. | accepted | 604882 | Hibiscus vitifolius L. | Malvaceae | Malvales | 1 | ||
| 604883 | Hibiscus | vitifolius | species | nil | Mill. | synonym | 603660 | Hibiscus cannabinus L. | Malvaceae | Malvales | 1 |
Both lcvp_search
and lcvp_fuzzy_search
allows multiple species search. If you need to run for many species
consider setting the argument progress_bar = TRUE
to track
the execution progress.
splist <- c(
"Hibiscus abelmoschus var. betulifolius Mast.",
"Hibiscus abutiloides Willd.",
"Hibiscus aculeatus",
"Hibiscus acuminatus",
"Hibiscus furcatuis"
)
x <- lcvp_search(splist, max_distance = 0.2, show_correct = TRUE)
x
| Search | global.Id | Input.Genus | Input.Epitheton | Rank | Input.Subspecies.Epitheton | Input.Authors | Status | globalId.of.Output.Taxon | Output.Taxon | Family | Order | Literature | Comments | Correct |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Hibiscus abelmoschus var. betulifolius Mast. | 603424 | Hibiscus | abelmoschus | var. | betulifolius | Mast. | synonym | 518 | Abelmoschus moschatus Medik. | Malvaceae | Malvales | TRUE | ||
| Hibiscus abutiloides Willd. | 603426 | Hibiscus | abutiloides | species | nil | Willd. | synonym | 1211289 | Talipariti tiliaceum (L.) Fryxell | Malvaceae | Malvales | TRUE | ||
| Hibiscus aculeatus | 603439 | Hibiscus | aculeatus | species | nil | Walter | accepted | 603439 | Hibiscus aculeatus Walter | Malvaceae | Malvales | TRUE | ||
| Hibiscus acuminatus | 603440 | Hibiscus | acuminatus | species | nil | Cav. | synonym | 683056 | Kosteletzkya acuminata (Cav.) Britten | Malvaceae | Malvales | TRUE | ||
| Hibiscus furcatuis | 603937 | Hibiscus | furcatus | species | nil | Roxb. | accepted | 603937 | Hibiscus furcatus Roxb. | Malvaceae | Malvales | FALSE |
The function lcvp_summary
gives a report on the
searching results using the lcvp_search
function.
Indicating the number of species searched, how many were matched. Among
the matched names, it indicates how many were exactly or fuzzy matched.
Then it checks how many author and infracategory names were exactly
matched. Note that if authors or infracategory is not provided, it will
be considered a no match.
lcvp_summary(x)
#> Species searched: 5
#> Species matched: 5 (100%)
#> Species exactly matched: 4 (80%)
#> Species fuzzy matched: 1 (20%)
#> Authors exactly matched: 2 (40%)
#> Infracategories exactly matched: 5 (100%)
Notice that the warning message indicated that more than one name was matched for some species. You can easily access these species using the following code.
sps_mult <- attr(x, "matched_mult")
sps_mult
#> [1] "Hibiscus aculeatus" "Hibiscus furcatuis"
And you can quickly use the lcvp_fuzzy_search
on
them.
lcvp_fuzzy_search(sps_mult)
| global.Id | Input.Genus | Input.Epitheton | Rank | Input.Subspecies.Epitheton | Input.Authors | Status | globalId.of.Output.Taxon | Output.Taxon | Family | Order | Literature | Comments | Name.Distance |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 603436 | Hibiscus | aculeatus | species | nil | F.Dietr. | synonym | 604041 | Hibiscus heterophyllus Vent. | Malvaceae | Malvales | 0 | ||
| 603437 | Hibiscus | aculeatus | species | nil | G.Don | synonym | 604734 | Hibiscus surattensis L. | Malvaceae | Malvales | 0 | ||
| 603438 | Hibiscus | aculeatus | species | nil | Roxb. | synonym | 603937 | Hibiscus furcatus Roxb. | Malvaceae | Malvales | 0 | ||
| 603439 | Hibiscus | aculeatus | species | nil | Walter | accepted | 603439 | Hibiscus aculeatus Walter | Malvaceae | Malvales | 0 | ||
| 603934 | Hibiscus | furcatus | species | nil | Craib | unresolved | 604561 | Hibiscus rosa-sinensis L. | — | — | 1 | ||
| 603935 | Hibiscus | furcatus | species | nil | Harv. | synonym | 603462 | Hibiscus altissimus Hornby | Malvaceae | Malvales | 1 | ||
| 603936 | Hibiscus | furcatus | species | nil | Mullend. | synonym | 604338 | Hibiscus noldeae Baker.f. | Malvaceae | Malvales | 1 | ||
| 603937 | Hibiscus | furcatus | species | nil | Roxb. | accepted | 603937 | Hibiscus furcatus Roxb. | Malvaceae | Malvales | 1 | ||
| 603940 | Hibiscus | furcatus | species | nil | Wall. | synonym | 603937 | Hibiscus furcatus Roxb. | Malvaceae | Malvales | 1 | ||
| 603941 | Hibiscus | furcatus | species | nil | Willd. | synonym | 604573 | Hibiscus rostellatus Guill. & Perr. | Malvaceae | Malvales | 1 |
Users can search all plant taxa names listed in the “Leipzig
Catalogue of Vascular Plants” (LCVP) by order, family, genus or author,
using the lcvp_group_search
function.
# Search by Genus
x <- lcvp_group_search(c("AA", "Adansonia"), search_by = "Genus")
head(x)
| global.Id | Input.Genus | Input.Epitheton | Rank | Input.Subspecies.Epitheton | Input.Authors | Status | globalId.of.Output.Taxon | Output.Taxon | Family | Order | Literature | Comments |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Aa | argyrolepis | species | nil | (Rchb.f.) Rchb.f. | accepted | 1 | Aa argyrolepis (Rchb.f.) Rchb.f. | Orchidaceae | Asparagales | ||
| 2 | Aa | aurantiaca | species | nil | D.Trujillo | accepted | 2 | Aa aurantiaca D.Trujillo | Orchidaceae | Asparagales | Lankesteriana 2011.11.1 1-8; | |
| 3 | Aa | brevis | species | nil | Schltr. | synonym | 819078 | Myrosmodes breve (Schltr.) Garay | Orchidaceae | Asparagales | ||
| 4 | Aa | calceata | species | nil | (Rchb.f.) Schltr. | accepted | 4 | Aa calceata (Rchb.f.) Schltr. | Orchidaceae | Asparagales | ||
| 5 | Aa | chiogena | species | nil | Schltr. | synonym | 819080 | Myrosmodes chiogena (Schltr.) C.A.Vargas | Orchidaceae | Asparagales | ||
| 6 | Aa | colombiana | species | nil | Schltr. | accepted | 6 | Aa colombiana Schltr. | Orchidaceae | Asparagales | Ann Bot 2009.104.3 403-416; |
Users can choose to keep only certain taxonomic stauts, this includes “accepted”, “synonym”, “unresolved”, and “external”.
# Search by Author and keep only accepted names
x <- lcvp_group_search("Schltr.", search_by = "Author", status = "accepted")
head(x)
| global.Id | Input.Genus | Input.Epitheton | Rank | Input.Subspecies.Epitheton | Input.Authors | Status | globalId.of.Output.Taxon | Output.Taxon | Family | Order | Literature | Comments | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2 | 4 | Aa | calceata | species | nil | (Rchb.f.) Schltr. | accepted | 4 | Aa calceata (Rchb.f.) Schltr. | Orchidaceae | Asparagales | ||
| 4 | 6 | Aa | colombiana | species | nil | Schltr. | accepted | 6 | Aa colombiana Schltr. | Orchidaceae | Asparagales | Ann Bot 2009.104.3 403-416; | |
| 5 | 7 | Aa | denticulata | species | nil | Schltr. | accepted | 7 | Aa denticulata Schltr. | Orchidaceae | Asparagales | ||
| 6 | 8 | Aa | erosa | species | nil | (Rchb.f.) Schltr. | accepted | 8 | Aa erosa (Rchb.f.) Schltr. | Orchidaceae | Asparagales | ||
| 7 | 9 | Aa | fiebrigii | species | nil | (Schltr.) Schltr. | accepted | 9 | Aa fiebrigii (Schltr.) Schltr. | Orchidaceae | Asparagales | ||
| 9 | 14 | Aa | hieronymi | species | nil | (Cogn.) Schltr. | accepted | 14 | Aa hieronymi (Cogn.) Schltr. | Orchidaceae | Asparagales |
The output from lcvp_search
,
lcvp_fuzzy_search
and lcvp_group_search
is a
data.frame
(or list of data.frames) with the following
columns:
If no match is found for one species it will return NA for the columns in the LCVP table. But, if no match is found for all species the function will return NULL and a warning message.
In many situations, researchers want to compare and match two lists
of species name coming from different sources (e.g. phylogenies, spatial
data, trait data, regional lists). The function lcvp_match
can be used for that. It matches and compares two name lists based on
the taxonomic resolution of plant taxa names listed in the LCVP.
# Generate two lists of species name
splist1 <- sample(apply(LCVP::tab_lcvp[2:10, 2:3], 1, paste, collapse = " "))
splist2 <- sample(apply(LCVP::tab_lcvp[11:3, 2:3], 1, paste, collapse = " "))
Ordered is based on the splist1, and followed by non-matched names in splist2.
# Match both lists
x <- lcvp_match(splist1, splist2)
head(x)
| Species.List.1 | Species.List.2 | global.Id | Input.Genus | Input.Epitheton | Rank | Input.Subspecies.Epitheton | Input.Authors | Status | globalId.of.Output.Taxon | Output.Taxon | Family | Order | Literature | Comments | Match.Position.2to1 | Duplicated.Output.Position | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 10 | Aa figueroi | Aa figueroi | 10 | Aa | figueroi | species | nil | Szlach. & S.Nowak | accepted | 10 | Aa figueroi Szlach. & S.Nowak | Orchidaceae | Asparagales | 7 | NA | ||
| 8 | Aa erosa | Aa erosa | 8 | Aa | erosa | species | nil | (Rchb.f.) Schltr. | accepted | 8 | Aa erosa (Rchb.f.) Schltr. | Orchidaceae | Asparagales | 3 | NA | ||
| 5 | Aa chiogena | Aa chiogena | 5 | Aa | chiogena | species | nil | Schltr. | synonym | 819080 | Myrosmodes chiogena (Schltr.) C.A.Vargas | Orchidaceae | Asparagales | 8 | NA | ||
| 7 | Aa denticulata | Aa denticulata | 7 | Aa | denticulata | species | nil | Schltr. | accepted | 7 | Aa denticulata Schltr. | Orchidaceae | Asparagales | 5 | NA | ||
| 4 | Aa calceata | Aa calceata | 4 | Aa | calceata | species | nil | (Rchb.f.) Schltr. | accepted | 4 | Aa calceata (Rchb.f.) Schltr. | Orchidaceae | Asparagales | 6 | NA | ||
| 6 | Aa colombiana | Aa colombiana | 6 | Aa | colombiana | species | nil | Schltr. | accepted | 6 | Aa colombiana Schltr. | Orchidaceae | Asparagales | Ann Bot 2009.104.3 403-416; | 2 | NA |
If include_all = FALSE
, non-matched names in splist2 are
not included. And users can use the column
Match.Position.2to1
to reorder splist2 to match
splist1.
# Match both lists
matchLists <- lcvp_match(splist1, splist2, include_all = FALSE)
splist2_reordered <- splist2[matchLists$Match.Position.2to1]
The lcvp_join
function provides a quicker way to join
two tables based on the taxonomic resolution of plant taxa names listed
in the LCVP. It is inspired by the dplyr::join
function.
The function add the columns from a first table to a second table based
on the list of species name in both tables. It first standardizes the
species names in both tables based on the “Leipzig Catalogue of Vascular
Plants” (LCVP) using the algorithm in lcvp_search
. These
standardized names of both tables are then matched using the algorithm
in lcvp_match
. The argument type
indicates the
kind of join to be done. The option “full” join will keep all species,
“left” return all rows from the first table and drops the non-matches
from the second, “right” do the same for the second table, and “inner”
keep only matched species.
# Create data.frame1
splist1 <- sample(apply(LCVP::tab_lcvp[2:10, 2:3], 1, paste, collapse = " "))
tbl1 <-
data.frame("Species" = splist1, "Trait1" = runif(length(splist1)))
# Create data.frame2
splist2 <- sample(apply(LCVP::tab_lcvp[11:3, 2:3], 1, paste, collapse = " "))
tbl2 <- data.frame(
"Species" = splist2,
"Trait2" = runif(length(splist2)),
"Trait3" = runif(length(splist2)),
"Trait4" = sample(c("a", "b"), length(splist2), replace = TRUE),
"Trait5" = sample(c(TRUE, FALSE), length(splist2), replace = TRUE)
)
head(x)
| Species.List.1 | Species.List.2 | global.Id | Input.Epitheton | Rank | Input.Subspecies.Epitheton | Input.Authors | Status | globalId.of.Output.Taxon | Output.Taxon | Family | Order | Literature | Comments | Trait1 | Trait2 | Trait3 | Trait4 | Trait5 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Aa aurantiaca | NA | 2 | aurantiaca | species | nil | D.Trujillo | accepted | 2 | Aa aurantiaca D.Trujillo | Orchidaceae | Asparagales | Lankesteriana 2011.11.1 1-8; | 0.5655017 | NA | NA | NA | NA | |
| Aa fiebrigii | Aa fiebrigii | 9 | fiebrigii | species | nil | (Schltr.) Schltr. | accepted | 9 | Aa fiebrigii (Schltr.) Schltr. | Orchidaceae | Asparagales | 0.5269464 | 0.5306969 | 0.3474194 | a | FALSE | ||
| Aa figueroi | Aa figueroi | 10 | figueroi | species | nil | Szlach. & S.Nowak | accepted | 10 | Aa figueroi Szlach. & S.Nowak | Orchidaceae | Asparagales | 0.3552532 | 0.9947610 | 0.4730215 | b | FALSE | ||
| Aa colombiana | Aa colombiana | 6 | colombiana | species | nil | Schltr. | accepted | 6 | Aa colombiana Schltr. | Orchidaceae | Asparagales | Ann Bot 2009.104.3 403-416; | 0.8892958 | 0.9176120 | 0.6219212 | a | FALSE | |
| Aa brevis | Aa brevis | 3 | brevis | species | nil | Schltr. | synonym | 819078 | Myrosmodes breve (Schltr.) Garay | Orchidaceae | Asparagales | 0.5377264 | NA | NA | NA | NA | ||
| Aa calceata | Aa calceata | 4 | calceata | species | nil | (Rchb.f.) Schltr. | accepted | 4 | Aa calceata (Rchb.f.) Schltr. | Orchidaceae | Asparagales | 0.2948948 | 0.6416213 | 0.7076298 | a | FALSE |
Because some names may turn out to be synonyms based on the LCVP
taxonomic resolution, users may opt to solve duplicated species names by
summarizing traits given provided functions for common classes of
variables (numeric, character, and logical). This can be done by turning
the option solve_duplicated = TRUE
, and the algorithm will
combine duplicated output names (Output.Taxon
column) based
on users-defined functions to summarize the information.
head(x)
| Species.List.1 | Species.List.2 | global.Id | Input.Epitheton | Rank | Input.Subspecies.Epitheton | Input.Authors | Status | globalId.of.Output.Taxon | Output.Taxon | Family | Order | Literature | Comments | Trait1 | Trait2 | Trait3 | Trait4 | Trait5 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Aa aurantiaca | NA | 2 | aurantiaca | species | nil | D.Trujillo | accepted | 2 | Aa aurantiaca D.Trujillo | Orchidaceae | Asparagales | Lankesteriana 2011.11.1 1-8; | 0.5655017 | NA | NA | NA | NA | |
| Aa fiebrigii | Aa fiebrigii | 9 | fiebrigii | species | nil | (Schltr.) Schltr. | accepted | 9 | Aa fiebrigii (Schltr.) Schltr. | Orchidaceae | Asparagales | 0.5269464 | 0.5306969 | 0.3474194 | a | FALSE | ||
| Aa figueroi | Aa figueroi | 10 | figueroi | species | nil | Szlach. & S.Nowak | accepted | 10 | Aa figueroi Szlach. & S.Nowak | Orchidaceae | Asparagales | 0.3552532 | 0.9947610 | 0.4730215 | b | FALSE | ||
| Aa colombiana | Aa colombiana | 6 | colombiana | species | nil | Schltr. | accepted | 6 | Aa colombiana Schltr. | Orchidaceae | Asparagales | Ann Bot 2009.104.3 403-416; | 0.8892958 | 0.9176120 | 0.6219212 | a | FALSE | |
| Aa brevis | Aa brevis | 3 | brevis | species | nil | Schltr. | synonym | 819078 | Myrosmodes breve (Schltr.) Garay | Orchidaceae | Asparagales | 0.5377264 | NA | NA | NA | NA | ||
| Aa calceata | Aa calceata | 4 | calceata | species | nil | (Rchb.f.) Schltr. | accepted | 4 | Aa calceata (Rchb.f.) Schltr. | Orchidaceae | Asparagales | 0.2948948 | 0.6416213 | 0.7076298 | a | FALSE |
Because some users may want to solve duplicated names outside the
lcvp_join
function, the algorithm is available as an
individual function names lcvp_solve_dups
.
# Create a data.frame with duplicated names and different traits
splist <- sample(apply(LCVP::tab_lcvp[1:100, 2:3], 1, paste, collapse = " "))
search <- lcvp_search(splist)
tbl <- data.frame("Species" = search$Output.Taxon,
"Trait1" = runif(length(splist)),
"Trait2" = sample(c("a", "b"), length(splist), replace = TRUE),
"Trait3" = sample(c(TRUE, FALSE), length(splist), replace = TRUE))
# Solve with default parameters
x <- lcvp_solve_dups(tbl, 1)
head(x)
| Species | Trait1 | Trait2 | Trait3 |
|---|---|---|---|
| Triplophyllum varians (T.Moore) Holttum | 0.9759653 | b | FALSE |
| Abacopteris nudata (Roxb. ex Griff.) S.E.Fawc. & A.R.Sm. | 0.3822746 | a, b | TRUE |
| Aa maderoi Schltr. | 0.3413011 | b | FALSE |
| Breynia brevipes (Müll.Arg.) Chakrab. & N.P.Balakr. | 0.1605878 | b | FALSE |
| Breynia androgyna (L.) Chakrab. & N.P.Balakr. | 0.2620146 | a, b | TRUE |
| Abacopteris insularis K.Iwats. | 0.5997044 | a | FALSE |
Some users may want to use lcvplants functions to perform taxonomic harmonization for many thousands of species and wish to use the entire computational capacity available to speed up the processing time. Check this article, where we show how to run lcvp_search or lcvp_fuzzy_search in parallel in an efficient way to reduce computational time.
Freiberg, M., Winter, M., Gentile, A. et al. LCVP, The Leipzig catalogue of vascular plants, a new taxonomic reference list for all known vascular plants. Sci Data 7, 416 (2020). https://doi.org/10.1038/s41597-020-00702-z