In this notebook, some template code has already been provided for you, and you will need to implement additional functionality to successfully complete this project. You will not need to modify the included code beyond what is requested. Sections that begin with '(IMPLEMENTATION)' in the header indicate that the following block of code will require additional functionality which you must provide. Instructions will be provided for each section, and the specifics of the implementation are marked in the code block with a 'TODO' statement. Please be sure to read the instructions carefully!
Note: Once you have completed all of the code implementations, you need to finalize your work by exporting the Jupyter Notebook as an HTML document. Before exporting the notebook to html, all of the code cells need to have been run so that reviewers can see the final implementation and output. You can then export the notebook by using the menu above and navigating to File -> Download as -> HTML (.html). Include the finished document along with this notebook as your submission.
In addition to implementing code, there will be questions that you must answer which relate to the project and your implementation. Each section where you will answer a question is preceded by a 'Question X' header. Carefully read each question and provide thorough answers in the following text boxes that begin with 'Answer:'. Your project submission will be evaluated based on your answers to each of the questions and the implementation you provide.
Note: Code and Markdown cells can be executed using the Shift + Enter keyboard shortcut. Markdown cells can be edited by double-clicking the cell to enter edit mode.
The rubric contains optional "Stand Out Suggestions" for enhancing the project beyond the minimum requirements. If you decide to pursue the "Stand Out Suggestions", you should include the code in this Jupyter notebook.
In this notebook, you will make the first steps towards developing an algorithm that could be used as part of a mobile or web app. At the end of this project, your code will accept any user-supplied image as input. If a dog is detected in the image, it will provide an estimate of the dog's breed. If a human is detected, it will provide an estimate of the dog breed that is most resembling. The image below displays potential sample output of your finished project (... but we expect that each student's algorithm will behave differently!).
In this real-world setting, you will need to piece together a series of models to perform different tasks; for instance, the algorithm that detects humans in an image will be different from the CNN that infers dog breed. There are many points of possible failure, and no perfect algorithm exists. Your imperfect solution will nonetheless create a fun user experience!
We break the notebook into separate steps. Feel free to use the links below to navigate the notebook.
 Unzip the folder and place it in this project's home directory, at the location /dogImages
.
Download the human dataset. Unzip the folder and place it in the home diretcory, at location /lfw
.
Note: If you are using a Windows machine, you are encouraged to use 7zip to extract the folder.
In the code cell below, we save the file paths for both the human (LFW) dataset and dog dataset in the numpy arrays human_files
and dog_files
.
import numpy as np
from glob import glob
# load filenames for human and dog images
human_files = np.array(glob("lfw/*/*"))
dog_files = np.array(glob("dogImages/*/*/*"))
# print number of images in each dataset
print('There are %d total human images.' % len(human_files))
print('There are %d total dog images.' % len(dog_files))
In this section, we use OpenCV's implementation of Haar feature-based cascade classifiers to detect human faces in images.
OpenCV provides many pre-trained face detectors, stored as XML files on github. We have downloaded one of these detectors and stored it in the haarcascades
directory. In the next code cell, we demonstrate how to use this detector to find human faces in a sample image.
import cv2
import matplotlib.pyplot as plt
%matplotlib inline
# extract pre-trained face detector
face_cascade = cv2.CascadeClassifier('haarcascades/haarcascade_frontalface_alt.xml')
# load color (BGR) image
img = cv2.imread(human_files[0])
# convert BGR image to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# find faces in image
faces = face_cascade.detectMultiScale(gray)
# print number of faces detected in the image
print('Number of faces detected:', len(faces))
# get bounding box for each detected face
for (x,y,w,h) in faces:
# add bounding box to color image
cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)
# convert BGR image to RGB for plotting
cv_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
# display the image, along with bounding box
plt.imshow(cv_rgb)
plt.show()
Before using any of the face detectors, it is standard procedure to convert the images to grayscale. The detectMultiScale
function executes the classifier stored in face_cascade
and takes the grayscale image as a parameter.
In the above code, faces
is a numpy array of detected faces, where each row corresponds to a detected face. Each detected face is a 1D array with four entries that specifies the bounding box of the detected face. The first two entries in the array (extracted in the above code as x
and y
) specify the horizontal and vertical positions of the top left corner of the bounding box. The last two entries in the array (extracted here as w
and h
) specify the width and height of the box.
We can use this procedure to write a function that returns True
if a human face is detected in an image and False
otherwise. This function, aptly named face_detector
, takes a string-valued file path to an image as input and appears in the code block below.
# returns "True" if face is detected in image stored at img_path
def face_detector(img_path):
img = cv2.imread(img_path)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray)
return len(faces) > 0
Question 1: Use the code cell below to test the performance of the face_detector
function.
human_files
have a detected human face? dog_files
have a detected human face?  You will see that our algorithm falls short of this goal, but still gives acceptable performance. We extract the file paths for the first 100 images from each of the datasets and store them in the numpy arrays human_files_short
and dog_files_short
.
Answer: (You can print out your results and/or write your percentages in this cell)
96% have a detected human face 18% of the dog files have a detected human face
from tqdm import tqdm
import functools as func
human_files_short = human_files[:100]
dog_files_short = dog_files[:100]
#-#-# Do NOT modify the code above this line. #-#-#
## TODO: Test the performance of the face_detector algorithm
## on the images in human_files_short and dog_files_short.
print('getting human set')
humanset = [(face_detector(i), i) for i in human_files_short]
print('getting dog set')
dogset = [(face_detector(i), i) for i in dog_files_short]
print('counting totals')
hcount = sum(y[0] == True for y in humanset)
dcount = sum(y[0] == False for y in dogset)
print(f'{hcount} is the percent of correct face classifications')
print(f'{dcount} is the percent of correct dog classifications')
#get our dog images with a human face and our human images without a face, sample 2 them
inhuman = list(filter(lambda x: x[0] == False, humanset))
undog = list(filter(lambda x: x[0] == True, dogset))
def SampleImages(images, x, label):
for i in range(1, x + 1):
index = i - 1
file = str(images[index][1])
print(f'{label} set is wrong due to open cv face classification error.')
img = cv2.imread(file)
cv_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
plt.imshow(img)
plt.show()
print('Now printing errors')
#now print out a sample of X of the incorrect face classifications
SampleImages(inhuman, 2, 'human')
#now print out a sample of X of the incorrect dog classifications
SampleImages(undog, 2, 'dog')
We suggest the face detector from OpenCV as a potential way to detect human images in your algorithm, but you are free to explore other approaches, especially approaches that make use of deep learning :). Please use the code cell below to design and test your own face detection algorithm. If you decide to pursue this optional task, report performance on human_files_short
and dog_files_short
.
### (Optional)
### TODO: Test performance of anotherface detection algorithm.
### Feel free to use as many code cells as needed.
In this section, we use a pre-trained model to detect dogs in images.
The code cell below downloads the VGG-16 model, along with weights that have been trained on ImageNet, a very large, very popular dataset used for image classification and other vision tasks. ImageNet contains over 10 million URLs, each linking to an image containing an object from one of 1000 categories.
import torch
import torchvision.models as models
# define VGG16 model
VGG16 = models.vgg16(pretrained=True)
# check if CUDA is available
use_cuda = torch.cuda.is_available()
# move model to GPU if CUDA is available
if use_cuda:
VGG16 = VGG16.cuda()
Given an image, this pre-trained VGG-16 model returns a prediction (derived from the 1000 possible categories in ImageNet) for the object that is contained in the image.
In the next code cell, you will write a function that accepts a path to an image (such as 'dogImages/train/001.Affenpinscher/Affenpinscher_00001.jpg'
) as input and returns the index corresponding to the ImageNet class that is predicted by the pre-trained VGG-16 model. The output should always be an integer between 0 and 999, inclusive.
Before writing the function, make sure that you take the time to learn how to appropriately pre-process tensors for pre-trained models in the PyTorch documentation.
from PIL import Image
import torchvision.transforms as transforms
def VGG16_predict(img_path):
'''
Use pre-trained VGG-16 model to obtain index corresponding to
predicted ImageNet class for image at specified path
Args:
img_path: path to an image
Returns:
Index corresponding to VGG-16 model's prediction
'''
## TODO: Complete the function.
## Load and pre-process an image from the given img_path
## Return the *index* of the predicted class for that image
#load and preprocess the image
image = Image.open(img_path)
#create our transform
imageTransform = transforms.Compose([transforms.Resize(256), transforms.CenterCrop(224),transforms.ToTensor()])
#now create higher dimensional tensor of image
tensor = imageTransform(image).unsqueeze_(0)
if use_cuda:
tensor = tensor.cuda()
#now evaluate our tensor
VGG16.eval()
eval = VGG16(tensor)
predict = torch.argmax(eval).item()
#return predicted class
return predict
While looking at the dictionary, you will notice that the categories corresponding to dogs appear in an uninterrupted sequence and correspond to dictionary keys 151-268, inclusive, to include all categories from 'Chihuahua'
to 'Mexican hairless'
. Thus, in order to check to see if an image is predicted to contain a dog by the pre-trained VGG-16 model, we need only check if the pre-trained model predicts an index between 151 and 268 (inclusive).
Use these ideas to complete the dog_detector
function below, which returns True
if a dog is detected in an image (and False
if not).
### returns "True" if a dog is detected in the image stored at img_path
def dog_detector(img_path):
lrange = 151
urange = 268
## TODO: Complete the function.
prediction = VGG16_predict(img_path)
return prediction >= lrange and prediction <= urange
Question 2: Use the code cell below to test the performance of your dog_detector
function.
human_files_short
have a detected dog? dog_files_short
have a detected dog?Answer:
### TODO: Test the performance of the dog_detector function
### on the images in human_files_short and dog_files_short.
print('getting human set')
humanset = [(dog_detector(i), i) for i in human_files_short]
print('getting dog set')
dogset = [(dog_detector(i), i) for i in dog_files_short]
print('counting totals')
hcount = sum(y[0] == False for y in humanset)
dcount = sum(y[0] == True for y in dogset)
print(f'{hcount} is the percent of correct human classifications')
print(f'{dcount} is the percent of correct dog classifications')
We suggest VGG-16 as a potential network to detect dog images in your algorithm, but you are free to explore other pre-trained networks (such as Inception-v3, ResNet-50, etc). Please use the code cell below to test other pre-trained PyTorch models. If you decide to pursue this optional task, report performance on human_files_short
and dog_files_short
.
### (Optional)
### TODO: Report the performance of another pre-trained network.
### Feel free to use as many code cells as needed.
Now that we have functions for detecting humans and dogs in images, we need a way to predict breed from images. In this step, you will create a CNN that classifies dog breeds. You must create your CNN from scratch (so, you can't use transfer learning yet!), and you must attain a test accuracy of at least 10%. In Step 4 of this notebook, you will have the opportunity to use transfer learning to create a CNN that attains greatly improved accuracy.
We mention that the task of assigning breed to dogs from images is considered exceptionally challenging. To see why, consider that even a human would have trouble distinguishing between a Brittany and a Welsh Springer Spaniel.
| Brittany | Welsh Springer Spaniel |
|---|---|
It is not difficult to find other dog breed pairs with minimal inter-class variation (for instance, Curly-Coated Retrievers and American Water Spaniels).
| Curly-Coated Retriever | American Water Spaniel |
|---|---|
Likewise, recall that labradors come in yellow, chocolate, and black. Your vision-based algorithm will have to conquer this high intra-class variation to determine how to classify all of these different shades as the same breed.
| Yellow Labrador | Chocolate Labrador | Black Labrador |
|---|---|---|
We also mention that random chance presents an exceptionally low bar: setting aside the fact that the classes are slightly imabalanced, a random guess will provide a correct answer roughly 1 in 133 times, which corresponds to an accuracy of less than 1%.
Remember that the practice is far ahead of the theory in deep learning. Experiment with many different architectures, and trust your intuition. And, of course, have fun!
Use the code cell below to write three separate data loaders for the training, validation, and test datasets of dog images (located at dogImages/train
, dogImages/valid
, and dogImages/test
, respectively). You may find this documentation on custom datasets to be a useful resource. If you are interested in augmenting your training and/or validation data, check out the wide variety of transforms!
import os
from torchvision import datasets
from torchvision import transforms
import torch
import torchvision.models as models
from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = True
### TODO: Write data loaders for training, validation, and test sets
## Specify appropriate transforms, and batch_sizes
crop_constant = 224
resize_constant = 250
rotation_constant = 15
# a lot of CNN architectures seem to include image normalization as a way of training faster, so we include this in our transforms
normalize_transform = transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])
test_transform = transforms.Compose([transforms.Resize(size=resize_constant), transforms.CenterCrop(crop_constant), transforms.ToTensor(), normalize_transform])
validation_transform = transforms.Compose([transforms.Resize(size=resize_constant),transforms.CenterCrop(crop_constant), transforms.ToTensor(), normalize_transform])
train_transform = transforms.Compose([transforms.Resize(size=resize_constant),
transforms.CenterCrop(crop_constant),
transforms.RandomRotation(rotation_constant),
transforms.RandomHorizontalFlip(),
transforms.RandomVerticalFlip(),
transforms.ToTensor(),
normalize_transform])
test_set = datasets.ImageFolder('dogImages/test', transform = test_transform)
validation_set = datasets.ImageFolder('dogImages/valid', transform = validation_transform)
train_set = datasets.ImageFolder('dogImages/train', transform = train_transform)
train_loader = torch.utils.data.DataLoader(train_set,
batch_size=64,
shuffle=True,
num_workers=0)
validation_loader = torch.utils.data.DataLoader(validation_set,
batch_size=64,
shuffle=True,
num_workers=0)
test_loader = torch.utils.data.DataLoader(test_set,
batch_size=16,
shuffle=False,
num_workers=0)
Question 3: Describe your chosen procedure for preprocessing the data.
Answer: I decided to pick a 224 x 224 center cropped image as my input tensor because this seemed to tbe default that people were suggesting for a standard VGG19 architecture.
My code does a center crop to pre-process the images to the appropriate size. For the training set, I also add in a random rotation, random horizontal flip, random vertical flip. This is so that there will be a random re-orientation of the image with p 0.5 on horizontal and vertical flilp and a slight rotation of the image based off of the arbitrarily chosen costant of 15 degrees.
I decided to augment the dataset in this way so that the deep neural network would have more randomness in its data set and hopefully be less likely to overfit this training data.
Finally, I added in a normalization step because I read that this was useful for training conv. nets.
Create a CNN to classify dog breed. Use the template in the code cell below.
import torch.nn as nn
import torch.nn.functional as F
# check if CUDA is available
use_cuda = torch.cuda.is_available()
# define the CNN architecture
class Net(nn.Module):
### TODO: choose an architecture, and complete the class
###We're picking a modified VGG 19
def __init__(self):
super(Net, self).__init__()
## Define layers of a CNN
## 20% chance of dropout when training
self.number_classes = 133
self.dropout_constant = 0.3
#five convolutional layers like in VGG 19
self.c1 = nn.Conv2d(3, 64, kernel_size=3, padding=1)
#self.c12 = nn.Conv2d(64, 64, kernel_size=3, padding=1)
#convolutional layer 2
self.c2 = nn.Conv2d(64, 128, kernel_size=3, padding=1)
#self.c22 = nn.Conv2d(128, 128, kernel_size=3, padding=1)
#layer 3
self.c3 = nn.Conv2d(128, 256, kernel_size=3, padding=1)
#self.c32 = nn.Conv2d(256, 256, kernel_size=3, padding=1)
#layer 4
self.c4 = nn.Conv2d(256, 512, kernel_size=3, padding=1)
#self.c42 = nn.Conv2d(512, 512, kernel_size=3, padding=1)
#Layer 5
self.c5 = nn.Conv2d(512, 512, kernel_size=3, padding=1)
#self.c52 = nn.Conv2d(512, 512, kernel_size=3, padding=1)
## now define max pooling layer
self.max_pool = nn.MaxPool2d(2, 2)
## now define fully connected layers corresponding to image classes
self.fc1 = nn.Linear(7 * 512 * 7, 1280)
self.fc2 = nn.Linear(1280, self.number_classes)
#self.fc3 = nn.Linear(256, self.number_classes)
self.dropout = nn.Dropout(self.dropout_constant)
def forward(self, x):
## Define forward behavior
x = F.relu(self.c1(x))
##x = F.relu(self.c12(x))
x = self.max_pool(x)
x = F.relu(self.c2(x))
##x = F.relu(self.c22(x))
x = self.max_pool(x)
x = F.relu(self.c3(x))
#x = F.relu(self.c32(x))
#We exclude most of the layers (all but one) in each convolutional set in order to avoid running out of memory with CUDA
#x = F.relu(self.c32(x))
#x = F.relu(self.c32(x))
x = self.max_pool(x)
x = F.relu(self.c4(x))
#x = F.relu(self.c42(x))
#x = F.relu(self.c42(x))
#x = F.relu(self.c42(x))
x = self.max_pool(x)
x = F.relu(self.c5(x))
#x = F.relu(self.c52(x))
#x = F.relu(self.c52(x))
#x = F.relu(self.c52(x))
x = self.max_pool(x)
#now we transition to the fully connected layers
x = x.view(x.size(0), -1)
x = F.relu(self.fc1(x))
x = self.dropout(x)
x = self.fc2(x)
return x
#-#-# You so NOT have to modify the code below this line. #-#-#
# instantiate the CNN
model_scratch = Net()
# move tensors to GPU if CUDA is available
if use_cuda:
model_scratch.cuda()
Question 4: Outline the steps you took to get to your final CNN architecture and your reasoning at each step.
Answer: I decided to take from the VGG19 architecture in order to train this neural net. I understand this may have been overkill for a 10% accuracy, but I wanted to try one of the more complicated convolutional neural net architectures on the from-scratch portion to see how long it would take to train.
The architecture is pretty simple - multiple convolutional layers followed by max pooling layers, culminating in a fully connected layer with dimensionality of 7 x 7 x 512. (512 is the final convolutional layer, we have 2 ^ 5 max pooling layers on an initial image size of 224 x 224, which gives us 7 x 7 fo this piece).
Of course, one thing I simplified from the VGG19 architecture is taking out some of the convolutional layers, this is so that the model could actually train on my machine without running out of memory...
After all this, we have two fully connected layers. Normally it would be three, but due to memory constraints (and strangely, the model trained better with two, I bumped it down to two layers) I modified these to reduce to a final class score of 133 since this is the number of classes we need to classify.
The only other piece I added was a dropout layer for each fully connected layer to evenly train all parts of the MLP. This helped avoid overtraining.
Use the next code cell to specify a loss function and optimizer. Save the chosen loss function as criterion_scratch
, and the optimizer as optimizer_scratch
below.
import torch.optim as optim
### TODO: select loss function
criterion_scratch = nn.CrossEntropyLoss()
### TODO: select optimizer
optimizer_scratch = optim.Adam(model_scratch.parameters(), lr = 0.001)
Train and validate your model in the code cell below. Save the final model parameters at filepath 'model_scratch.pt'
.
import numpy as np
def train(n_epochs, loaders, model, optimizer, criterion, use_cuda, save_path):
"""returns trained model"""
# initialize tracker for minimum validation loss
valid_loss_min = np.Inf
for epoch in range(1, n_epochs+1):
# initialize variables to monitor training and validation loss
train_loss = 0.0
valid_loss = 0.0
###################
# train the model #
###################
model.train()
for batch_idx, (data, target) in enumerate(loaders['train']):
# move to GPU
if use_cuda:
data, target = data.cuda(), target.cuda()
## find the loss and update the model parameters accordingly
## record the average training loss, using something like
## train_loss = train_loss + ((1 / (batch_idx + 1)) * (loss.data - train_loss))
#zero out the gradient
optimizer.zero_grad()
#forward pass and calculate loss
forward = model(data)
loss = criterion(forward, target)
loss.backward()
#perform optimization
optimizer.step()
#finally, lets update the training loss
train_loss = train_loss + ((1 / (batch_idx + 1)) * (loss.data - train_loss))
######################
# validate the model #
######################
model.eval()
for batch_idx, (data, target) in enumerate(loaders['valid']):
# move to GPU
if use_cuda:
data, target = data.cuda(), target.cuda()
## update the average validation loss
with torch.no_grad():
forward = model(data)
loss = criterion(forward, target)
valid_loss = valid_loss + ((1 / (batch_idx + 1)) * (loss.data - valid_loss))
# print training/validation statistics
print('Epoch: {} \tTraining Loss: {:.6f} \tValidation Loss: {:.6f}'.format(
epoch,
train_loss,
valid_loss
))
## TODO: save the model if validation loss has decreased
if valid_loss < valid_loss_min:
print(f'Validation loss is less than minimum. Saving model. Loss is: {valid_loss}')
torch.save(model.state_dict(), save_path)
valid_loss_min = valid_loss
# return trained model
return model
loaders_scratch = {'train' : train_loader,
'valid' : validation_loader,
'test' : test_loader}
# train the model
model_scratch = train(22, loaders_scratch, model_scratch, optimizer_scratch,
criterion_scratch, use_cuda, 'model_scratch.pt')
# load the model that got the best validation accuracy
model_scratch.load_state_dict(torch.load('model_scratch.pt'))
Try out your model on the test dataset of dog images. Use the code cell below to calculate and print the test loss and accuracy. Ensure that your test accuracy is greater than 10%.
import numpy as np
def test(loaders, model, criterion, use_cuda):
#empty cuda cache so that we can run this without out of memory exceptions
torch.cuda.empty_cache()
# monitor test loss and accuracy
test_loss = 0.
correct = 0.
total = 0.
model.eval()
for batch_idx, (data, target) in enumerate(loaders['test']):
# move to GPU
if use_cuda:
data, target = data.cuda(), target.cuda()
# forward pass: compute predicted outputs by passing inputs to the model
output = model(data)
# calculate the loss
loss = criterion(output, target)
# update average test loss
test_loss = test_loss + ((1 / (batch_idx + 1)) * (loss.data - test_loss))
# convert output probabilities to predicted class
pred = output.data.max(1, keepdim=True)[1]
# compare predictions to true label
correct += np.sum(np.squeeze(pred.eq(target.data.view_as(pred))).cpu().numpy())
total += data.size(0)
print('Test Loss: {:.6f}\n'.format(test_loss))
print('\nTest Accuracy: %2d%% (%2d/%2d)' % (
100. * correct / total, correct, total))
print('assigning loaders')
#load loaders scratch
loaders_scratch = {'train' : train_loader,
'valid' : validation_loader,
'test' : test_loader}
print('loading model from model_scratch.pt')
#load model from model_scratch.pt
model_scratch.load_state_dict(torch.load('model_scratch.pt'))
print('testing model')
# call test function
test(loaders_scratch, model_scratch, criterion_scratch, use_cuda)
You will now use transfer learning to create a CNN that can identify dog breed from images. Your CNN must attain at least 60% accuracy on the test set.
Use the code cell below to write three separate data loaders for the training, validation, and test datasets of dog images (located at dogImages/train
, dogImages/valid
, and dogImages/test
, respectively).
If you like, you are welcome to use the same data loaders from the previous step, when you created a CNN from scratch.
## TODO: Specify data loaders
import os
from torchvision import datasets
from torchvision import transforms
import torch
import torchvision.models as models
from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = True
### TODO: Write data loaders for training, validation, and test sets
## Specify appropriate transforms, and batch_sizes
crop_constant = 224
resize_constant = 250
rotation_constant = 15
# a lot of CNN architectures seem to include image normalization as a way of training faster, so we include this in our transforms
normalize_transform = transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])
test_transform = transforms.Compose([transforms.Resize(size=resize_constant), transforms.CenterCrop(crop_constant), transforms.ToTensor(), normalize_transform])
validation_transform = transforms.Compose([transforms.Resize(size=resize_constant),transforms.CenterCrop(crop_constant), transforms.ToTensor(), normalize_transform])
train_transform = transforms.Compose([transforms.Resize(size=resize_constant),
transforms.CenterCrop(crop_constant),
transforms.RandomRotation(rotation_constant),
transforms.RandomHorizontalFlip(),
transforms.RandomVerticalFlip(),
transforms.ToTensor(),
normalize_transform])
test_set = datasets.ImageFolder('dogImages/test', transform = test_transform)
validation_set = datasets.ImageFolder('dogImages/valid', transform = validation_transform)
train_set = datasets.ImageFolder('dogImages/train', transform = train_transform)
train_loader = torch.utils.data.DataLoader(train_set,
batch_size=64,
shuffle=True,
num_workers=0)
validation_loader = torch.utils.data.DataLoader(validation_set,
batch_size=64,
shuffle=True,
num_workers=0)
test_loader = torch.utils.data.DataLoader(test_set,
batch_size=16,
shuffle=False,
num_workers=0)
Use transfer learning to create a CNN to classify dog breed. Use the code cell below, and save your initialized model as the variable model_transfer
.
import torchvision.models as models
import torch.nn as nn
dropout_constant = 0.3
## TODO: Specify model architecture
use_cuda = torch.cuda.is_available()
#download VGG19 pretrained model
model_transfer = models.vgg19(pretrained=True)
for param in model_transfer.parameters():
param.requires_grad = False
#now define the fully connected part of the network architecture
final_model = nn.Sequential(nn.Linear(512 * 7 * 7, 1280),
nn.ReLU(),
nn.Dropout(dropout_constant),
nn.Linear(1280, len(train_set.classes)))
#replace our transfer model classifier with our new classifier
model_transfer.classifier = final_model
if use_cuda:
model_transfer = model_transfer.cuda()
Question 5: Outline the steps you took to get to your final CNN architecture and your reasoning at each step. Describe why you think the architecture is suitable for the current problem.
Answer: :
I have assumed that because most of the classification of a convolutional neural net architecture like VGG19 is based off of generalizations of shapes, colors, etc. that most of the pre-training used to train VGG19 will be useful in creating a dog classifier.
Because of this, I use all of the parts of the pre-existing VGG19 architecture, the only difference being replacing the classifier portion of that model with my own.
The classifier portion was modeled after my own modified convolutional neural network architecture, with an input of 512 7 7, connected to a final layer with an output of 133.
Use the next code cell to specify a loss function and optimizer. Save the chosen loss function as criterion_transfer
, and the optimizer as optimizer_transfer
below.
import torch.optim as optim
criterion_transfer = nn.CrossEntropyLoss()
optimizer_transfer = optim.Adam(model_transfer.classifier.parameters(), lr = 0.001)
Train and validate your model in the code cell below. Save the final model parameters at filepath 'model_transfer.pt'
.
## Redefine test and train (copied from above) so that we can execute these without re-training our scratch network
import numpy as np
def train(n_epochs, loaders, model, optimizer, criterion, use_cuda, save_path):
"""returns trained model"""
# initialize tracker for minimum validation loss
valid_loss_min = np.Inf
for epoch in range(1, n_epochs+1):
# initialize variables to monitor training and validation loss
train_loss = 0.0
valid_loss = 0.0
###################
# train the model #
###################
model.train()
for batch_idx, (data, target) in enumerate(loaders['train']):
# move to GPU
if use_cuda:
data, target = data.cuda(), target.cuda()
## find the loss and update the model parameters accordingly
## record the average training loss, using something like
## train_loss = train_loss + ((1 / (batch_idx + 1)) * (loss.data - train_loss))
#zero out the gradient
optimizer.zero_grad()
#forward pass and calculate loss
forward = model(data)
loss = criterion(forward, target)
loss.backward()
#perform optimization
optimizer.step()
#finally, lets update the training loss
train_loss = train_loss + ((1 / (batch_idx + 1)) * (loss.data - train_loss))
######################
# validate the model #
######################
model.eval()
for batch_idx, (data, target) in enumerate(loaders['valid']):
# move to GPU
if use_cuda:
data, target = data.cuda(), target.cuda()
## update the average validation loss
with torch.no_grad():
forward = model(data)
loss = criterion(forward, target)
valid_loss = valid_loss + ((1 / (batch_idx + 1)) * (loss.data - valid_loss))
# print training/validation statistics
print('Epoch: {} \tTraining Loss: {:.6f} \tValidation Loss: {:.6f}'.format(
epoch,
train_loss,
valid_loss
))
## TODO: save the model if validation loss has decreased
if valid_loss < valid_loss_min:
print(f'Validation loss is less than minimum. Saving model. Loss is: {valid_loss}')
torch.save(model.state_dict(), save_path)
valid_loss_min = valid_loss
# return trained model
return model
def test(loaders, model, criterion, use_cuda):
#empty cuda cache so that we can run this without out of memory exceptions
torch.cuda.empty_cache()
# monitor test loss and accuracy
test_loss = 0.
correct = 0.
total = 0.
model.eval()
for batch_idx, (data, target) in enumerate(loaders['test']):
# move to GPU
if use_cuda:
data, target = data.cuda(), target.cuda()
# forward pass: compute predicted outputs by passing inputs to the model
output = model(data)
# calculate the loss
loss = criterion(output, target)
# update average test loss
test_loss = test_loss + ((1 / (batch_idx + 1)) * (loss.data - test_loss))
# convert output probabilities to predicted class
pred = output.data.max(1, keepdim=True)[1]
# compare predictions to true label
correct += np.sum(np.squeeze(pred.eq(target.data.view_as(pred))).cpu().numpy())
total += data.size(0)
print('Test Loss: {:.6f}\n'.format(test_loss))
print('\nTest Accuracy: %2d%% (%2d/%2d)' % (
100. * correct / total, correct, total))
n_epochs = 12
# train the model
loaders_transfer = {'train' : train_loader,
'valid' : validation_loader,
'test' : test_loader}
model_transfer = train(n_epochs, loaders_transfer, model_transfer, optimizer_transfer, criterion_transfer, use_cuda, 'model_transfer.pt')
# load the model that got the best validation accuracy (uncomment the line below)
model_transfer.load_state_dict(torch.load('model_transfer.pt'))
Try out your model on the test dataset of dog images. Use the code cell below to calculate and print the test loss and accuracy. Ensure that your test accuracy is greater than 60%.
test(loaders_transfer, model_transfer, criterion_transfer, use_cuda)
Write a function that takes an image path as input and returns the dog breed (Affenpinscher
, Afghan hound
, etc) that is predicted by your model.
### TODO: Write a function that takes a path to an image as input
### and returns the dog breed that is predicted by the model.
# list of class names by index, i.e. a name can be accessed like class_names[0]
class_names = [item[4:].replace("_", " ") for item in train_set.classes]
#First, re-load our state transfer network for use to predict the dog breed
model_transfer.load_state_dict(torch.load('model_transfer.pt'))
def predict_breed_transfer(img_path):
# load the image and return the predicted breed
image = Image.open(img_path)
#Transform our image similarly to how we transformed them with data loaders in our previous step...
image_transforms = transforms.Compose([transforms.Resize(250), transforms.CenterCrop(224), transforms.ToTensor()])
tensor_data = image_transforms(image)
#now unqueeze the data
tensor_data = tensor_data.unsqueeze_(0)
if use_cuda:
tensor_data = tensor_data.cuda()
#now evaluate using our convolutional network...
with torch.no_grad():
output = model_transfer(tensor_data)
class_result = torch.argmax(output).item()
return class_names[class_result]
Write an algorithm that accepts a file path to an image and first determines whether the image contains a human, dog, or neither. Then,
You are welcome to write your own functions for detecting humans and dogs in images, but feel free to use the face_detector
and human_detector
functions developed above. You are required to use your CNN from Step 4 to predict dog breed.
Some sample output for our algorithm is provided below, but feel free to design your own user experience!
### TODO: Write your algorithm.
### Feel free to use as many code cells as needed.
def run_app(img_path):
## handle cases for a human face, dog, and neither
#use face detector for human, even though we know its far from perfect
is_human = face_detector(img_path)
is_dog = dog_detector(img_path)
image = Image.open(img_path)
if is_dog:
print('Dog detected!')
plt.imshow(image)
plt.show()
breed = predict_breed_transfer(img_path)
print(f'Dog breed is a .... {breed}')
elif is_human:
print('Human detected!')
plt.imshow(image)
plt.show()
breed = predict_breed_transfer(img_path)
print(f'You look like a .... {breed}')
else:
plt.imshow(image)
plt.show()
print(f'Sorry, this image is neither human nor dog')
In this section, you will take your new algorithm for a spin! What kind of dog does the algorithm think that you look like? If you have a dog, does it predict your dog's breed accurately? If you have a cat, does it mistakenly think that your cat is a dog?
Test your algorithm at least six images on your computer. Feel free to use any images you like. Use at least two human and two dog images.
Question 6: Is the output better than you expected :) ? Or worse :( ? Provide at least three possible points of improvement for your algorithm.
Answer: (Three possible points for improvement)
I would say the output is worse than I expected. For one, the face detector does not seem to work as well as I'd like it to, so I would probably transfer-learn train a different neural network with better accuracy or pick a better network than the CV2 face detector that we used.
At first glance, I thought that the dog classifier was pretty off... I did a google image search and picked a doberman pinscher, pitbull, and dachshund, and only the dacshund was classified correctly. However, it seems that the misclassified breeds are VERY close to the ones I picked in appearance. I might add more MLP layers and data transforms to try and get a lower validation set error on my data... As well as perhaps including more training data that distinguishes between these breeds.
Another improvement to the algorithm could be made by substituting VGG19 with a more modern deep learning architecture, like inception.
## TODO: Execute your algorithm from Step 6 on
## at least 6 images on your computer.
## Feel free to use as many code cells as needed.
import cv2
import matplotlib.pyplot as plt
import numpy as np
from glob import glob
my_files = np.array(glob("customImages/*"))
## suggested code, below
for file in my_files:
run_app(file)


Manage your availability, and tell people how you want to be contacted
In a meeting? Prefer receiving texts? Tell your contacts how to reach you.
Want your friends to Skype you and your clients to call you? Customize access for each group.
New phone number? Simply update your profile. Your contacts always have your latest information.
Tired of "Hey, it's me... call me back." voicemails? Press a button to ask someone to contact you.
iPhone & Android versions coming soon! Stay tuned...
You hear so much about how instantly reachable we all are, how hyperconnected, with our smartphones, laptops, tablets and such. But the maddening truth is that we've become so accessible we're often inaccessible, the process of getting to any of us more tortured and tortuous than ever.
There are up to a dozen possible routes, and the direct one versus the scenic one versus the loop-de-loop versus the dead end changes from person to person. If you're not dealing with your closest business associates or friends, whose territory and tics you've presumably learned, you're lost.
Frank Bruni, New York Times


--- layout: post title: Rails Button Links In Embedded Forms disqus_id: /2010/01/05/rails-button-links-in-embedded-forms/ categories: - heuristics - ruby-rails ---
This is one I have had sitting around for almost 3 years now in my toolbox and thought I would share. Have you ever had complicated rails forms and needed simple form buttons that just took you to a simple link? Were you bitten by the button_to
helper code because it generates another form inside of a form? If so, here is a simple rails view helper I made that creates simple button links for embedded forms by making an input with a javascript function. Tag soup you ask, hell yeah, but worth if if you need it.


We stand their for our elders and spend time with them. Exploring their dream worlds in our mind and we accomplish this things whic is possible to make a smiley on our elderly face. We laugh, we play, we stand, we run, we talk, we shout, we are there always for you and your stories you old age people. We THE ADVISORY pray for your HEALTHY, INDEPENDENT, AND BETTER life. As, Life is a party!
We do counselling, advise elders for their healthy and fit body. We live your every moment either happy or sad.


Board books
Peppa’s pumpkin party
A jolly little board book filled with favourite characters from the TV programme.
Pumpkin party!
Toddlers who love the TV animation of Beatrix Potter’s Peter Rabbit will toddle along to borrow this book, even though there are rather a lot of words in it!
Boo!: a book of spooky surprises by Jonathan Litton
Yet another board book about pumpkins. This one would be great for very young children. It’s sturdy, brightly coloured and has die cut eyes on every page.
Picture books
Funnybones by Janet and Allen Ahlberg
Those playful skeletons are still in occupation down a dark, dark staircase in a dark, dark house. They’ve been there since 1980, but I suppose in skeleton years, 35 years is no time at all. If you enjoy this first episode, try Mystery tour and The ghost train.
Room on the broom by Julia Donaldson
An introduction in rhyme, witches, broomsticks and more. But be warned, it is rather long.
Spells by Emily Gravett
A beautifully illustrated book about a frog who uses the jumbled pages of a spells book to become a handsome prince.
Spot’s spooky fun by Eric Hill
Spooky it isn’t, despite the title, but then Spot is just too cute to be spooky, even when he dresses up.
Angelina’s Halloween by Katharine Holabird
Angelina goes trick or treating in beautiful costumes.
The haunted house by Kazuno Kohara
In a big old house at the edge of town, a little girl makes short work of the ghosts in this beautiful and unusual picture book.
Happy Halloween, Witch’s Cat! by Harriet Muncaster
A witch’s cat wonders what costume to wear for trick or treating on Halloween in this beautifully illustrated picture book, aimed originally at a US audience.
Ten Spooky skeletons by Garry Parsons
A counting picture book with cut outs for little fingers and a double page glow-in-the-dark ending.
Scaredy-cat, Splat! by Rob Scotton
Splat the cat is back, and this time he’s scared of mice. A perfect picture book for Halloween or anytime of year.
Winnie’s haunted house by Valerie Thomas and Korky Paul
“When Wilbur the cat accidentally smashes one of Winnie’s vases he hides to evade suspicion and Winnie begins to think her house must be haunted. She instigates what she thinks is a spell to fix a haunted house but ends up instead filling her house with skeletons, ghosts, spiders, bats and owls.” The Times calls it “pure magic” and we think it would go down well with KS1.
Meg and the dragon by David Walser and Jan Pienkowski
“Meg is getting ready for her Halloween tea when disaster strikes and her teapot breaks! Meg uses her spells to magic a new teapot, but as usual, all does not go to plan.”
Short chapter books
Winnie goes batty by Laura Owen and Korky Paul
We’ve got lots of silly picture books about Winnie, a witch with a black cat, a cauldron and extraordinary legs, but this book is one for children to read to themselves.
Horrid Henry tricks and treats by Francesca Simon
Imagine what Horrid Henry could be like if he were supposed to be playing tricks and eating loads of sweets.
Junior novels
The Moon-Faced Ghoul-Thing by Barry Hutchison
If you followed Benjamin Blank’s adventures in The Shark-Headed Bear Thing or The Swivel-Eyed Ogre Thing, you’ll know to expect a slightly crazy and very funny 15th century world.
Junior non-fiction
It’s fun to draw creepy spooks by Mark Bergin
Bags of inspiration for how to draw skeletons, werewolves and all manner of spooky stuff.
10 minute Halloween crafts by Annalees Lim
A practical and well-illustrated book of simple craft projects to make with younger children.
 A ghost that goes ‘BOO’ and you!”


My name is Kevin! When I'm not clacking away on the computer, I love to paint and go mountain biking. I love the outdoors and I love to walk my dog in the park, although I am known to endulge in the occasional bindge-watch-as-much-Netflix-as-possible session. I'm also an avid PC gamer when time allows. I enjoy seeing new cities and I hope to travel as much as I can in the future.


| Alert Attribute | Details |
|---|---|
| Medium | |
| URL | http://localhost:7000/help-to-save/verify-email |
| Description | This page contains an error/warning message that may disclose sensitive information like the location of the file that produced the unhandled exception. This information can be used to launch further attacks against the web application. The alert could be a false positive if the error message is found inside a documentation page. |
| Evidence | HTTP/1.1 500 Internal Server Error |


{% extends "base.html" %} {% block body_class %}template-404{% endblock %} {% block content %}
If you typed the web address, check it is correct.
If you pasted the web address, check you copied the entire address.
If the web address is correct or you selected a link or button, contact the Digital Workspace team to let us know.
{% endblock %}


FilmedIn
Sign Up!
Log In!
ERROR!
There were one or more errors in your submission.
Login!
×
That's not the right password. Please try again or click 'Forgot Password to request for a new one'
Forgot Password
×
SignUp!
×


{% extends "_layouts/base.html" %} {% block title %}Register | {{ config.EVE.auth_name }}{% endblock %} {% block content %}
Welcome to {{ config.EVE.auth_name }}!
In order to access our services, we need you to create an account so you can join us and access the different services we offer.
Who are you?
I'm in {{ config.EVE.auth_name }} at this very moment I have been accepted to {{ config.EVE.auth_name }} I am an ally of {{ config.EVE.auth_name }}


Examples:
Counter
Counter before update
Clock
Bind
Todo list
Form
SVG
SVG animation
Mount/unmount component at runtime
Mount/unmount component at runtime without destroy it
Before mount/unmount component
[Animation] Mount/unmount component at runtime
[Animation] Mount/unmount component at runtime without destroy it
[Animation] Show/hide element
[Animation] Show/hide element with Magic Animation
Player controls
Player controls using actions
Doz component as Web Component


native Robinia pollinators species invasive urban plant flower alien plants urbanisation Urban pollinator plant-pollinator Ecology interactions Cytisus {border-style effects hovering study different flowers px;} #ffffff;} ./s--- bee bees vs. attractiveness significantly impacts range Ecol floral habitats increasing nectar wild communities visits (Fig. access attractive blossom decision environmental number pseudoacacia using ./-. Berlin Biological Conservation However, Lett Pollinator affect compared food immediate networks px; resources stressors {font-family (). Invasive Kowarik area font-size higher impervious levels pollen pollination tree visit visitation ./j.-...x Fig. Global Society Urbanisation access, change changes city data double; ecosystems flowering honey pollinators, related role significant similar sites studies time (), (Appendix Applied Buchholz Department Ecological Ecosystems Environmental Insect Interactions Nature Plant Vila abundance attracted border-color border-width cities common conditions did ecological ecosystem environments gradient habitat important including information insect interaction meta-analysis mutualisms natural presence site solid; taxa total trees types urbanisation. use {font-style {font-weight (Chi (Springer, ./journal.pone. Berlin, Biodiversity Biology Climate Commons Creative Diversity Europe Flower Functional GLMM; Gaussian Native P = .; Royal Sascha Sciences Senate Trends behaviour biological bold branches bumblebees butterflies changing climate courier; decreased density der df = , direct effect em; evolutionary example, flowers. foraging gardens highly hovered included increase indicated interactions. invasion landscape landscapes long lower major modulates modulating morphology novel period processes provided pt;} quality rate rates requirements result results settings success = .,


Democrats hold on to Louisiana governor’s seat despite Trump’s best efforts
John Bel Edwards is the Democrats’ only governor in the Deep South.

Mr Edwards is the Democrats’ only governor in the Deep South and his win delivers a blow to US President Donald Trump, who has suffered another gubernatorial loss this year.
In the heart of Trump country, the moderate Mr Edwards cobbled together enough cross-party support with his focus on bipartisan, state-specific issues to defeat Republican businessman Eddie Rispone, getting about 51% of the vote.
Coming after a defeat in the Kentucky governor’s race and sizeable losses in Virginia’s legislative races, the Louisiana result seems certain to rattle Republicans as they head into the 2020 presidential election. Mr Trump fought to return the seat to the Republicans, making three trips to Louisiana to rally against Mr Edwards.
In a victory rally late on Saturday, Mr Edwards thanked supporters. He said: “Tonight the people of Louisiana have chosen to chart their own path.”
Mr Trump had made the run-off election between Mr Edwards and Mr Rispone a test of his own popularity and political prowess heading into the 2020 presidential race. On Saturday, Mr Trump went on Twitter in a vigorous plug for Mr Rispone.
The president’s intense attention motivated not only conservative Republicans, but also powered a surge in anti-Trump and black voter turnout that helped Mr Edwards.
As he conceded the race, Mr Rispone called on supporters to give a round of applause for Mr Trump, saying: “That man loves America and he loves Louisiana.”
Democrats who argue that nominating a moderate presidential candidate is the best approach to beat Mr Trump are certain to say Louisiana’s race bolsters their case. Mr Edwards opposes gun restrictions, signed one of the nation’s strictest abortion bans and dismissed the impeachment effort as a distraction.
But while Mr Rispone’s loss raises questions about the strength of Mr Trump’s coattails, its relevance to his re-election chances are less clear. Louisiana is expected to easily back Mr Trump next year, and Mr Edwards’ views in many ways are out of step with his own party.


WITNESS LIST Natural Resources Committee March 25, 1997-2:00P HB 520 For: White, Barney (Zapata Protein Inc.), Houston Registering, but not testifying: On: Israel, Paul L. (TPWD), Austin King, Jack M. (TPWD), Austin Osburn, Hal Richard (TPWD), Austin SB 707 Against: Boyd, Gary Mark (Self), Garden Ridge Brymer, Amanda R. (Self), Jourdanton Brymer, John H. IV (Self), Jourdanton Dodds, James E. (Self), Austin Dodds, James F. (Self), San Antonio Eanes, Charles W. (Self), Austin Goldston, William F. (Self), Corpus Christi Lindow, Mark A. (Self), San Antonio On: Reid, Spencer (GLO), Austin SB 988 Registering, but not testifying: On: Higgins, Gerald F. (Comptroller), Austin SB 1199 Against: Woods, Paul Mark (Boating Trades Assn), Austin On: Garcia, Buddy (Senator Lucio), Austin Registering, but not testifying: For: Conway, Ken L. (Cameron County), S. Padre Island Against: Turrieta, Gilbert (Nat Marine Manf. Assn), Austin On: Hall, Steve (TPWD), Austin King, Jack M. (TPWD), Austin SB 1465 For: Barnett, Dale T. (Tx Quarter Horse Assoc), Austin Hutchins, Elbert C. (Tx Veterinary Med Assoc), Austin On: Beals, Terry (Animal Health Comm.), Austin


Neglecting the nonlinear terms, drift-kinetic equation (305) is written
Next let us derive the parallel momentum equation from the linear drift kinetic equation (this is needed in my simulation). Multiplying the linear drift kinetic equation (306) by qv∥ and then integrating over velocity space, we obtain Equation (307) involve ∇Xδf and this should be avoided in particle methods whose goal is to avoid directly evaluating the derivatives of δf over phase-space coordinates. On the other hand, the partial derivatives of velocity moment of δf are allowed. Therefore, we would like to make the velocity integration of δf appear. Note that ∇Xδf here is taken by holding (𝜀,μ) constant and thus v∥ is not a constant and thus can not be moved inside ∇X. Next, to facilitate performing the integration over v∥, we transform the linear drift kinetic equation (306) into variable (X,μ,v∥).


Kernel Main Features¶
MVC and Dependency Injection¶
The implementation is based in two design principles:
- MVC (Model-View-Control) - We separate the model and view componentes in the implementation. The modules: model, basicAgents, basicSpaces, etc, are the model part of the framework. The modules Simulation and BasicSchedule are responsible for the control component and the modules Interface and BasicObserver are responsible for the view part.
- Dependency Injection - Is used to create the objects in the model. The Agents, the Spaces, the Observers and the Schedule are all created using a json file. This file also contains all the necessary information for the initialization of the model, like, the number of agents of each type, the spaces, the intialization of the variables of the agents, the variables that will be observed, etc. The idea is to flexibilize the construcion of scenarios and models based in the same set of classes. These classes can be initialized using the information contained in the json file and we can define each scenario initialization in this file.
Creating a Model¶
The creation of a model is done in the models folder, in the Ecos_p or in other folder depending on the modeller preferences. The paths to the kernel and model, are defined in a model_config.json in the model folder.
A model is composed of three modules:
- The agents module (that can be specialised using submodules) - that contains the definitions of the agents, its variables (compatible with the json file of the definition of the agents) and its behaviour, defined in a method named step. The step method is called by a schedule object that controls the execution of the agent step, the space updades and the observers observation method. An agent in a discrete event simulation is a subclass of DiscreteEventAgent
- The action_set module - that contains the agents actions, the idea behind to separate the agents actions and behavior is that one can implement multiple action sets for new methods representing actions of the agent without interference in the agent code.
- The Space module - that contains the model spaces. The spaces are the way the agents interact. In a canonical simulation model the agents send a message to a space, and this space updates the information for the agents an treat the interactions between the agents in the method update that in executed by the schedule, after the execution of the agent steps.
Executing a Simulation¶
To execute a simulation using a bash file (there are bash files to execute the simulation in the examples) you can just execute the following command in a shell window (using the iterated prisioners’ dillema model) directly in the example folder (../examples/ipd_model):
$ ./ipd.sh
This command will execute the following file (ipd.sh):
rm runs/*.csv
cd ../../kernel
python3 main.py ../examples/ipd_model/ ipd_model_config.json ipd.json
cd ../examples/ipd_model
Rscript -e 'rmarkdown::render("ipd.Rmd", output_format="html_document", output_dir="results")'
The script will:
- erase the csv files from previous simulations of the model;
- change the folder for the kernel (to execute the file main.py)
- Execute the simulation (the main.py file with the configuration files for the model)
- Change the folder again for the model (../examples/ipd_model)
- Execute an R script to show the results
The simulation module¶
When the simulation script is executed, the main.py file creates a Simulation Object this object creates the two other main simulation objects, the Model and the Scenario. These objects are created from the definitions existing in the json simulation file (as well all the other objects in the simulation).
Graphical Representation of the objects in a simulation
The model module¶
The Model object is responsible for the model part of the simulation. It creates the Schedule, the Agents, the Spaces and the observers. Then, the model leads the control to the Scenario, that will have the information for the initialization and execution of the model.
The basicScenarios module¶
Use cases for a typical Simulation
The basicScenarios module has the Scenario object. The scenario is responsible for the initialization of the variables and the control of the execution of the model. In the json file you can have multiple scenarios, each one with multiple runs and variables settings.
The basicSchedule module¶
The Schedule object, controls the execution of the simulation. The schedule type is defined from the json file and controls the execution of the model. Initially we have only a discrete-event schedule.
The sequence of events executed by the schedule for each scenario are:
- The schedule executes the pre_scenario method to make the setups for the scenario.
- The schedule executes a run (there are multiples runs in each scenario).
- For each run, the schedule executes the total steps established in the json file.
- For each step the schedule executes:
- The step method for all agents - This method executes the agents behavior
- The update method for all spaces - The spaces update the agents status, the interaction between the agents and send messages for the agents with the updates of the messages that they are observing (is a publish-subscribe pattern)
- The observe method for all observers defined in the json file.
- In the end of the scenario, the schedule executes the post_scenario method.
Agents, Spaces and Observers¶
The basicAgents module contains the classes defined for all types of agents. We have only the DiscreteEventAgent implemented.
The agent has a method step that is implemented by the specific agents in the model defined by the user. An agent can enter multiple spaces and exit these spaces any time in the simulation. The methods enter and exit from the class Space are used for this.
The basicSpaces module has an update method that is called by the schedule. This method executes the space update in a simulation and is implemented in the model.
Finally, the basicObservers module implement the observers. in this version of the framework, the observers only get the variables that where declared in the json file (for each agent) and write it in a data frame that is written in a csv file at the end of the simulation. The csv file have the values of each observed variable, for each agent for all runs and scenarios.


EuroScorer - Home
Home
Televoting
My number of votes :
Authentification
before voting for EUROVISION
Online Edition for the Fans
Enter your phone number
Enter the code sent by message


MSU preparing for showdown against #1 ranked Lindenwood University at the ECRHA invitational
Michigan State will play tournaments on 3 consecutive weekends come January
MSU sits at 9-0 through three tournaments
Spartans move up to number 3 on November 30 rankings
Grand Valley and MSU accept invitations to ECRHA tournament in Pittsburgh, PA
HUGE upset as MSU beats Putmam (MVP) led Miami 7-6 to start the season
The MSU Roller Hockey program is known as one of the most respected and elite programs in the country. The team is consistently ranked among the top ten teams in the National Collegiate Roller Hockey Association. The success MSU Roller Hockey has enjoyed comes from its unmatched determination to work hard and exceed expectations.
National Runner-Up: 2006, 2013
Final Four: 2005, 2009, 2012
Elite Eight: 2004, 07, 08, 10, 11, 14
MCRHL Regional Championships: 2008, 09, 10, 12, 13, 14


{% extends 'layouts/base.html' %} {% import 'room_request/macros.html' as room_request_macros %} {% import 'macros/page_macros.html' as page %} {% block content %}


Crotale
an EBU R128 loudness correction web application, inspired by
Xylo
Upload an audio file to get started.
Upload


Matt James Boyle
Full Stack Developer- General Technology Aficionado
As a PRINCE2 Practitioner with over 2 years enterprise project management experience as well as enterprise level Full Stack Development experience, I have a proven track record of delivering complex technical projects from both a developmental and a managerial perspective. I pride myself on my stakeholder management capability and calm demeanor. I have a genuine passion for technology and spend my free time learning new programming languages & reading about the latest developments.


Radio Jogjakarta
Jogja Radio App
Notice that the theme is different for this page because we've added a few data-theme
swatch assigments here to show off how flexible it is.
- Radio's name
- Swaragama 101.7 FM
- i-Radio Jogja 88.7 FM
- Geronimo 106.1 FM
- Prambors 95.8 FM
- Unisi 104.5 FM
- Retjo Buntung 99.4 FM
- Radio Anak Jogja 99.9 FM
I'm a themed collapsible with a themed content
I have data-content-theme
attribute set manually on my container to set the color to match the content block I'm in.


CHAPTER 852
OBSTRUCTION OF INGRESS OR EGRESS
Section
852-1 Refusal to provide ingress or egress
852-2 Penalty


$def with (moduleListWithCount, list_of_ay_sems, target_ay_sem) $var title:View Modules Taken Prior to Internship $ home = ['/', 'Home'] $ modulesPriorIntern = ['#', 'Modules Taken Prior to Internship'] $var hierarchy = [home, modulesPriorIntern]
Shows all modules taken by students prior to taking internship for a particular semester, and how many students took them.
| Code | Name | Number of Students |
|---|---|---|
| $module_with_count[0] | $module_with_count[1] | $module_with_count[2] |


| slanczos {mgcv} | R Documentation |
Uses Lanczos iteration to find the truncated eigen-decomposition of a symmetric matrix.
slanczos(A,k=10,kl=-1,tol=.Machine$double.eps^.5,nt=1)
A |
A symmetric matrix. |
k |
Must be non-negative. If |
kl |
If |
tol |
tolerance to use for convergence testing of eigenvalues. Error in eigenvalues will be less
than the magnitude of the dominant eigenvalue multiplied by |
nt |
number of threads to use for leading order iterative multiplication of A by vector. May show no speed improvement on two processor machine. |
If kl
is non-negative, returns the highest k
and lowest kl
eigenvalues,
with their corresponding eigenvectors. If kl
is negative, returns the largest magnitude k
eigenvalues, with corresponding eigenvectors.
The routine implements Lanczos iteration with full re-orthogonalization as described in Demmel (1997). Lanczos
iteraction iteratively constructs a tridiagonal matrix, the eigenvalues of which converge to the eigenvalues of A
,
as the iteration proceeds (most extreme first). Eigenvectors can also be computed. For small k
and kl
the
approach is faster than computing the full symmetric eigendecompostion. The tridiagonal eigenproblems are handled using LAPACK.
The implementation is not optimal: in particular the inner triadiagonal problems could be handled more efficiently, and there would be some savings to be made by not always returning eigenvectors.
A list with elements values
(array of eigenvalues); vectors
(matrix with eigenvectors in its columns);
iter
(number of iterations required).
Simon N. Wood<EMAIL_ADDRESS>Demmel, J. (1997) Applied Numerical Linear Algebra. SIAM
require(mgcv) ## create some x's and knots... set.seed(1); n <- 700;A <- matrix(runif(n*n),n,n);A <- A+t(A) ## compare timings of slanczos and eigen system.time(er <- slanczos(A,10)) system.time(um <- eigen(A,symmetric=TRUE)) ## confirm values are the same... ind <- c(1:6,(n-3):n) range(er$values-um$values[ind]);range(abs(er$vectors)-abs(um$vectors[,ind]))


CBE 141 is a lecture based course which focuses on the principles of thermodynamics as relevant to chemical engineers. The basics of thermodynamics are reviewed such as the definitions of entropy and enthalpy, but the course also teaches thermodynamic concepts that would not have been previously taught such as fugacity. The course also focuses on applying thermodynamic principles to chemical engineering applications. See topics below.
CBE 140, Engineering 7 (May be taken concurrently)
3 hours of lecture and 1 hour of discussion per week, ~4-6 hours per problem set per week.
This course is typically taken immediately following CBE 140 and concurrently with CBE 150A.
Typically, CBE 142 and CBE 150B are taken the semester following this course.
Thermodynamics is a tricky concept to understand for some people, and it definitely makes more sense to some than to others. It seemed, to me, like students either REALLY understood or REALLY struggled with the materia. It can be difficult to take an intangible value like that of entropy and make sense of it, so take time to do practice problems and understand all the reasoning behind the concepts!
Written by: Samantha Marinkovich
Last edited: Fall 2018


I was born at Agoo, La Union but grew up at Al-alinao Norte, Naguilian, La Union. I studied at Al-alinao Elamentary School. I was First in Honor when I was in Kindergarten I to Grade II, Second in Honor starting Grade II to Grade V, and Salutatorian in Grade VI. I joined Girls Scout of the Philippines starting Grade I till Grade VI.
I went to Baguio City after graduating in Elementary. I studied at Baguio City National High School - Main. I was in Star Sections. Star Sections are sections for regular students who have high average. I still joined the Girls Scout of the Philippines, although only on my first year.
Now, I'm studying at the University of the Cordilleras. I am taking the course of Bachelor of Science in Information Technology major in Network and Security Track. I am currently a 2nd Year Student. I was one of the Dean's Lister when I was on my First year, First Term and Third Term. I am one of the College Scholar on my Second year First Term
I like Anime. My favorite anime is Sword Art Online, Second is Guilty Crown. I like color Blue, Black, White and sometimes Red. I like sweets and chocolates. I also like anything cute like Chibis, Pandas, and Puppies. I like romance, comedy, mysteries, music and art, too.
I only have few dislikes. I dislike wearing skirts. I also dislike anything that is dirty. I dislike peolpe who are arrogant, have no respect, have no manners and of course liars.
My hobbbies are watching anime or movies, reading manga or books, drawing or sketching, and sing. I also do cooking, cleaning, collecting stuff, taking pictures and playing piano.


In the fall of 1996 a series of incidents and circumstances in Charlotte-Mecklenburg focused attention on racial division within the community. A police shooting, the prospect of public school resegregation, as well as the increasing ethnic diversity of the community, all contributed to a climate of unrest. Another factor contributing to civic unrest was the influx of large numbers of new residents who were either unaware of or unconnected to Charlotte’s historical image of having dealt positively with issues of race and discrimination.
The Community Building Task Force was created in 1997 to respond to these pressures at the request and with the support of Foundation For The Carolinas, the Mecklenburg Board of County Commissioners and the Charlotte City Council. A landmark civic engagement event entitled "Something Has Begun" drew over 600 residents to launch the effort in December 1997.
The effort, re-named Community Building Initiative, became a permanent organization in 1999, and thousands of residents have become involved since that time. From large-scale civic engagement events, small-group and organizational resource team work, to individual growth and discovery, Community Building Initiative evolved to serve the community as a permanent resource for individuals, community organizations and institutions to turn to for strategic assistance in addressing issues of racial and ethnic inclusion and equity.


Last updated: 2020-05-03
Checks: 7 0
Knit directory: 033_purrr_learning/
This reproducible R Markdown analysis was created with workflowr (version 1.6.2). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.
Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.
Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.
The command set.seed(20200501)
was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.
Great job! Recording the operating system, R version, and package versions is critical for reproducibility.
Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.
Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.
Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.
The results in this page were generated with repository version 9848eb4. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.
Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish
or wflow_git_commit
). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:
Ignored files:
Ignored: .Rproj.user/
Untracked files:
Untracked: data/gap_copy.rds
Untracked: data/gap_mod.rds
Untracked: data/gapminder_raw.csv
Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.
These are the previous versions of the repository in which changes were made to the R Markdown (analysis/04_map-over-multiple-input-objects.Rmd
) and HTML (docs/04_map-over-multiple-input-objects.html
) files. If you’ve configured a remote Git repository (see ?wflow_git_remote
), click on the hyperlinks in the table below to view the files as they were in that past version.
| File | Version | Author | Date | Message |
|---|---|---|---|---|
| Rmd | 9848eb4 | ogorodriguez | 2020-05-03 | Section 4 examples |
map()
can be used to iterate over more than one element of a list or over more than one column in a data frame. For that the purrr package has functions to work with iterations over two objects, the map2()
function and the pmap()
function if we need to iterate with 2 or more.
In this section we will work with an example that is going to create a list of plots to compare two of the columns of the gapminder data set: life expectancy and GDP for each continent/year combination.
The arguments of the map2()
are the two objects to iterate over and the function that will combine them both.
map2(.x = object1, .y = object2, .f = name_of_function(.x, .y))
For this specific example I will define a vector list of continents and a paired vector or list of years to iterate through.
An important observation on how the iteration works. For example.
if .x = c(“Americas”, “Asia”) is my continent vector and, .y = c(1952, 2007) is my year vector; the iteration is not going to over all of the combinations possible with .x and .y. Meaning, it is not going to start pairing something like (Americas, 1952) and then (Americas, 2007), and then (Asia, 1952) and then (Asia, 2007).
The correct iteration will be first (America, 1952) then (Asia, 2007) only.
It goes over the list row by corresponding row, it seems.
In this case we will get our two element from the gapminder data set identified. One for the year and one for the continent.
gap_mod <- read_rds(here::here("data", "gap_mod.rds"))
continent_year <- gap_mod %>% distinct(continent, year)
continent_year
#> # A tibble: 60 x 2
#> continent year
#> <fct> <dbl>
#> 1 Asia 1952
#> 2 Asia 1957
#> 3 Asia 1962
#> 4 Asia 1967
#> 5 Asia 1972
#> 6 Asia 1977
#> 7 Asia 1982
#> 8 Asia 1987
#> 9 Asia 1992
#> 10 Asia 1997
#> # ... with 50 more rows
# extract the continent and year as separate vectors
continents <- continent_year %>%
pull(continent) %>%
as.character()
years <- continent_year %>%
pull(year)
To be consistent with the basic arguments, we will pass the previous vectors to .x and .y variables.
Before jumping into getting the corresponding graphs for all countries, we will try the code with one continent and one year to see the life expectancy of all the countries in that continent that given year.
.x <- continents[1]
.y <- years[1]
gap_mod %>%
filter(continent == .x,
year == .y) %>%
ggplot() +
geom_point(aes(x = gdpPercap, y = lifeExp)) +
ggtitle(glue::glue(.x, " ", .y))
This seems to work. I will use the log version of the gdpPercap variable to better see the differences between the countris with lower GDP per capita.
.x <- continents[1]
.y <- years[1]
gap_mod %>%
filter(continent == .x,
year == .y) %>%
ggplot() +
geom_point(aes(x = log(gdpPercap), y = lifeExp)) +
ggtitle(glue::glue(.x, " ", .y))
I will pass this code into a map2() function to gett all countries.
plot_list <- map2(.x = continents,
.y = years,
.f = ~ (
gap_mod %>%
filter(continent == .x,
year == .y) %>%
ggplot() +
geom_point(aes(x = log(gdpPercap),
y = lifeExp)) +
ggtitle(glue::glue(.x, " ", .y))
))
The result is the creation of a list of graphs saved in the variable plot_list. We can start pulling graphs by calling them from their corresponding arguments.
# GDP per capita in America in 1967
plot_list[[40]]
# GDP per capita in Europe in 1997
plot_list[[22]]
Tibbles are data frames where the columns can also be lists (not only vectors as in base R.)
A tibble can nested, which means that some columns can contain grouped objects such as other data frames o lists.
The next example will nest the gapminder dataset by continent.
gap_copy <- read_rds(here::here("data", "gap_copy.rds"))
gap_nest <- gap_copy %>%
group_by(continent) %>%
nest()
gap_nest
#> # A tibble: 5 x 2
#> # Groups: continent [5]
#> continent data
#> <chr> <list>
#> 1 Asia <tibble [396 x 5]>
#> 2 Europe <tibble [360 x 5]>
#> 3 Africa <tibble [624 x 5]>
#> 4 Americas <tibble [300 x 5]>
#> 5 Oceania <tibble [24 x 5]>
In the previous result the first is the variable we grouped by, continent, and the second column is the rest of the data fram corresponding to that group.
We can see the fifth entry displayed in detail here (Ocenia):
gap_nest$data[[5]]
#> # A tibble: 24 x 5
#> country year pop lifeExp gdpPercap
#> <chr> <dbl> <dbl> <dbl> <dbl>
#> 1 Australia 1952 8691212 69.1 10040.
#> 2 Australia 1957 9712569 70.3 10950.
#> 3 Australia 1962 10794968 70.9 12217.
#> 4 Australia 1967 11872264 71.1 14526.
#> 5 Australia 1972 13177000 71.9 16789.
#> 6 Australia 1977 14074100 73.5 18334.
#> 7 Australia 1982 15184200 74.7 19477.
#> 8 Australia 1987 16257249 76.3 21889.
#> 9 Australia 1992 17481977 77.6 23425.
#> 10 Australia 1997 18565243 78.8 26998.
#> # ... with 14 more rows
The same can be achieved using purrr’s pluck() function.
# extract the fifth entry of the data column in gap_nest
gap_nest %>%
pluck("data", 5)
#> # A tibble: 24 x 5
#> country year pop lifeExp gdpPercap
#> <chr> <dbl> <dbl> <dbl> <dbl>
#> 1 Australia 1952 8691212 69.1 10040.
#> 2 Australia 1957 9712569 70.3 10950.
#> 3 Australia 1962 10794968 70.9 12217.
#> 4 Australia 1967 11872264 71.1 14526.
#> 5 Australia 1972 13177000 71.9 16789.
#> 6 Australia 1977 14074100 73.5 18334.
#> 7 Australia 1982 15184200 74.7 19477.
#> 8 Australia 1987 16257249 76.3 21889.
#> 9 Australia 1992 17481977 77.6 23425.
#> 10 Australia 1997 18565243 78.8 26998.
#> # ... with 14 more rows
Nesting data frames can offer the possibility to use dplyr on more complex objects, not simply mutate() and the like. mutate() work when the columns are simply vectors.
In order for mutate() to work with nested columns we need to use the map() function.
test_list <- tibble(list_col = list(c(1, 5, 7),
5,
c(10, 10, 11))) %>%
mutate(list_sum = map(list_col, ~ sum(.x)))
test_list %>%
pluck("list_sum", 3)
#> [1] 31
Ir we want all sums as a list we can check:
test_list %>%
pull(list_sum)
#> [[1]]
#> [1] 13
#>
#> [[2]]
#> [1] 5
#>
#> [[3]]
#> [1] 31
If we wanted the result to be a vector, we can pass map_dbl() instead.
test_dbl <- tibble(list_col = list(c(1, 5, 7),
5,
c(10, 10, 11))) %>%
mutate(list_sum = map_dbl(list_col, sum))
test_dbl
#> # A tibble: 3 x 2
#> list_col list_sum
#> <list> <dbl>
#> 1 <dbl [3]> 13
#> 2 <dbl [1]> 5
#> 3 <dbl [3]> 31
The idea is to calculate the average life expectancy within each continent and add it as a new column using mutate()
.
If we use mutate()
the traditional way, this will not work:
gap_nest %>%
mutate(avg_life_exp = mean(data$lifeExp))
#> # A tibble: 5 x 3
#> # Groups: continent [5]
#> continent data avg_life_exp
#> <chr> <list> <dbl>
#> 1 Asia <tibble [396 x 5]> NA
#> 2 Europe <tibble [360 x 5]> NA
#> 3 Africa <tibble [624 x 5]> NA
#> 4 Americas <tibble [300 x 5]> NA
#> 5 Oceania <tibble [24 x 5]> NA
The code will not extract the lifeExp
column for each data frame. Applying mutate to a data collumn will result in such errors since the data column in gap_nest is a list of data frames. The way to access data in list columns is via map().
We will try doing the mean lifeExp of a single continent and then we will expand it to the rest.
.x
will be an individual nested data frame of gap_nest, say the first one, “Asia.” The following code defines it.
# the first entry of the nested column "data" in gap_nest, which is Asia
.x <- gap_nest %>%
pluck("data", 1)
.x
#> # A tibble: 396 x 5
#> country year pop lifeExp gdpPercap
#> <chr> <dbl> <dbl> <dbl> <dbl>
#> 1 Afghanistan 1952 8425333 28.8 779.
#> 2 Afghanistan 1957 9240934 30.3 821.
#> 3 Afghanistan 1962 10267083 32.0 853.
#> 4 Afghanistan 1967 11537966 34.0 836.
#> 5 Afghanistan 1972 13079460 36.1 740.
#> 6 Afghanistan 1977 14880372 38.4 786.
#> 7 Afghanistan 1982 12881816 39.9 978.
#> 8 Afghanistan 1987 13867957 40.8 852.
#> 9 Afghanistan 1992 16317921 41.7 649.
#> 10 Afghanistan 1997 22227415 41.8 635.
#> # ... with 386 more rows
To calculate the mean life expectancy then will be.
mean(.x$lifeExp)
#> [1] 60.0649
Now we need to copy this into the tilde-dot anonymous function for the rest. We will show this as a double column.
gap_nest %>%
mutate(avg_lifeExp = map_dbl(data, ~ mean(.x$lifeExp)))
#> # A tibble: 5 x 3
#> # Groups: continent [5]
#> continent data avg_lifeExp
#> <chr> <list> <dbl>
#> 1 Asia <tibble [396 x 5]> 60.1
#> 2 Europe <tibble [360 x 5]> 71.9
#> 3 Africa <tibble [624 x 5]> 48.9
#> 4 Americas <tibble [300 x 5]> 64.7
#> 5 Oceania <tibble [24 x 5]> 74.3
Without the nested columns…
gap_nest %>%
mutate(avg_lifeExp = map_dbl(data, ~ mean(.x$lifeExp))) %>%
select(continent, avg_lifeExp)
#> # A tibble: 5 x 2
#> # Groups: continent [5]
#> continent avg_lifeExp
#> <chr> <dbl>
#> 1 Asia 60.1
#> 2 Europe 71.9
#> 3 Africa 48.9
#> 4 Americas 64.7
#> 5 Oceania 74.3
Using summarise() will remove the nested column…
gap_nest %>%
summarise(avg_lifeExp2 = map_dbl(data, ~ mean(.x$lifeExp)))
#> # A tibble: 5 x 2
#> continent avg_lifeExp2
#> <chr> <dbl>
#> 1 Africa 48.9
#> 2 Americas 64.7
#> 3 Asia 60.1
#> 4 Europe 71.9
#> 5 Oceania 74.3
Fitting models using nested columns in data frames allows to run such models all within a single data frame and then extracting values from the model as we need to test, evaluate, compare, and predict.
Building a simple linear model we will see how lifeExp can be modeled by population and gdpPercap.
First, we will fit a linear model for each continent and save its result as a nested column.
Starting from a basic unit of one continent as .x, again I will choose “Asia”, the linear model function will be..
lm(lifeExp ~ pop + gdpPercap + year, data = .x)
or simply lm(lifeExp ~ ., data = .x)
Testing.
lm(lifeExp ~ ., data = .x) %>% broom::tidy()
#> # A tibble: 36 x 5
#> term estimate std.error statistic p.value
#> <chr> <dbl> <dbl> <dbl> <dbl>
#> 1 (Intercept) -842. 20.1 -42.0 1.08e-140
#> 2 countryBahrain 28.2 1.32 21.3 7.91e- 66
#> 3 countryBangladesh 11.9 1.29 9.22 2.52e- 18
#> 4 countryCambodia 10.5 1.27 8.23 3.35e- 15
#> 5 countryChina 18.2 2.85 6.40 4.96e- 10
#> 6 countryHong Kong China 36.1 1.31 27.5 2.13e- 90
#> 7 countryIndia 11.3 2.25 5.01 8.52e- 7
#> 8 countryIndonesia 16.0 1.32 12.1 1.48e- 28
#> 9 countryIran 21.0 1.28 16.4 1.82e- 45
#> 10 countryIraq 19.1 1.28 14.9 1.36e- 39
#> # ... with 26 more rows
It does. Now let’s fit this into our nested df.
gap_lm <- gap_nest %>%
mutate(lm_lifeExp = map(data, ~ (lm(lifeExp ~ pop + gdpPercap + year, data = .x))))
gap_lm
#> # A tibble: 5 x 3
#> # Groups: continent [5]
#> continent data lm_lifeExp
#> <chr> <list> <list>
#> 1 Asia <tibble [396 x 5]> <lm>
#> 2 Europe <tibble [360 x 5]> <lm>
#> 3 Africa <tibble [624 x 5]> <lm>
#> 4 Americas <tibble [300 x 5]> <lm>
#> 5 Oceania <tibble [24 x 5]> <lm>
To find the values of the model for Ocenia we use then…
gap_lm %>%
pluck("lm_lifeExp", 5)
#>
#> Call:
#> lm(formula = lifeExp ~ pop + gdpPercap + year, data = .x)
#>
#> Coefficients:
#> (Intercept) pop gdpPercap year
#> -2.097e+02 8.365e-09 2.027e-04 1.415e-01
Imagine we want to predict now the response of the data stored in the data column using the corresponding linear model.
gap_pred <- gap_lm %>%
mutate(pred = map2(lm_lifeExp, data, function(.lm, .data) predict(.lm, .data)))
gap_pred
#> # A tibble: 5 x 4
#> # Groups: continent [5]
#> continent data lm_lifeExp pred
#> <chr> <list> <list> <list>
#> 1 Asia <tibble [396 x 5]> <lm> <dbl [396]>
#> 2 Europe <tibble [360 x 5]> <lm> <dbl [360]>
#> 3 Africa <tibble [624 x 5]> <lm> <dbl [624]>
#> 4 Americas <tibble [300 x 5]> <lm> <dbl [300]>
#> 5 Oceania <tibble [24 x 5]> <lm> <dbl [24]>
We can then calculate the correlation between the predicted response and the actual lifeExp value to see if our model is good fit. We can use map2_dbl() since the correlation is one single value and it can be shown in the column.
gap_cor <- gap_pred %>%
mutate(cor = map2_dbl(pred, data, function(.pred, .data) cor(.pred, .data$lifeExp)))
gap_cor
#> # A tibble: 5 x 5
#> # Groups: continent [5]
#> continent data lm_lifeExp pred cor
#> <chr> <list> <list> <list> <dbl>
#> 1 Asia <tibble [396 x 5]> <lm> <dbl [396]> 0.723
#> 2 Europe <tibble [360 x 5]> <lm> <dbl [360]> 0.834
#> 3 Africa <tibble [624 x 5]> <lm> <dbl [624]> 0.645
#> 4 Americas <tibble [300 x 5]> <lm> <dbl [300]> 0.779
#> 5 Oceania <tibble [24 x 5]> <lm> <dbl [24]> 0.987
sessionInfo()
#> R version 3.6.1 (2019-07-05)
#> Platform: x86_64-w64-mingw32/x64 (64-bit)
#> Running under: Windows 10 x64 (build 18362)
#>
#> Matrix products: default
#>
#> locale:
#> [1] LC_COLLATE=Spanish_Spain.1252 LC_CTYPE=Spanish_Spain.1252
#> [3] LC_MONETARY=Spanish_Spain.1252 LC_NUMERIC=C
#> [5] LC_TIME=Spanish_Spain.1252
#>
#> attached base packages:
#> [1] stats graphics grDevices utils datasets methods base
#>
#> other attached packages:
#> [1] forcats_0.5.0 stringr_1.4.0 dplyr_0.8.5 purrr_0.3.3
#> [5] readr_1.3.1 tidyr_1.0.2 tibble_3.0.0 tidyverse_1.3.0
#> [9] ggplot2_3.3.0
#>
#> loaded via a namespace (and not attached):
#> [1] tidyselect_1.0.0 xfun_0.12 haven_2.2.0 lattice_0.20-40
#> [5] colorspace_1.4-1 vctrs_0.2.4 generics_0.0.2 htmltools_0.4.0
#> [9] yaml_2.2.1 utf8_1.1.4 rlang_0.4.5 later_1.0.0
#> [13] pillar_1.4.3 glue_1.4.0 withr_2.1.2 DBI_1.1.0
#> [17] dbplyr_1.4.2 readxl_1.3.1 modelr_0.1.6 lifecycle_0.2.0
#> [21] cellranger_1.1.0 munsell_0.5.0 gtable_0.3.0 workflowr_1.6.2
#> [25] rvest_0.3.5 evaluate_0.14 labeling_0.3 knitr_1.28
#> [29] httpuv_1.5.2 fansi_0.4.0 broom_0.5.5 Rcpp_<IP_ADDRESS>
#> [33] promises_1.1.0 scales_1.1.0 backports_1.1.6 jsonlite_1.6.1
#> [37] farver_2.0.3 fs_1.4.1 hms_0.5.3 digest_0.6.25
#> [41] stringi_1.4.6 grid_3.6.1 rprojroot_1.3-2 here_0.1
#> [45] cli_2.0.2 tools_3.6.1 magrittr_1.5 crayon_1.3.4
#> [49] whisker_0.4 pkgconfig_2.0.3 ellipsis_0.3.0 xml2_1.3.1
#> [53] reprex_0.3.0 lubridate_1.7.8 rstudioapi_0.11 assertthat_0.2.1
#> [57] rmarkdown_2.1 httr_1.4.1 R6_2.4.1 nlme_3.1-144
#> [61] git2r_0.26.1 compiler_3.6.1


Before Change
worker = ray.worker.global_worker
worker.check_connected()
if detached:
logger.warning("The detached flag is deprecated. To create a "
"detached actor, use the name parameter.")
if name is not None:
if not isinstance(name, str):
raise TypeError(
f"name must be None or a string, got: "{type(name)}".")
After Change
if lifetime is None:
detached = False
elif lifetime == "detached":
detached = True
else:
raise ValueError("lifetime must be either `None` or "detached"")
if placement_group is None:
placement_group = PlacementGroup.empty()


WSO2Event event adaptor handles WSO2 Common Events. It receives events over Thrift using TCP, SSL, HTTP, HTTPS and also JSON events via HTTP and HTTPS. By default, the CEP comes with an WSO2Event event adaptor, which is used to receive fired events
Note: You can use the WSO2Event event adaptor that comes with the CEP only to receive incoming events. But, CEP can send output events to any (including external) WSO2Event event adaptors.
By default, Input WSO2Event event adaptor starts with CEP, and its default ports are as follows:
tcp port : 7611
ssl port : 7711
http port : 9763
https port : 9443
The steps below explain how to configure using the management console or using an XML file as explained in step 5.
<PRODUCT_HOME>/repository/deployment/server/inputeventadaptors
directory, which is the Input event adaptor deployment directory. Since hot deployment is
enabled, you can simply add/remove files to deploy/undeploy from the server.<inputEventAdaptor>
root element.
<inputEventAdaptor name="localEventReceiver" type="wso2event" xmlns="http://wso2.org/carbon/eventadaptormanager"></inputEventAdaptor>
Stream definition stores are used to store the stream definitions of the incoming event streams. WSO2Event event adaptor supports three types of stream definition stores as follows:
By default, CEP uses the registry-based stream definition store. To change the default stream
definition store, add the appropriate stream definition store implementation class to <PRODUCT_HOME>/repository/conf/data-bridge/data-bridge-config.xml
file as follows:
<dataBridgeConfiguration xmlns="http://wso2.org/carbon/databridge"> <StreamDefinitionStore> org.wso2.carbon.databridge.streamdefn.cassandra.datastore.CassandraStreamDefinitionStore </StreamDefinitionStore> ... </dataBridgeConfiguration>
Info: : In registry-based stream definition store, the stream
definitions are stored in
/_system/governance/StreamDefinitions
.
In Cassandra-based stream definition store, the stream definitions
are stored in META_KS
key space.
For more details on configuring different types of event adaptors, Please refer the official documentation Here.


public interface DnsClient
| Modifier and Type | Method and Description |
|---|---|
DnsClient |
lookup(java.lang.String name,
Handler<AsyncResult<java.net.InetAddress>> handler)
Try to lookup the A (ipv4) or AAAA (ipv6) record for the given name.
|
DnsClient |
lookup4(java.lang.String name,
Handler<AsyncResult<java.net.Inet4Address>> handler)
Try to lookup the A (ipv4) record for the given name.
|
DnsClient |
lookup6(java.lang.String name,
Handler<AsyncResult<java.net.Inet6Address>> handler)
Try to lookup the AAAA (ipv6) record for the given name.
|
DnsClient |
resolveA(java.lang.String name,
Handler<AsyncResult<java.util.List<java.net.Inet4Address>>> handler)
Try to resolve all A (ipv4) records for the given name.
|
DnsClient |
resolveAAAA(java.lang.String name,
Handler<AsyncResult<java.util.List<java.net.Inet6Address>>> handler)
Try to resolve all AAAA (ipv6) records for the given name.
|
DnsClient |
resolveCNAME(java.lang.String name,
Handler<AsyncResult<java.util.List<java.lang.String>>> handler)
Try to resolve the CNAME record for the given name.
|
DnsClient |
resolveMX(java.lang.String name,
Handler<AsyncResult<java.util.List<MxRecord>>> handler)
Try to resolve the MX records for the given name.
|
DnsClient |
resolveNS(java.lang.String name,
Handler<AsyncResult<java.util.List<java.lang.String>>> handler)
Try to resolve the NS records for the given name.
|
DnsClient |
resolvePTR(java.lang.String name,
Handler<AsyncResult<java.lang.String>> handler)
Try to resolve the PTR record for the given name.
|
DnsClient |
resolveSRV(java.lang.String name,
Handler<AsyncResult<java.util.List<SrvRecord>>> handler)
Try to resolve the SRV records for the given name.
|
DnsClient |
resolveTXT(java.lang.String name,
Handler<AsyncResult<java.util.List<java.lang.String>>> handler)
Try to resolve the TXT records for the given name.
|
DnsClient |
reverseLookup(java.lang.String ipaddress,
Handler<AsyncResult<java.net.InetAddress>> handler)
Try to do a reverse lookup of an ipaddress.
|
DnsClient lookup(java.lang.String name, Handler<AsyncResult<java.net.InetAddress>> handler)
name
- The name to resolvehandler
- the Handler
to notify with the AsyncResult
. The AsyncResult
will get
notified with the resolved InetAddress
if a record was found. If non was found it will
get notifed with null
.
If an error accours it will get failed.DnsClient lookup4(java.lang.String name, Handler<AsyncResult<java.net.Inet4Address>> handler)
name
- The name to resolvehandler
- the Handler
to notify with the AsyncResult
. The AsyncResult
will get
notified with the resolved Inet4Address
if a record was found. If non was found it will
get notifed with null
.
If an error accours it will get failed.DnsClient lookup6(java.lang.String name, Handler<AsyncResult<java.net.Inet6Address>> handler)
name
- The name to resolvehandler
- the Handler
to notify with the AsyncResult
. The AsyncResult
will get
notified with the resolved Inet6Address
if a record was found. If non was found it will
get notifed with null
.
If an error accours it will get failed.DnsClient resolveA(java.lang.String name, Handler<AsyncResult<java.util.List<java.net.Inet4Address>>> handler)
name
- The name to resolvehandler
- the Handler
to notify with the AsyncResult
. The AsyncResult
will get
notified with a List
that contains all the resolved Inet4Address
es. If non was found
and empty List
will be used.
If an error accours it will get failed.DnsClient resolveAAAA(java.lang.String name, Handler<AsyncResult<java.util.List<java.net.Inet6Address>>> handler)
name
- The name to resolvehandler
- the Handler
to notify with the AsyncResult
. The AsyncResult
will get
notified with a List
that contains all the resolved Inet6Address
es. If non was found
and empty List
will be used.
If an error accours it will get failed.DnsClient resolveCNAME(java.lang.String name, Handler<AsyncResult<java.util.List<java.lang.String>>> handler)
name
- The name to resolve the CNAME forhandler
- the Handler
to notify with the AsyncResult
. The AsyncResult
will get
notified with the resolved String
if a record was found. If non was found it will
get notified with null
.
If an error accours it will get failed.DnsClient resolveMX(java.lang.String name, Handler<AsyncResult<java.util.List<MxRecord>>> handler)
name
- The name for which the MX records should be resolvedhandler
- the Handler
to notify with the AsyncResult
. The AsyncResult
will get
notified with a List that contains all resolved MxRecord
s, sorted by their
MxRecord.priority()
. If non was found it will get notified with an empty List
If an error accours it will get failed.DnsClient resolveTXT(java.lang.String name, Handler<AsyncResult<java.util.List<java.lang.String>>> handler)
name
- The name for which the TXT records should be resolvedhandler
- the Handler
to notify with the AsyncResult
. The AsyncResult
will get
notified with a List that contains all resolved String
s. If non was found it will
get notified with an empty List
If an error accours it will get failed.DnsClient resolvePTR(java.lang.String name, Handler<AsyncResult<java.lang.String>> handler)
name
- The name to resolve the PTR forhandler
- the Handler
to notify with the AsyncResult
. The AsyncResult
will get
notified with the resolved String
if a record was found. If non was found it will
get notified with null
.
If an error accours it will get failed.DnsClient resolveNS(java.lang.String name, Handler<AsyncResult<java.util.List<java.lang.String>>> handler)
name
- The name for which the NS records should be resolvedhandler
- the Handler
to notify with the AsyncResult
. The AsyncResult
will get
notified with a List that contains all resolved String
s. If non was found it will
get notified with an empty List
If an error accours it will get failed.DnsClient resolveSRV(java.lang.String name, Handler<AsyncResult<java.util.List<SrvRecord>>> handler)
name
- The name for which the SRV records should be resolvedhandler
- the Handler
to notify with the AsyncResult
. The AsyncResult
will get
notified with a List that contains all resolved SrvRecord
s. If non was found it will
get notified with an empty List
If an error accours it will get failed.DnsClient reverseLookup(java.lang.String ipaddress, Handler<AsyncResult<java.net.InetAddress>> handler)
ipaddress
- The ipaddress to resolve the PTR forhandler
- the Handler
to notify with the AsyncResult
. The AsyncResult
will get
notified with the resolved String
if a record was found. If non was found it will
get notified with null
.
If an error accours it will get failed.


Variable that holds the value for a confidence band.
Variable that holds the confidence interval for a confidence band.
Value that modifies the confidence interval for a confidence band.
ex)If you input 0.95, confidence = 0.95 * confidence
(Optional) If you don't want, leave it blank
| Type | Detail | Actions |
|---|---|---|
| {{stats.type}} | Show | Edit Delete |
Variable that holds the value for a confidence band.
Variable that holds the confidence interval for a confidence band.
Value that modifies the confidence interval for a confidence band.
ex)If you input 0.95, confidence = 0.95 * confidence
(Optional) If you don't want, leave it blank
Variable that holds the value for a confidence band.
Variable that holds the confidence interval for a confidence band.
Value that modifies the confidence interval for a confidence band.
ex)If you input 0.95, confidence = 0.95 * confidence
(Optional) If you don't want, leave it blank