Modal Gallery is the main part of @ks89/angular-modal-gallery. It shows the navigable collection of Images
with buttons, previews and many other customizable features to fit your needs
Follow the getting started guide to install @ks89/angular-modal-gallery and all requirements.
To use Modal Gallery in your application, you have to invoke the .open(...)
method of ModalGalleryService
.
You are free to invoke this method wherever you want, for instance pressing a button.
Previous versions of @ks89/angular-modal-gallery were very limited forcing you to add a selector into the HTML file, however, now you can show Modal Gallery without adding UI element to you app, or even automatically after an event, invoking the open method.
To keep everything simple, let say that we want to open Modal Gallery with a button and specifically we want to open the first image (index = 0) of an array of images, exactly as the live demo above. So in the component HTML file you have to add this:
Then, in your Typescript component you have to init the images array with Image class instances and invoke the open method to show the modal gallery in this way:
To customize the modal gallery exploring all possibilities, see the table below to pass your configuration. To listen for modal gallery events, check the Output table.
To try more modal gallery live demos with custom configuration, please take a look at the example section of this website.
id |
Type:
number
A mandatory unique id >= 0 of the current modal gallery, used by
GalleryService .
|
images |
Type:
Image[]
The mandatory array of
Image objects, that represent the
modal images. In fact, an Image is an object with
image source paths and other additional configuration parameters. For more info check Image .
Please don't use this.images.push(img) , when you want to modify this array, instead re-assign it to trigger change detection.
For instance: this.images = [...this.images, newImage] to create a new array from the original with a new image at the end.
Attention: if you need to add the same image (same url) multiple times in the same modal gallery, you must append to the url a random but unique query param to prevent caching issues.
For more info, please read this issue.
|
currentImage |
Type:
Image
The image object to show as soon the modal gallery opens. You should simply pass one of the element of the images array.
For instance, if you want to open the first image in the array, pass
images[0] .
|
config
|
Type:
LibConfig optional
Interface to configure modal gallery. |
close |
An It contains an
|
show |
An It contains an Everytime the current image changes, a new
|
firstImage |
An It contains an
|
lastImage |
An It contains an
|
hasData |
An It contains an This event is useful to know when angular-modal-gallery is ready.
|
buttonBeforeHook |
A It contains an This event is a 'before hook', because you can execute your code before every action triggered by buttons. Also, since you have the full
|
buttonAfterHook |
A It contains an This event is an 'after hook', because you can execute your code after every action triggered by buttons. Also, since you have the full
|
Interface that represent the modal image.
img |
Type:
string | SafeResourceUrl
Mandatory source path of the modal image. To support base64 images, check the official example. If you need multiple non-base64 images with the same image path, to prevent caching issues you have to append something like '?index', for instance '?0', '?1'. For more information, please check the official example called 'Same images'. |
description |
Type:
string optional
Optional description text.
Attention: this is not the full description, but only the last part, after image numbers.
For instance: `Image 2/12 - |
extUrl |
Type:
string optional
Optional external URL path. You must use a valid URL, because |
title |
Type:
string optional
Optional title text. It sets 'title' html's attribute, useful for accessibility purposes. To ignore this feature, don't add this property. |
alt |
Type:
string optional
Optional alt text. It sets 'alt' html's attribute, useful for accessibility purposes. To ignore this feature, don't add this property. |
arialLabel |
Type:
string optional
Optional aria label text. It sets 'aria-label' html's attribute, useful for accessibility purposes. To ignore this feature, don't add this property. |
invertSwipe |
Type:
boolean optional
Optional boolean property to invert the default swipe direction on devices with touchscreen. |
The payload of events related to action, emitted by this library.
action |
Type:
Action
It's the enum |
result |
Type:
number | boolean
The result/data of the emitted event. For example, it could be the index of the current image or a boolean to say if images are ready to display and so on. |
Interface to customize current image in modal gallery.
downloadable |
Type:
boolean optional
If true, you'll be able to download the current modal image always with keyboard shortcut (ctrl+s
or cmd+s),
but the download button will be visible only when Please be careful: on SmartPhones and Tablets you can't use keyboard shortcuts, so to download images, please enable |
loadingConfig
|
Type:
LoadingConfig optional
Interface to hide/show and to configure the modal loading spinner. |
description |
Type:
Description interfaceoptional
Interface to specify a custom description either a full string or an object with string properties.
The first one is the entire string (for instance 'Description of Image 1/2 - My family - bla bla'),
the second one is an object with these properties
To customize part of the description, you can refer to this example: ` Attention please! There aren't spaces between values. So if you want a 'text - text' you have to specify exactly ' - ', with the two spaces Also, from version 5.4.0, you can use this interface to change description's style. |
navigateOnClick |
Type:
boolean optional
If false, click events on the current image will be ignored, otherwise will navigate to the next image. |
navigateOnClick |
Type:
boolean optional
If false, click events on the current image will be ignored, otherwise will navigate to the next image. |
Interface to customize keyboard's buttons.
esc |
Type:
number optional
Javascript keycode to use instead of the default esc |
right |
Type:
number optional
Javascript keycode to use instead of the default → |
left |
Type:
number optional
Javascript keycode to use instead of the default ← |
Interface to customize loading spinner.
enable |
Type:
boolean
If true, spinner is visible, otherwise not. |
type |
Type:
LoadingType
Enum to choose a loading spinner (already bundled in angular-modal-gallery in pure css). |
Interface to customize sliding features of modal gallery (both current image and side-previews).
infinite |
Type:
boolean optional
If |
sidePreviews |
Type:
SidePreviewsConfig optional
With this property, you can hide/show side-previews or change sizes. |
playConfig |
Type:
PlayConfig optional
With this property, you can enable autoPlay, change default interval and enable pauseOnHover. |
Service to interact with modal gallery without clicking on plain-gallery. For instance, you can open modal gallery from your Component's code instead of clicking on a thumb in plain gallery.
openGallery ( galleryId, index ) |
Input types:
galleryId: number | undefined and index: number
Method to open an image (index) of modal gallery (id). You have to provide both an index >= 0 and a galleryId >= 0. Also, remember that ids must be unique across your web page, so every instance of modal gallery must have a different id. |
closeGallery ( galleryId ) |
Input types:
galleryId: number | undefined
Method to close the modal gallery with id equals to the input parameter galleryId. You have to provide a galleryId >= 0. Also, remember that ids must be unique across your web page, so every instance of modal gallery must have a different id. |
update ( galleryId, index, newImage ) |
Input types:
galleryId: number | undefined , index: number and newImage: Image
Method to update the images array at position |
Enum that represents the type of the loading spinner. It can be one of these enum values:
STANDARD
: the default spinnerCIRCULAR
: a circular spinnerBARS
: spinner built with some animated vertical barsDOTS
: spinner composed by animated dotsCUBE_FLIPPING
: flipping cube spinnerCIRCLES
: spinner composed by multiple circlesEXPLOSING_SQUARES
: spinner composed by 4 squares moving outsideEnum that represents keyboard's keycodes. It can be one of these enum values:
ESC
: 27LEFT_ARROW
: 37RIGHT_ARROW
: 39UP_ARROW
: 38DOWN_ARROW
: 40


Construct and maintain a visual identity for your brand.
Adobe guru (Photoshop, Illustrator, InDesign, Dreamweaver, Acrobat, XD)
Build wireframe prototypes. Design webpages and emails.
Started college at ACC, with a focus on Interactive Design. Graduated and transferred to Texas State. A focus on Visual Communications.
Hired on at Noesis Energy as a Junior UI/UX Designer.
Hired on at Dell EMC.
Hired on at Illuminas.
Hired on at TxDOT.


%firstname% %lastname%
%platform%
%version%
Free memory: %heap% bytes
Uptime: %uptime%
%volt% %rssi%
Local time: %localtime%
Admin auth settings:
Full Security Config Security
Show Password
WiFi options:
Station mode AP mode
Show Passwords
MQTT settings:
NTP settings:
WaterMeter module settings:
Reboot Default setting (New start)
Save config
Main Uploader Log


Building out the Back End
In this session, we’ll add the rest of our models and controllers that expose them. We’ll also refactor our application, moving our DTOs to a shared project so they can be used by our front-end application later.
Add a ConferenceDTO project
We’ll start by creating the new shared project to hold our data transfer objects.
Adding the ConferenceDTO Project using Visual Studio
- If using Visual Studio, right-click on the Solution and select Add / New Project….
- Select .NET Standard from the project types on the left and select the Class Library (.NET Standard) template. Name the project ConferenceDTO and press OK.
- Delete the generated
Class1.cs
file from this new project.- Right-click the ‘Dependencies’ node under the BackENd project, select “Add Reference…” and put a checkmark near ConferenceDTO.
Adding the ConferenceDTO project via the Command Line
- Open a command prompt and navigate to the root
ConferencePlanner
directory.- Run the following command:
dotnet new classlib -o ConferenceDTO -f netstandard2.0
- Next we’ll need to add a reference to the ConferenceDTO project from the BackEnd project. From the command line, navigate to the BackEnd project directory and execute the following command:
dotnet add reference ../ConferenceDTO
- Add the ConferenceDTO project to the solution:
dotnet sln add ConferenceDTO/ConferenceDTO.csproj
Refactoring the Speaker model into the ConferenceDTO project
- Copy the
Speaker.cs
class from the BackEnd application into the root of the new ConferenceDTO project, and change the namespace toConferenceDTO
. - The data annotations references should be broken at this point, to resovle it, we need to add a nuget the missing NuGet package into the
ConferenceDTO
project. - Add a reference to the NuGet package
System.ComponentModel.Annotations
version4.6.0
.This can be done from the command line using
dotnet add package System.ComponentModel.Annotations --version 4.6.0
- When the package restore completes, you should see that your data annotations are now resolved.
- Go back to the BackEnd application and modify the code in
Speaker.cs
as shown:public class Speaker : ConferenceDTO.Speaker { }
- Run the application and view the Speakers data using the Swagger UI to verify everything still works.
Adding the remaining models to ConferenceDTO
We’ve got several more models to add, and unfortunately it’s a little mechanical. You can copy the following classes manually, or open the completed solution which is shown at the end.
- Create an
Attendee.cs
class in the ConferenceDTO project with the following code:using System; using System.Collections.Generic; using System.ComponentModel.DataAnnotations; namespace ConferenceDTO { public class Attendee { public int Id { get; set; } [Required] [StringLength(200)] public virtual string FirstName { get; set; } [Required] [StringLength(200)] public virtual string LastName { get; set; } [Required] [StringLength(200)] public string UserName { get; set; } [StringLength(256)] public virtual string EmailAddress { get; set; } } }
- Create a
Session.cs
class with the following code:using System; using System.Collections; using System.Collections.Generic; using System.ComponentModel.DataAnnotations; namespace ConferenceDTO { public class Session { public int Id { get; set; } [Required] [StringLength(200)] public string Title { get; set; } [StringLength(4000)] public virtual string Abstract { get; set; } public virtual DateTimeOffset? StartTime { get; set; } public virtual DateTimeOffset? EndTime { get; set; } // Bonus points to those who can figure out why this is written this way public TimeSpan Duration => EndTime?.Subtract(StartTime ?? EndTime ?? DateTimeOffset.MinValue) ?? TimeSpan.Zero; public int? TrackId { get; set; } } }
- Create a new
Track.cs
class with the following code:using System; using System.Collections.Generic; using System.ComponentModel.DataAnnotations; namespace ConferenceDTO { public class Track { public int Id { get; set; } [Required] [StringLength(200)] public string Name { get; set; } } }
Creating Derived Models in the BackEnd project
We’re not going to create our EF models directly from the ConferenceDTO
classes. Instead, we’ll create some composite classes such as SessionSpeaker
, since these will map more closely to what our application will be working with.
We’re also going to take this opportunity to rename the Models
directory in the BackEnd project to Data
since it no longer just contains models.
- Right-click the
Models
directory and selectRename
, changing the name toData
.Note: If you are using Visual Studio, you can use refactoring to rename the namespace.
- Add a
SessionSpeaker.cs
class to theData
directory with the following code:using System; using System.Collections.Generic; using System.Linq; using System.Threading.Tasks; namespace BackEnd.Data { public class SessionSpeaker { public int SessionId { get; set; } public Session Session { get; set; } public int SpeakerId { get; set; } public Speaker Speaker { get; set; } } }
-
Add an
SessionAttendee.cs
class with the following code:using System; using System.Collections.Generic; using System.Linq; using System.Threading.Tasks; namespace BackEnd.Data { public class SessionAttendee { public int SessionId { get; set; } public Session Session { get; set; } public int AttendeeId { get; set; } public Attendee Attendee { get; set; } } }
- Add an
Attendee.cs
class with the following code:using System; using System.Collections.Generic; namespace BackEnd.Data { public class Attendee : ConferenceDTO.Attendee { public virtual ICollection<SessionAttendee> SessionsAttendees { get; set; } } }
- Add a
Session.cs
class with the following code:using System; using System.Collections; using System.Collections.Generic; namespace BackEnd.Data { public class Session : ConferenceDTO.Session { public virtual ICollection<SessionSpeaker> SessionSpeakers { get; set; } public virtual ICollection<SessionAttendee> SessionAttendees { get; set; } public Track Track { get; set; } } }
- Add a
Track.cs
class with the following code:using System; using System.Collections.Generic; using System.ComponentModel.DataAnnotations; namespace BackEnd.Data { public class Track : ConferenceDTO.Track { public virtual ICollection<Session> Sessions { get; set; } } }
- Modify the
Speaker.cs
class we wrote previously to make the following two changes: update to the namespace to match our directory rename, and add a referece to theSessionSpeaker
composite class:using System; using System.Collections.Generic; namespace BackEnd.Data { public class Speaker : ConferenceDTO.Speaker { public virtual ICollection<SessionSpeaker> SessionSpeakers { get; set; } = new List<SessionSpeaker>(); } }
Update the ApplicationDbContext
Okay, now we need to update our ApplicationDbContext
so Entity Framework knows about our new models.
- Update
ApplicationDbContext.cs
to use the following code:using Microsoft.EntityFrameworkCore; namespace BackEnd.Data { public class ApplicationDbContext : DbContext { public ApplicationDbContext(DbContextOptions<ApplicationDbContext> options) : base(options) { } protected override void OnModelCreating(ModelBuilder modelBuilder) { modelBuilder.Entity<Attendee>() .HasIndex(a => a.UserName) .IsUnique(); // Many-to-many: Session <-> Attendee modelBuilder.Entity<SessionAttendee>() .HasKey(ca => new { ca.SessionId, ca.AttendeeId }); // Many-to-many: Speaker <-> Session modelBuilder.Entity<SessionSpeaker>() .HasKey(ss => new { ss.SessionId, ss.SpeakerId }); } public DbSet<Session> Sessions { get; set; } public DbSet<Track> Tracks { get; set; } public DbSet<Speaker> Speakers { get; set; } public DbSet<Attendee> Attendees { get; set; } } }
- Fix errors due to the rename from
BackEnd.Models
toBackEnd.Data
. You can either do this using a find / replace (replacing “BackEnd.Models” with “BackEnd.Data”) or you can do a build and fix errors. - Ensure that the application builds now.
Add a new database migration
Visual Studio: Package Manager Console
- Run the following commands in the Package Manager Console (specify the
BackEnd
project)Add-Migration Refactor Update-Database
Command line
- Run the following commands in the command prompt in the
BackEnd
project directory:dotnet ef migrations add Refactor dotnet ef database update
- Now take a deep breath and run the application and navigate to
/swagger
. You should see the Swagger UI.
Updating the Speakers API controller
- Modify the query for the
GetSpeakers()
method as shown below:var speakers = await _context.Speakers.AsNoTracking() .Include(s => s.SessionSpeakers) .ThenInclude(ss => ss.Session) .ToListAsync(); return speakers;
- While the above will work, this is directly returning our model class. A better practice is to return an output model class. Create a
SpeakerResponse.cs
class in theConferenceDTO
project with the following code:using System; using System.Collections.Generic; using System.Text; namespace ConferenceDTO { public class SpeakerResponse : Speaker { public ICollection<Session> Sessions { get; set; } = new List<Session>(); } }
- Now we’ll add a utility method to map between these classes. In the BackEnd project, create an
Infrastructure
directory. Add a class namedEntityExtensions.cs
with the following mapping code:using BackEnd.Data; using System; using System.Collections.Generic; using System.Linq; using System.Threading.Tasks; namespace BackEnd.Data { public static class EntityExtensions { public static ConferenceDTO.SpeakerResponse MapSpeakerResponse(this Speaker speaker) => new ConferenceDTO.SpeakerResponse { Id = speaker.Id, Name = speaker.Name, Bio = speaker.Bio, WebSite = speaker.WebSite, Sessions = speaker.SessionSpeakers? .Select(ss => new ConferenceDTO.Session { Id = ss.SessionId, Title = ss.Session.Title }) .ToList() }; } }
- Now we can update the
GetSpeakers()
method of the SpeakersController so that it returns our response model. Update the last few lines so that the method reads as follows:[HttpGet] public async Task<ActionResult<List<ConferenceDTO.SpeakerResponse>>> GetSpeakers() { var speakers = await _context.Speakers.AsNoTracking() .Include(s => s.SessionSpeakers) .ThenInclude(ss => ss.Session) .Select(s => s.MapSpeakerResponse()) .ToListAsync(); return speakers; }
- Update the
GetSpeaker()
method to use our mapped response models as follows:[HttpGet("{id}")] public async Task<ActionResult<ConferenceDTO.SpeakerResponse>> GetSpeaker(int id) { var speaker = await _context.Speakers.AsNoTracking() .Include(s => s.SessionSpeakers) .ThenInclude(ss => ss.Session) .SingleOrDefaultAsync(s => s.Id == id); if (speaker == null) { return NotFound(); } return speaker.MapSpeakerResponse(); }
- Remove the other actions (
PutSpeaker
,PostSpeaker
,DeleteSpeaker
), on theSpeakersController
.
Adding the remaining API Controllers and DTOs
- Add the following response DTO classes from the save point folder
AttendeeResponse
SessionResponse
ConferenceResponse
TrackResponse
TagResponse
- Update the
EntityExtensions
class with theMapSessionResponse
andMapAttendeeResponse
methods from the save point folder - Copy the following controllers from the save point folder into the current project’s
BackEnd/Controllers
directory:SessionsController
AttendeesController
Adding Conference Upload support
- Copy the
DataLoader.cs
class from here into theData
directory of theBackEnd
project. - Copy the
SessionizeLoader.cs
andDevIntersectionLoader.cs
classes from here into the current project’s/src/BackEnd/Data/
directory.Note: We have data loaders from the two conference series where this workshop has been presented most; you can update this to plug in your own conference file format.
- To improve the UI for upload, turn on the option to display enums as strings by changing
AddSwaggerGen
inStartup.cs
to the following:services.AddSwaggerGen(options => { options.SwaggerDoc("v1", new OpenApiInfo { Title = "Conference Planner API", Version = "v1" }); options.DescribeAllEnumsAsStrings(); });
- Run the application to see the updated data via Swagger UI.
- Use the Swagger UI to upload NDC_Sydney_2019.json to the
/api/Sessions/upload
API.
—
Sessions
This is series of articles on Building Conf planner app with Asp.net Core:
- 1 - Build the back-end API with basic EF model
- 2 - Finish the back-end API and EF model, refactor into view modelsl
- 3 - Add front-end, render agenda, set up front-end models
- 4 - Add authentication, add admin policy, allow editing sessions, users can sign-in with Identity, custom auth tag helper
- 5 - Add user association and personal agenda
- 6 - Deployment, Azure and other production environments, configuring environments, diagnostics
- 7 - Challenges
- 8 - SPA front-end


Your email address is sent directly to a DOMAIN.com email address and is not shared with, nor will it ever be sold to, any third party. If you would like to stop receiving notifications of new blog posts simply reply with UNSUBSCRIBE FROM BLOG POSTS in the subject line and you will receive no further notifications.


---
tags:
- data
- history
- overview
author:
display_name: JP
email<EMAIL_ADDRESS>date: 2011-11-07 14:12:43 -0700
title: History Overview
categories: []
---
{% include table_overview.html name="History" %}
| Historic Lake Bonneville |
map data |
UGRC |
The geographic extent of the Lake Bonneville shoreline. |
| Markers and Monuments |
map data |
Utah State History |
Location, text, photos, and other information about historic markers and monuments located throughout Utah. |
| Historic Districts |
map data |
UGRC |
Historic Districts in Utah. |
| Historic Trails |
map data |
UGRC |
Historic Trails in Utah. |
| Cemeteries |
map data |
Utah State History |
Cemetery locations in Utah. |
Related Data


If you are group administrator or system administrator (provided that you are member of the message group), you have right to delete your message group.
Under group menu (figure 3.3.1 or figure 3.11.1), select item 'Delete Group', then screen like figure 3.12.1 will be seen. If you select 'OK' of dialog
box shown on figure 3.12.1, then another dialog box will be displayed for you to make the finally decision. If you select 'OK' again, the SMS system will
purge your current message group. All group messages and uploaded files will be deleted, message group profile will then be removed. All group members
still stayed on the deleted message group will be switched to their home page automatically.
|


Welcome to the third project of the Machine Learning Engineer Nanodegree! In this notebook, some template code has already been provided for you, and it will be your job to implement the additional functionality necessary to successfully complete this project. Sections that begin with 'Implementation' in the header indicate that the following block of code will require additional functionality which you must provide. Instructions will be provided for each section and the specifics of the implementation are marked in the code block with a 'TODO'
statement. Please be sure to read the instructions carefully!
In addition to implementing code, there will be questions that you must answer which relate to the project and your implementation. Each section where you will answer a question is preceded by a 'Question X' header. Carefully read each question and provide thorough answers in the following text boxes that begin with 'Answer:'. Your project submission will be evaluated based on your answers to each of the questions and the implementation you provide.
Note: Code and Markdown cells can be executed using the Shift + Enter keyboard shortcut. In addition, Markdown cells can be edited by typically double-clicking the cell to enter edit mode.
In this project, you will analyze a dataset containing data on various customers' annual spending amounts (reported in monetary units) of diverse product categories for internal structure. One goal of this project is to best describe the variation in the different types of customers that a wholesale distributor interacts with. Doing so would equip the distributor with insight into how to best structure their delivery service to meet the needs of each customer.
The dataset for this project can be found on the UCI Machine Learning Repository. For the purposes of this project, the features 'Channel'
and 'Region'
will be excluded in the analysis — with focus instead on the six product categories recorded for customers.
Run the code block below to load the wholesale customers dataset, along with a few of the necessary Python libraries required for this project. You will know the dataset loaded successfully if the size of the dataset is reported.
# Import libraries necessary for this project
import numpy as np
import pandas as pd
from IPython.display import display # Allows the use of display() for DataFrames
# Import supplementary visualizations code visuals.py
import visuals as vs
# Pretty display for notebooks
%matplotlib inline
# Load the wholesale customers dataset
try:
data = pd.read_csv("customers.csv")
data.drop(['Region', 'Channel'], axis = 1, inplace = True)
print "Wholesale customers dataset has {} samples with {} features each.".format(*data.shape)
except:
print "Dataset could not be loaded. Is the dataset missing?"
data.head()
In this section, you will begin exploring the data through visualizations and code to understand how each feature is related to the others. You will observe a statistical description of the dataset, consider the relevance of each feature, and select a few sample data points from the dataset which you will track through the course of this project.
Run the code block below to observe a statistical description of the dataset. Note that the dataset is composed of six important product categories: 'Fresh', 'Milk', 'Grocery', 'Frozen', 'Detergents_Paper', and 'Delicatessen'. Consider what each category represents in terms of products you could purchase.
# Display a description of the dataset
display(np.round(data.describe(), 2))
To get a better understanding of the customers and how their data will transform through the analysis, it would be best to select a few sample data points and explore them in more detail. In the code block below, add three indices of your choice to the indices
list which will represent the customers to track. It is suggested to try different sets of samples until you obtain customers that vary significantly from one another.
# TODO: Select three indices of your choice you wish to sample from the dataset
indices = [85, 181, 184]
# Create a DataFrame of the chosen samples
samples = pd.DataFrame(data.loc[indices], columns = data.keys()).reset_index(drop = True)
print "Chosen samples of wholesale customers dataset:"
display(samples)
Consider the total purchase cost of each product category and the statistical description of the dataset above for your sample customers.
What kind of establishment (customer) could each of the three samples you've chosen represent?
Hint: Examples of establishments include places like markets, cafes, and retailers, among many others. Avoid using names for establishments, such as saying "McDonalds" when describing a sample customer as a restaurant.
Answer:
One interesting thought to consider is if one (or more) of the six product categories is actually relevant for understanding customer purchasing. That is to say, is it possible to determine whether customers purchasing some amount of one category of products will necessarily purchase some proportional amount of another category of products? We can make this determination quite easily by training a supervised regression learner on a subset of the data with one feature removed, and then score how well that model can predict the removed feature.
In the code block below, you will need to implement the following:
new_data
a copy of the data by removing a feature of your choice using the DataFrame.drop
function.sklearn.cross_validation.train_test_split
to split the dataset into training and testing sets.test_size
of 0.25
and set a random_state
.random_state
, and fit the learner to the training data.score
function.from sklearn.tree import DecisionTreeRegressor
from sklearn.cross_validation import train_test_split
from sklearn.cross_validation import cross_val_score
for feature in data.keys():
# TODO: Make a copy of the DataFrame, using the 'drop' function to drop the given feature
new_data = data.drop([feature], axis = 1 )
y = data[feature]
# TODO: Split the data into training and testing sets using the given feature as the target
X_train, X_test, y_train, y_test = train_test_split(new_data, y, test_size = 0.25, random_state = 12)
# TODO: Create a decision tree regressor and fit it to the training set
regressor = DecisionTreeRegressor(max_depth = 3, random_state = 3)
regressor.fit(X_train, y_train)
y_pred = regressor.predict(X_test)
# TODO: Report the score of the prediction using the testing set
score = regressor.score(X_test, y_test)
print feature, np.round(score, 3), regressor.feature_importances_
Which feature did you attempt to predict? What was the reported prediction score? Is this feature is necessary for identifying customers' spending habits?
Hint: The coefficient of determination, R^2
, is scored between 0 and 1, with 1 being a perfect fit. A negative R^2
implies the model fails to fit the data.
Answer:
All 6 features have been checked for their relevance to other features
To get a better understanding of the dataset, we can construct a scatter matrix of each of the six product features present in the data. If you found that the feature you attempted to predict above is relevant for identifying a specific customer, then the scatter matrix below may not show any correlation between that feature and the others. Conversely, if you believe that feature is not relevant for identifying a specific customer, the scatter matrix might show a correlation between that feature and another feature in the data. Run the code block below to produce a scatter matrix.
# Produce a scatter matrix for each pair of features in the data
axes = pd.scatter_matrix(data, alpha = 0.3, figsize = (14,8), diagonal = 'kde')
corr = data.corr().as_matrix()
for i, j in zip(*np.triu_indices_from(axes, k=1)):
axes[i, j].annotate("%.3f" %corr[i,j], (0.8, 0.8), xycoords='axes fraction', ha='center', va='center')
Are there any pairs of features which exhibit some degree of correlation? Does this confirm or deny your suspicions about the relevance of the feature you attempted to predict? How is the data for those features distributed?
Hint: Is the data normally distributed? Where do most of the data points lie?
Answer:
In this section, you will preprocess the data to create a better representation of customers by performing a scaling on the data and detecting (and optionally removing) outliers. Preprocessing data is often times a critical step in assuring that results you obtain from your analysis are significant and meaningful.
If data is not normally distributed, especially if the mean and median vary significantly (indicating a large skew), it is most often appropriate to apply a non-linear scaling — particularly for financial data. One way to achieve this scaling is by using a Box-Cox test, which calculates the best power transformation of the data that reduces skewness. A simpler approach which can work in most cases would be applying the natural logarithm.
In the code block below, you will need to implement the following:
log_data
after applying logarithmic scaling. Use the np.log
function for this.log_samples
after applying logarithmic scaling. Again, use np.log
.# TODO: Scale the data using the natural logarithm
log_data = np.log(data)
# TODO: Scale the sample data using the natural logarithm
log_samples = np.log(samples)
# Produce a scatter matrix for each pair of newly-transformed features
#pd.scatter_matrix(log_data, alpha = 0.3, figsize = (14,8), diagonal = 'kde');
axes = pd.scatter_matrix(log_data, alpha = 0.3, figsize = (14,8), diagonal = 'kde')
corr = data.corr().as_matrix()
for i, j in zip(*np.triu_indices_from(axes, k=1)):
axes[i, j].annotate("%.3f" %corr[i,j], (0.2, 0.2), xycoords='axes fraction', ha='center', va='center')
After applying a natural logarithm scaling to the data, the distribution of each feature should appear much more normal. For any pairs of features you may have identified earlier as being correlated, observe here whether that correlation is still present (and whether it is now stronger or weaker than before).
Run the code below to see how the sample data has changed after having the natural logarithm applied to it.
# Display the log-transformed sample data
display(log_samples)
Detecting outliers in the data is extremely important in the data preprocessing step of any analysis. The presence of outliers can often skew results which take into consideration these data points. There are many "rules of thumb" for what constitutes an outlier in a dataset. Here, we will use Tukey's Method for identfying outliers: An outlier step is calculated as 1.5 times the interquartile range (IQR). A data point with a feature that is beyond an outlier step outside of the IQR for that feature is considered abnormal.
In the code block below, you will need to implement the following:
Q1
. Use np.percentile
for this.Q3
. Again, use np.percentile
.step
.outliers
list.NOTE: If you choose to remove any outliers, ensure that the sample data does not contain any of these points!
Once you have performed this implementation, the dataset will be stored in the variable good_data
.
display(log_data.describe())
# For each feature find the data points with extreme high or low values
outlier_list = [] # All outliers
outlier_multiple = [] # Outliers for more than one feature
for feature in log_data.keys():
# TODO: Calculate Q1 (25th percentile of the data) for the given feature
Q1 = np.percentile(log_data[feature], 25)
# TODO: Calculate Q3 (75th percentile of the data) for the given feature
Q3 = np.percentile(log_data[feature], 75)
# TODO: Use the interquartile range to calculate an outlier step (1.5 times the interquartile range)
step = 1.5*(Q3 - Q1)
# Display the outliers
print "Data points considered outliers for the feature '{}':".format(feature)
display(log_data[~((log_data[feature] >= Q1 - step) & (log_data[feature] <= Q3 + step))])
outlier_list.extend(log_data[~((log_data[feature] >= Q1 - step) & (log_data[feature] <= Q3 + step))].index.tolist())
# Find indices of outliers for multiple features (equal or more than 2)
outlier_all = {x:outlier_list.count(x) for x in set(outlier_list)}
for x in outlier_all.keys():
if outlier_all.get(x) >= 2:
outlier_multiple.append(x)
print "Indices of outliers for multiple features: {}".format(outlier_multiple)
# OPTIONAL: Select the indices for data points you wish to remove
outliers = outlier_multiple
# Remove the outliers, if any were specified
good_data = log_data.drop(log_data.index[outliers]).reset_index(drop = True)
# Heatmap using percentiles to display outlier data
import matplotlib.pyplot as plt
import seaborn as sns
percentiles = data.rank(pct=True)
percentiles = percentiles.iloc[outliers]
plt.title('Multiple Outliers Heatmap', fontsize=14)
heat = sns.heatmap(percentiles, annot=True)
display(heat)
Are there any data points considered outliers for more than one feature based on the definition above? Should these data points be removed from the dataset? If any data points were added to the outliers
list to be removed, explain why.
Answer:
In this section you will use principal component analysis (PCA) to draw conclusions about the underlying structure of the wholesale customer data. Since using PCA on a dataset calculates the dimensions which best maximize variance, we will find which compound combinations of features best describe customers.
Now that the data has been scaled to a more normal distribution and has had any necessary outliers removed, we can now apply PCA to the good_data
to discover which dimensions about the data best maximize the variance of features involved. In addition to finding these dimensions, PCA will also report the explained variance ratio of each dimension — how much variance within the data is explained by that dimension alone. Note that a component (dimension) from PCA can be considered a new "feature" of the space, however it is a composition of the original features present in the data.
In the code block below, you will need to implement the following:
sklearn.decomposition.PCA
and assign the results of fitting PCA in six dimensions with good_data
to pca
.log_samples
using pca.transform
, and assign the results to pca_samples
.log_samples
from sklearn.decomposition import PCA
# TODO: Apply PCA by fitting the good data with the same number of dimensions as features
pca = PCA()
pca.fit(good_data)
# TODO: Transform log_samples using the PCA fit above
pca_samples = pca.transform(log_samples)
# Generate PCA results plot
pca_results = vs.pca_results(good_data, pca)
# Cumulative explained variance
np.cumsum(pca.explained_variance_ratio_)
How much variance in the data is explained in total by the first and second principal component? What about the first four principal components? Using the visualization provided above, discuss what the first four dimensions best represent in terms of customer spending.
Hint: A positive increase in a specific dimension corresponds with an increase of the positive-weighted features and a decrease of the negative-weighted features. The rate of increase or decrease is based on the indivdual feature weights.
Answer:
Run the code below to see how the log-transformed sample data has changed after having a PCA transformation applied to it in six dimensions. Observe the numerical value for the first four dimensions of the sample points. Consider if this is consistent with your initial interpretation of the sample points.
# Display sample log-data after having a PCA transformation applied
display(pd.DataFrame(np.round(pca_samples, 4), columns = pca_results.index.values))
When using principal component analysis, one of the main goals is to reduce the dimensionality of the data — in effect, reducing the complexity of the problem. Dimensionality reduction comes at a cost: Fewer dimensions used implies less of the total variance in the data is being explained. Because of this, the cumulative explained variance ratio is extremely important for knowing how many dimensions are necessary for the problem. Additionally, if a signifiant amount of variance is explained by only two or three dimensions, the reduced data can be visualized afterwards.
In the code block below, you will need to implement the following:
good_data
to pca
.good_data
using pca.transform
, and assign the results to reduced_data
.log_samples
using pca.transform
, and assign the results to pca_samples
.# TODO: Apply PCA by fitting the good data with only two dimensions
pca = PCA(n_components=2)
pca.fit(good_data)
# TODO: Transform the good data using the PCA fit above
reduced_data = pca.transform(good_data)
# TODO: Transform log_samples using the PCA fit above
pca_samples = pca.transform(log_samples)
# Create a DataFrame for the reduced data
reduced_data = pd.DataFrame(reduced_data, columns = ['Dimension 1', 'Dimension 2'])
Run the code below to see how the log-transformed sample data has changed after having a PCA transformation applied to it using only two dimensions. Observe how the values for the first two dimensions remains unchanged when compared to a PCA transformation in six dimensions.
# Display sample log-data after applying PCA transformation in two dimensions
display(pd.DataFrame(np.round(pca_samples, 4), columns = ['Dimension 1', 'Dimension 2']))
A biplot is a scatterplot where each data point is represented by its scores along the principal components. The axes are the principal components (in this case Dimension 1
and Dimension 2
). In addition, the biplot shows the projection of the original features along the components. A biplot can help us interpret the reduced dimensions of the data, and discover relationships between the principal components and original features.
Run the code cell below to produce a biplot of the reduced-dimension data.
# Create a biplot
vs.biplot(good_data, reduced_data, pca)
Once we have the original feature projections (in red), it is easier to interpret the relative position of each data point in the scatterplot. For instance, a point the lower right corner of the figure will likely correspond to a customer that spends a lot on 'Milk'
, 'Grocery'
and 'Detergents_Paper'
, but not so much on the other product categories.
From the biplot, which of the original features are most strongly correlated with the first component? What about those that are associated with the second component? Do these observations agree with the pca_results plot you obtained earlier?
In this section, you will choose to use either a K-Means clustering algorithm or a Gaussian Mixture Model clustering algorithm to identify the various customer segments hidden in the data. You will then recover specific data points from the clusters to understand their significance by transforming them back into their original dimension and scale.
What are the advantages to using a K-Means clustering algorithm? What are the advantages to using a Gaussian Mixture Model clustering algorithm? Given your observations about the wholesale customer data so far, which of the two algorithms will you use and why?
Answer:
K-Means is very simple, fast and efficient in terms of computational cost. However, K-Means is strongly sensitivity to outliers and noise, and does not work well with non-circular cluster shapes. Also, it has low capability to pass the local optimum.
Gaussian Mixture Model (GMM) is a probabilistic model for representing normally distributed subpopulations within an overall population. It implements an expectation-maximization (EM) algorithm for fitting mixture-of-Gaussian models. Unlike K-Means, GMM allows for clusters to take elliptical shapes (elongated axes) and mixed membership of points to clusters (soft clustering). Therefore, it is much more flexible than K-Means.
Looking at the biplot, it is difficult to visualize cicular clusterings with hard boundaries separating the data points. Based on the observation, GMM would be a better option due to its capability to allow for mixed membership of points to multiple clusers.
Depending on the problem, the number of clusters that you expect to be in the data may already be known. When the number of clusters is not known a priori, there is no guarantee that a given number of clusters best segments the data, since it is unclear what structure exists in the data — if any. However, we can quantify the "goodness" of a clustering by calculating each data point's silhouette coefficient. The silhouette coefficient for a data point measures how similar it is to its assigned cluster from -1 (dissimilar) to 1 (similar). Calculating the mean silhouette coefficient provides for a simple scoring method of a given clustering.
In the code block below, you will need to implement the following:
reduced_data
and assign it to clusterer
.reduced_data
using clusterer.predict
and assign them to preds
.centers
.pca_samples
and assign them sample_preds
.sklearn.metrics.silhouette_score
and calculate the silhouette score of reduced_data
against preds
.score
and print the result.reduced_data.head()
from sklearn.mixture import GMM
from sklearn.metrics import silhouette_score
print "Silhouette scores for n clusters"
for i in range(2, 6):
# TODO: Apply your clustering algorithm of choice to the reduced data
clusterer = GMM(n_components=i).fit(reduced_data)
# TODO: Predict the cluster for each data point
preds = clusterer.predict(reduced_data)
# TODO: Find the cluster centers
centers = clusterer.means_
# TODO: Predict the cluster for each transformed sample data point
sample_preds = clusterer.predict(pca_samples)
# TODO: Calculate the mean silhouette coefficient for the number of clusters chosen
score = silhouette_score(reduced_data, preds)
print "{} clusters: {}.".format(i, score)
Report the silhouette score for several cluster numbers you tried. Of these, which number of clusters has the best silhouette score?
Answer:
As displayed above, the silhouette score for 2 clusters is the highest.
Once you've chosen the optimal number of clusters for your clustering algorithm using the scoring metric above, you can now visualize the results by executing the code block below. Note that, for experimentation purposes, you are welcome to adjust the number of clusters for your clustering algorithm to see various visualizations. The final visualization provided should, however, correspond with the optimal number of clusters.
clusterer = GMM(n_components=2).fit(reduced_data)
preds = clusterer.predict(reduced_data)
centers = clusterer.means_
sample_preds = clusterer.predict(pca_samples)
# Display the results of the clustering from implementation
vs.cluster_results(reduced_data, preds, centers, pca_samples)
Each cluster present in the visualization above has a central point. These centers (or means) are not specifically data points from the data, but rather the averages of all the data points predicted in the respective clusters. For the problem of creating customer segments, a cluster's center point corresponds to the average customer of that segment. Since the data is currently reduced in dimension and scaled by a logarithm, we can recover the representative customer spending from these data points by applying the inverse transformations.
In the code block below, you will need to implement the following:
centers
using pca.inverse_transform
and assign the new centers to log_centers
.np.log
to log_centers
using np.exp
and assign the true centers to true_centers
.# TODO: Inverse transform the centers
log_centers = pca.inverse_transform(centers)
# TODO: Exponentiate the centers
true_centers = np.exp(log_centers)
# Display the true centers
segments = ['Segment {}'.format(i) for i in range(0,len(centers))]
true_centers = pd.DataFrame(np.round(true_centers), columns = data.keys())
true_centers.index = segments
display(true_centers)
# Plot centers and median values
true_centers = true_centers.append(data.describe().loc['25%'])
true_centers = true_centers.append(data.describe().loc['50%'])
true_centers = true_centers.append(data.describe().loc['75%'])
true_centers.plot(kind = 'bar', figsize = (16, 4))
Consider the total purchase cost of each product category for the representative data points above, and reference the statistical description of the dataset at the beginning of this project. What set of establishments could each of the customer segments represent?
Hint: A customer who is assigned to 'Cluster X'
should best identify with the establishments represented by the feature set of 'Segment X'
.
Answer:
Looking at the cluster visualization plot and the biplot, the two clusters would be well distinguished by Dimension 1. Cluster 1 customers have high spendings on product categories, detergent/paper, grocery and milk, and low spendings on fresh and frozen. Cluster 0 customer spending patterns are vice versa.
For each sample point, which customer segment from Question 8 best represents it? Are the predictions for each sample point consistent with this?
Run the code block below to find which cluster each sample point is predicted to be.
# Display the predictions
for i, pred in enumerate(sample_preds):
print "Sample point", i, "predicted to be in Cluster", pred
for i,j in enumerate(pca_samples):
print "Probability of Sample {}: {}".format(i,clusterer.predict_proba([j])[0])
Answer:
Based on the observations made, my predictions are:
The predicted results differ for Sample point 2.
In this final section, you will investigate ways that you can make use of the clustered data. First, you will consider how the different groups of customers, the customer segments, may be affected differently by a specific delivery scheme. Next, you will consider how giving a label to each customer (which segment that customer belongs to) can provide for additional features about the customer data. Finally, you will compare the customer segments to a hidden variable present in the data, to see whether the clustering identified certain relationships.
Companies will often run A/B tests when making small changes to their products or services to determine whether making that change will affect its customers positively or negatively. The wholesale distributor is considering changing its delivery service from currently 5 days a week to 3 days a week. However, the distributor will only make this change in delivery service for customers that react positively. How can the wholesale distributor use the customer segments to determine which customers, if any, would react positively to the change in delivery service?
Hint: Can we assume the change affects all customers equally? How can we determine which group of customers it affects the most?
Answer:
Two customer clusters show different spending behaviors on the six product categories. Depending on the products the customers purchase, reduction of delivery frequency can negatively impact the customers. We can perform A/B test each cluster separately. Test A can be 5 day delivery, and Test B can be 3 day delivery. The test results can be analyzed respectively within the cluster to see if there is any difference in customer satisfaction between 5 day (Test A) and 3 day (Test B) deliveries.
It would be intersting to provide an incentive to 3 day delivery by lowering the shipping cost since only 3 deliveries are made.
Additional structure is derived from originally unlabeled data when using clustering techniques. Since each customer has a customer segment it best identifies with (depending on the clustering algorithm applied), we can consider 'customer segment' as an engineered feature for the data. Assume the wholesale distributor recently acquired ten new customers and each provided estimates for anticipated annual spending of each product category. Knowing these estimates, the wholesale distributor wants to classify each new customer to a customer segment to determine the most appropriate delivery service.
How can the wholesale distributor label the new customers using only their estimated product spending and the customer segment data?
Hint: A supervised learner could be used to train on the original customers. What would be the target variable?
Answer:
A supervised learner could be used to train original customers and to make predictions on the new customers. The target labels 0 and 1 can indicate the customer segment, and the 6 product categories can be the input features. The predicted classification result for the new customers can be used to make decisions on delivery services and other contract policies that may differ between the two customer segments.
Another idea is to sue the subset of the newly engineered PCA components as new features to reduce the dimension of the problem. This would be helpful computationally as the number of features increase.
At the beginning of this project, it was discussed that the 'Channel'
and 'Region'
features would be excluded from the dataset so that the customer product categories were emphasized in the analysis. By reintroducing the 'Channel'
feature to the dataset, an interesting structure emerges when considering the same PCA dimensionality reduction applied earlier to the original dataset.
Run the code block below to see how each data point is labeled either 'HoReCa'
(Hotel/Restaurant/Cafe) or 'Retail'
the reduced space. In addition, you will find the sample points are circled in the plot, which will identify their labeling.
# Display the clustering results based on 'Channel' data
vs.channel_results(reduced_data, outliers, pca_samples)
How well does the clustering algorithm and number of clusters you've chosen compare to this underlying distribution of Hotel/Restaurant/Cafe customers to Retailer customers? Are there customer segments that would be classified as purely 'Retailers' or 'Hotels/Restaurants/Cafes' by this distribution? Would you consider these classifications as consistent with your previous definition of the customer segments?
# Find percentage of correctly classified customers
data = pd.read_csv("customers.csv")
data = data.drop(data.index[outliers]).reset_index(drop = True)
df = np.where(data['Channel'] == 2, 1, 0)
print "Percentage of correctly classified customers: {:.2%}".format(sum(df == preds)/float(len(preds)))
Answer:
It is quite amazing that the unsupervised clustering could actually capture the underlying distribution of the two customer types. The percentage of correctly classified customers is 85.52%. Customers that are extremely high or low in Dimension 1 would fall into purely Retailer or HoReCa, respectively. However, there are mislabeled customers that range between -2 and 3 in Dimension 1.
I think it would be interesting to perform further analysis within the two customer segments (Retailer or HoReCa) to understand what makes them behave more like the other (e.g., a retailer that has spendings similar to HoReCa and vice versa). A supervised learnging algorithm could be applied to only HoReCa with labels indicating those behaving like HoReCa (those predicted as Segment 0 by GMM and are actually HoReCa) and those behaving like Retailer (those predcted as Segement 1 by GMM but actually HoReCa). New features (or information) may need to be added to identify latent features involved with subcategorization.
Note: Once you have completed all of the code implementations and successfully answered each question above, you may finalize your work by exporting the iPython Notebook as an HTML document. You can do this by using the menu above and navigating to
File -> Download as -> HTML (.html). Include the finished document along with this notebook as your submission.


--- layout: barebones ---
Go-IoT specializes in bringing BACnet enabled buildings to the cloud, securely and safely using the BACnet/WS standard.
It's implementation of the BACnet/WS standard enables developers and users to integrate BACnet information into other applications, such as Accounting Systems, Facility Management Systems and Cognitive Control without deep BACnet knowledge.
It also makes accessing BACnet-enabled buildings location independent as BACnet/WS can be used locally in the building as well as remotely from the cloud.
Node-RED makes it very easy to use BACnet/WS. It allows customers to create business logic that operates both on-site and in the cloud to retrieve data from the BACnet enabled buildings as well as controlling the systems within.
It's visual graphical flow development system is very suitable for creating control logic for sensors and actuators in buildings. Go-IoT BACnet/WS implementation forms the glue between BACnet and Node-RED locally in the building or remotely in the cloud.
To extend previous capabilities even further Go-IoT supports also a BACnet to IBM Watson IoT Platform Gateway, to make it possible to use Node-RED to communicate with BACnet directly from IBM Watson using the MQTT standard.
For further information about this product refer to those links:


Cross-complementary
Buttons
complementary 1 complementary 2 complementary 3 complementary 4Links
complementary 1 complementary 2 complementary 3 complementary 4Custom usage
Custom class from function: [background-color: get-palette-color(2, "triad");]
Custom button from mixin: [@include get-palette-button(4, "monochrome", #fff);]
Main heading h1
Sub-heading h2
Bacon ipsum dolor amet hamburger shankle ground round, brisket tail landjaeger frankfurter pork belly ham. Ground round venison bresaola corned beef. Pork loin fatback landjaeger kielbasa rump bresaola. Turkey andouille kevin pastrami pork loin venison short ribs bresaola tenderloin.
- Kevin rump meatloaf ball tip sausage brisket boudin bacon.
- Andouille chicken porchetta ham hock, venison spare ribs frankfurter filet mignon.
- Shoulder pastrami tongue fatback short ribs picanha. Meatball ribeye flank, chicken picanha salami ball tip. Tenderloin doner beef ribs pastrami, bacon chuck cow.
Drumstick cow biltong swine. Shoulder pork chop alcatra capicola prosciutto filet mignon porchetta salami turkey.


Question: How can I track my order?
A) Once you have submitted your order, you will receive an order confirmation by email to confirm that your order has been received. This email will contain your order number, you can tracke using that order iD.
Question: How can I change/cancel my order?
A) nfortunately, once an order has been shipped, it cannot be changed or cancelled. If you are not fully satisfied with your purchase, you will have the option of returning it.
Question: How can I return/exchange my order?
A) Please allow 1-2 business days for handling time. Once you receive a shipping confirmation email, your item (s) will be delivered depending on the shipping method you select.
Question: What are your size conversions?
A) Our footwear is labeled with US sizes.
Question: How can I locate styles available in my size?
a) You can shop our full collection by size by using the SHOP BY SIZE filter on the webpage.
Question: Is it safe to shop on our site?
A) yes.. it is safe to shop on our site.
Question: Are there any stores available?
A) no.. not yet started any physical stores.


--- layout: home author_profile: true header: image: /assets/images/sample_head.png ---
I have put on various hats over the course of my career. I am trained as a physicist and pursued research both in academia and in industry. In general, I am interested in applying mathematical models and computer simulations to various problems, as well as building computational work flows. My current focus is on using data analytic methods and their applications in industry.


We believe that helping people to reach their goals is the best way to accomplish ours. At Plus Collective we can help you to strengthen your connections with your audience by developing inspiring communication tools.
Check out who we are and how we work.
Development and
Communication
Graphic Design and
ideas
We have a (virtual) coffee, chat and get to know you, your project and its necessities
By brainstorming, exploring and sharing ideas we design a strategy that will provide the solution you are looking for.
We bring our ideas to life, giving our best to provide optimal results.
With a rigorous approach we test everything we make to guarantee successful results.
We ensure the delivery and publication of your product or brand identity, always aligned with the strategy and your necessities.
We will be there for you. We want to know how our tools enhanced the relation with your clients.
Copyright 2020, PLUS COLLECTIVE
All rights reserved. Terms & conditions | Privacy


Billy Buchanan
Director of Data, Research, and Accountability
Fayette County Public Schools
Overview
Background
Use Cases
Doing data mining
Your Work
Ask questions as you think of them.
Honestly, I don't mind and don't bite.
I'll be switching back and forth between the slide deck and some programming environments.
If you need me to go back over something or to flip back to the previous screen feel free to ask.
Background
Why should you care about this?
$1,507,849
$2,913,122
$548,435
$625,452
$770,304
Why should you care about this?
$1,507,849
$2,913,122
$548,435
$625,452
$770,304
What is data mining?
Exploratory Data Analysis
Text/Sentiment Analysis
Machine Learning/Inferential Statistics
Types of ML
Supervised machine learning is used to build predictive models when the outcomes are observed.
Unsupervised machine learning is used to classify observations into unobserved groups.
Were staff actually using the assessment data the district was paying so much for?
Could we classify users into distinct groups based on observable behavior?
How can we learn more about the user experience without having to do tons of surveys?
Understanding and Improving the User Experience
If you're developing reports, you should want to answer these questions:
Are users making use of the interactive components?
Are there specific pathways that lead users to leverage some of these tools more frequently?
What effect does changing one of the user interface elements have on the overall user experience?
Monitoring. The Vendor shall describe in detail what capabilities
the BI platform has to collect usage metrics from end-users using the product. This should
specifically address the level of detail that is possible to record (e.g., can the BI
platform allow us to track when specific elements in the user-interface are used, can this
be time and user stamped).
A/B Testing. The Vendor shall also discuss whether or not the BI
platform provides any native support or functionality to implement A/B testing. If the
platform does not support this functionality, the Vendor shall discuss how the support
clients testing and revising user interface elements and interaction functionality in
reports and dashboards.
UI/UX Collection. It is expected that [we] will have the ability to
collect extensive telemetry data to develop a more robust understanding of the user
experience, for continuous improvement of the reports delivered to end-users, and to more
carefully evaluate how data use and access affects teaching and learning. The Vendor shall
describe in detail what capabilities exist to collect these data, how those data would be
stored, and the level of detail at which these data can be collected. If the proposed
solution does not include the capacity to capture and store data on interactions with user
interface “widgets”, controllers, forms, and event-listener firings, the Vendor shall
propose methods by which [we] may accomplish this goal.
Funnel/Click-Thru. The logging and data collection capabilities of
the system must allow [us] to conduct funnel and/or click-thru analyses. The collection
requirements at this level are time- and user-stamped records that include the name/id of
the report being viewed and a session ID that is sequential by user and day. These data
should be collected from the point at which the user logs into the system and continue
until the user logs out or a timeout disconnects the user from the system.
Analyzing the Analyzers
How similar are the research/data requests that your department gets?
Are there seasonal components in the requests (e.g., specific requests happen regularly at certain points of the year)?
How does the quality/clarity of the request/research questions vary by end users?
Conversion/Click-Thru
When users go to your organization's website, where do they go from there?
Do highly effective educators navigate dashboard systems in similar ways to less experienced/skilled educators?
How long are users taking to view specific dashboards/reports?
Current/Historical Legislative Trends
How easy is it for leaders in your organization to quickly identify and see proposed legislative changes?
Using open APIs like Legiscan we can track state legislation and mine the text to identify trends in the proposed legislation
This can also make it easier to keep your parents and community members informed about proposed changes to laws that effect the education sector in your community.
These user interaction data are all generated from an R/Shiny-based application I put together
to help visualize the Mississippi's Statewide Accountability System (MSAS).
The code for
the tool is available here: MSASExplorer


The songview package provides a NetBeans Window that shows a song
in classical track representation.
Coordinate Systems
There are several coordinate systems involved when drawing the songview.
Window coordinates
TODO describe Window coordinates
Canvas pixel coordinates
TODO describe Canvas pixel coordinates
Canvas Midi coordinates
TODO describe Canvas Midi coordinates


Note that the current implementation is unsafe (eval), this code should not be used in a production website. It's OK for a local util though...


--- layout: post title: "Improving Typesetting" desc: "" thumbnail: "" categories: article tags: {} ---
With Word being regarded as the literal devil by universities (and with good reason), alternatives providing consistent output with high quality typesetting are quite popular.
LaTex now being the standard was originally released in 1985, and it's issues and limitations are starting to show.
Most notable LaTex's typesetting is just really good, but what constitutes high quality typesetting?
I’m currently working on a project requiring near perfect typesetting, but can we do more?
...I don’t know, I’m not specialist with typesetting, yet.
(** Update: Ongoing research https://github.com/MGApcDev/mlTex/issues/8 **)


New 1.0 turbo-petrol engine to power Renault Triber, upcoming HBC SUV
by Staff WriterNew Renault-Nissan HR10 1.0-litre turbo-petrol engine will sell alongside the existing BR10 unit.
The Renault Triber has got off to a good start in India with sales crossing 18,500 units since its launch on August 28, 2019. The Triber’s flexible interior that offers seating for seven, and keen pricing have seemingly found favour with buyers. At the Triber's unveil in July, we had reported that Renault was working on a more powerful turbo-petrol version of the sub-four-metre 7-seater. We can now confirm fresh details of the much-awaited, turbo-petrol engine that will make its debut in India sometime in 2020.
Powering the Triber turbo, as well the upcoming HBC compact SUV, will be an all-new Renault-Nissan-developed, 1.0-litre, three-cylinder, turbo-petrol engine that carries the code ‘HR10’. Essentially a three-cylinder version of the four-pot HR13, 1.3-litre turbo-petrol unit, the engine will power a range of Renault and Nissan products in India from 2020. While the official power figures have not yet been released, insiders involved with the project say the engine will make close to 95hp. The three-cylinder engine is also said to be fairly sophisticated with dual variable valve timing and an exhaust manifold partially integrated into the cylinder head.
The new turbo engine will be sold alongside the existing 72hp and 96Nm, 1.0-litre naturally-aspirated petrol engine (code: BR10) that will receive an update before the BS6 emission norms kick in.
Renault is also working on an AMT auto option for the Triber, though it remains to be seen if the transmission will be offered on the turbocharged versions.
Also see:
Up to Rs 1.5 lakh off on Renault Duster diesel
Renault gifts modified Duster to the Vatican
Renault Kwid, Duster, Lodgy now get 7-year extended warranty
Renault Triber
Rs 5.48 lakh * on road price (New Delhi)


Worked with Python, Django, Javascript, and AngularJS to develop a single page, user-based web application that manages multiple online marketing entities.
- Created views for affiliates, advertisers, campaigns, and offers using Angular directives and Django serializers and models
- Developed settings page, which supports system-wide as well as user-specific settings
- Created page that utilizes Amazon Redshift to manually add, remove, and edit conversions
- Supported user-specific permissions as well as frontend and backend validation.
Improved on Ruby on Rails application, founded by Laurene Powell Jobs, that assists underprivileged highschool students with getting into college.
- Behavior Driven Development using RSpec and Cucumber
- Worked with Agile practices - User stories and SCRUM meetings.
- Created page and model to allow admins to save, edit, and remove custom reusable search queries based off gender, age, gpa, and other criteria.
Created single page application for Obj-C fusion energy project ICEE, facilitating the observation of images taken from Korea's KSTAR fusion energy reactor.
- Used UIKit and Autolayout to create a side panel specifying the folder structure, a main display, and a button that grabs the images generated by KSTAR.
- Recreated GNU Plot functionality using Core Graphics in order to adjust coloration and precision of images
- Created video interface using Core Animation in order to smoothly display image trends
Worked on key calculations in C involving atmospheric rivers, which account for 80% of severe weather changes in the bay area.
- Improved the accuracy of geodesic dimension calculation for atmospheric rivers by over 25%
- Handled cutting-edge technology, including the Lawrence Berkeley National Labs supercomputer
Created instagram-like application in Swift specifically geared towards daily and recurring events
- Utilizes multiple iOS tools - Autolayout, UIKit, MapKit, Core Animation, Core Graphics, and custom-made frameworks.
- Created login page, main menu, events page, and personal profile.
- Ongoing development and business plan
Created watch/phone Android application which promotes responsible drinking by calculating BAC via user input (number/type of drinks consumed, gender, height, and weight)
- Features a user library of drink options as well as a user profile
- Includes a feature that enables the user to quickly call a cab when the user reaches a certain BAC
- Designed with an easy to use interface for both the watch and phone


Hi, This is Anup. I live in San Francisco,CA.I have been in SF for 3 years now.I moved here from Detroit.I am currently working at Credo Mobile as a Quality Assurance Engineer.I am originally from India.
These are some of my hobbies:-
Here are list of my favourite TV shows:
Links to my favourite sites:
Click on Link below to learn more about the New Bay Bridge-
If you have comments or suggestions send me an Email


CRAZY IN ALABAMA A film review by Steve Rhodes Copyright 1999 Steve Rhodes RATING (0 TO ****): *
Antonio Banderas, fresh from his awkward performance in the disastrous THE THIRTEENTH WARRIOR, makes his inauspicious directorial debut in CRAZY IN ALABAMA. A cinematic hash of stale ingredients lifted from other movies, CRAZY IN ALABAMA includes a wacky woman, a cheesy chase, a racial rebellion, an aspiring actress and stupid-sounding Southerners. It's a comedy that's not even close to being funny and a drama about the civil rights movement that's not the least bit moving.
The picture, set in the summer of 1965, stars Banderas's real-life wife Melanie Griffith as Lucille, a murderer with seven kids less than eight years old -- her husband, she claims, kept putting holes in her diaphragm.   Griffith seems to specialize these days in giving clichéd performances in roles that are already written as little more than caricatures.
 She's chosen Carolyn Clay as her stage name since it's "kind of like Marilyn Monroe, only different." Did I mention that the boxed head talks to her?
Meanwhile, back at the local swimming pool, some "Negroes" are daring to swim with the whites. Prejudiced Sheriff John Doggett (Meat Loaf) stops by to spout such classic lines as, "You got no rights in my town, boy." The sheriff unintentionally kills one of the black boys when the boy tries to escape. The sheriff claims the boy fell and doesn't admit that he was pulling him off of the fence at the time.
Lucille's nephew Peejoe (Lucas Black) is a 13-year-old who is troubled by discrimination. He complains to his Uncle Dove (David Morse) that Negroes shouldn't be kept from registering to vote. "You're right," his uncle tells him. "It's not fair. It's just the way things are." Peejoe ends up being a hero for telling the truth about the swimming pool incident.
Mark Childress's script, based on his novel, is a treasure trove of eclectic lines -- sometimes ponderously pretentious and other times overly cute. ("Life and death are only temporary, but freedom goes on forever," Peejoe lectures us in voice-over. "I just love the sounds of an authentic peasant accent," a Hollywood socialite, played by Elizabeth Perkins, says of Lucille's speech.)
Rod Steiger drops by towards the end to chew up the scenery as the local judge. Steiger tries without much luck to breathe life into the picture by recasting it as an over-the-top parody. In a movie that wastes vast amounts of money authentically recreating the mid-1960s, why do they let Steiger wear the same 1990s designer frames that he has in real-life? When I met him earlier this year, he had on these exact glasses.
Does this abysmal movie have any saving grace? Yes. The dazzling, vintage automobiles, like Lucille's cherry red 1964 Ford Galaxy, look so new that you'd swear no one has ever ridden in them before. The bright, primary colored cars shine with the same radiance that they must have given out on the day that they left the factory floor. Enjoy them as much as you can. They're the best part of the production.
CRAZY IN ALABAMA runs a long 1:51. It is rated PG-13 for some violence, thematic material, language and a scene of sensuality and would be acceptable for kids 10 and up.
Email<EMAIL_ADDRESS>Web: http://www.InternetReviews.com
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


Job abstract
- customerJobKey - SOAP::SOAPString
- context - AdWords::V201008::BulkMutateJobService::JobContext
- failureReason - AdWords::V201008::BulkMutateJobService::ApiErrorReason
- stats - AdWords::V201008::BulkMutateJobService::JobStats
- billingSummary - AdWords::V201008::BulkMutateJobService::BillingSummary
- job_Type - SOAP::SOAPString
Methods
- N
Attributes
| [RW] | customerJobKey | |
| [RW] | context | |
| [RW] | failureReason | |
| [RW] | stats | |
| [RW] | billingSummary | |
| [RW] | job_Type |
Class Public methods
new(customerJobKey = nil, context = nil, failureReason = nil, stats = nil, billingSummary = nil, job_Type = nil)
Source: show
# File lib/adwords4r/v201008/BulkMutateJobService.rb, line 3120 def initialize(customerJobKey = nil, context = nil, failureReason = nil, stats = nil, billingSummary = nil, job_Type = nil) @customerJobKey = customerJobKey @context = context @failureReason = failureReason @stats = stats @billingSummary = billingSummary @job_Type = job_Type end


.NET Core and JavaScript Encryption Decryption Library
(c) by Smart In Media, MIT License
Let's assume, the JS now received the data from C#:
We have:
Encrypted data in C# results in a JSON string, which can be also decrypted on the JS side (and vice versa):
'{
"DerivationType": "scrypt",
"Salt": "14beb3414afb8a1de8604de90a05407de29a280336a7b4aee5876bda55ab137f",
"Cost": 16384,
"BlockSize": 8,
"Parallel": 1,
"KeySizeInBytes": 32,
"DerivationIterations": 0,
"AesRijndaelIv": "35Gtt04sMAzf+9lMNDPr+Q==",
"CipherOutputText": "hWvekGAS39AN8SAyHbfjXYlyjhKPQXE2zOmgJVVpihE="
}'
and the passphrase: This_is_my_password!
Please note that the C# side also has a password generator (PasswordGenerator.GenerateRandomPassword(20); for password length of 20 characters, you can define the character set)
Please load the file "cartman.enc" with the File dialog button.
This is a test of the "cartman.enc" file, which was encrypted with C#, will be correctly decrypted by JS.
To decrypt correctly, JS not only needs the password, but also the settings (for SALT, IV, etc), which
can be passed with a JSON: var jsonBinarySetting =
{"DerivationType":"scrypt","Salt":"36fa904885f19e78152aa73366c40cc6df86aa2de20dbc5072aee7a912bdf142","Cost":16384,"BlockSize":8,"Parallel":1,"KeySizeInBytes":32,"DerivationIterations":0,"AesRijndaelIv":"FWhL0yPPay662/9IoE0ICw==","CipherOutputText":null};
Load file "cartman.enc" with button:
There is also an implementation for Scrypt. Scrypt is a new(er) way to hash passwords more securely, if
you want to store them in your database, etc. Scrypt takes longer to hash - you can set it according
to hardware performance. I. e. you can define that hashing a password takes a second on a fast machine.
The purpose of that is to make brute force (and dictionary) attacks considerably harder for attackers.
You can run Scrypt asynchronously and get a progress (plus you can cancel) by passing a callback function.
Scrypt will pass the hash key to that callback function. If you omit the callback function, then Scrypt runs
synchronously and returns the key. So either, you run sH.Hash(password, options, callback);
or you run var key = sH.Hash(password, options);
Progress:
Please load the file "./zip-test-data/test.zip" with the File dialog button.
This is a test of the "test.zip" file, where an image is included, which was encrypted with the standard ZIP AES. The image was not ZIP compressed (compression = store).
The offset and length of the to extract the image are offset: 1264, end: 146929 (length = 146929 - 1264 + 1 = 145666).
Password: 3rQA6zJX-sQwHT4iA-GWDoYjXn-Cuow6uBp


{%extends "adm/admin_base.html" %} {%load currency%} {%block title%}Pending bank transactions{%endblock%} {%block layoutblock %}
Here you will find a list of all bank transactions that are not matched to anything, as well as any pending matchers that have not yet matched.
Pending bank transactions are transactions that have showed up on any of the configured bank accounts, regardless of provider, but has not yet been processed. This can be because the details didn't match (e.g. wrong amount of payment reference), or because they were simply meant to be manually handled.
| Added | Amount | Text | Method | Comments | Action |
|---|---|---|---|---|---|
| {{trans.created}} | {{trans.amount|format_currency}} | {{trans.transtext}} | {{trans.method.internaldescription}} | {{trans.comments}} |
Pending matchers are regexps that other parts of the system have registered as "listening" for an expected transaction on a bank account. When this transaction shows up it will be automatically matched and the associated accounting record will be closed.
| Added | Account | Pattern | Amount | Journal entry | Action |
|---|---|---|---|---|---|
| {{matcher.created}} | {{matcher.foraccount}} | {{matcher.pattern}} | {{matcher.amount|format_currency}} | {{matcher.journalentry}} |


{{ rendezvous.title }}
{{ rendezvous.theme }}{{ rendezvous.date }}, {{ rendezvous.year }}
{{ rendezvous.location }}
{{ rendezvous.address }}
{{ rendezvous.city-state_zipcode }}
{{ rendezvous.phone }}
{{ rendezvous.website }}
{{ rendezvous.reervations }}
{{ rendezvous.early-bird-close }}
{{ rendezvous.early-bird-one-day }}
{{ rendezvous.early-bird-two-day }}
{{ rendezvous.membership-one-day }}
{{ rendezvous.membership-two-day }}
{{ rendezvous.non-member }}
{{ rendezvous.payment-online }}
{{ rendezvous.payment-mail-in }}
{% endif %} {% endfor %}


"
Beautiful hands are those that do
Work that is earnest, and brave and true
Moment by moment
The long day through"
~Abdul Kalam
Worked on developing survey6 tool from scratch which is aimed to help. Cyber-security researchers in study of IPv6 packets. Established communication between server and probe using gRPC. Coded mechanism to capture IPv6 packets and sending them over to server. Provided mechanism for probes to register them on server and storing their info in SQLite database. Created system service and provided bash scripts for setup.
Worked on testing real-time payment application that is built upon micro-services. Developed a generic db tables comparison viewer which outputs an excel sheet. Highlighting the difference in values mainly using spring boot, H2, jpa. Created an end to end independent module for json payload comparison.
Co-developed electric appliances automation mobile application in flutter. Set up communication of the application with ESP8266 NodeMCU that is attached to each device. Provided control features for UV sanitizer, fan and light.
 Exploring new places and eateries in the town, cycling and driving!! For me, books win over the web shows. 📚❤
Apart from that I also love cooking. Occasionally I love to visit nature: grateful to my strategic location between the mountains and beaches 🌊. I am usually a part of any of the active organizations working for social good. And ohh, I so am so much taken in by solving daily life issues through tech or otherwise too.


Errors and warnings
Maximum operation count reached
This error happens when a thread executes more than the configured maximum of synchronization actions. This typical happens when the test contains a not terminating loop.
You can set the maximum by using the method maximumSynchronizationActionsPerThread of the class AllInterleavingsBuilder. In the following example the value is set to 1000:
try (AllInterleavings testUpdate = AllInterleavings. builder("TestShowSharedMemory"). maximumSynchronizationActionsPerThread(1000). build();) { while( testUpdate.hasNext() ) { } }
The default value is 2000. Set the maximum to -1 to never terminate a thread with this mechanism.


#^html
The CRS data comes from CRS tables.
Other data is stored in a Google ctrack spreadsheet.
d-portal attempts to only reflect what is in the data and does not make corrections to data or information published to IATI, with only small exceptions to make the site user-friendly*
See also information about data quality (Afghanistan used as example)
Mismatch between ‘Identifier’ in publisher’s profile in the IATI Registry, and reporting-org in that publisher’s data – this will show zeroes in the publisher’s data until the two are matched.
Transaction type code not present.
Not using either Disbursements or Expenditure in transaction type field.
Dates in incorrect format in the data.
Illogical dates such as activity end date before activity start date.
No data relevant to the year being viewed on a pie chart or table (eg where a pie chart shows 2016 data and data published only relates to 2015).
If updating your Registry profile only, changes will show after the weekly full import (each weekend) as d-portal only updates nightly the data that has been changed in the xml.
Dashes show that that donor/organisation is not currently an IATI publisher.
CRS data is downloaded annually and then stored in ctrack.
Look at the issues on GitHub.
Contact us -<EMAIL_ADDRESS>‘Budgets’ in activity file, for Budget columns/data (eg in tables on d-portal) - if no budget data is present in the activity file, then ‘planned-disbursements’ are used if present.
Using the end date in the budget data (eg in Budget columns in tables).
Projects with invalid date ranges that cannot be placed in one of the other 3 categories (Active, Planned, Ended).
Currently (please note this may change soon) converting to USD using the exchange rate of the date specified in the value-date attribute. So when we display USD, it is very accurate to that date.
For other currencies, the USD value is just converted using today’s exchange rate (please note this may change soon).
Using the latest exchange rates (International Monetary Fund) every night and doing a full re-import once a week, so future budgets will use the current exchange rates and should be updated weekly.
For tables - as above, with transaction date relating to the relevant year column in the table.
For maps - any transactions (from any year) with valid location tag and recipient country code (to be valid it needs to be on the relevant IATI codelists).
1. Filters
These are a selection of filters relating to the IATI Standard allowing users to choose and refine the IATI data they want to explore.
2. Freetext
This is a freetext box allowing users to search for themes and topics, and specific activities published to IATI.
You can then click ‘Explore Results’ to find out more about those activities.
The organisations that have published to the IATI Standard, and the countries where funds have gone to as published within IATI data.
Sector Group and Sector
The numeric codes and names to describe sectors within the OECD DAC CRS, as used within the IATI Standard.
Activity Status
The numeric codes and names within the IATI activity status field.
Year Range
Years from 1960 to ‘present year + 2’.
Click ‘Explore Results’ to see lists, graphs and maps allowing further exploration of the data.
Refine your search by adding multiple words – e.g. ‘Hurricane Matthew’ will find more refined (fewer) results than ‘Hurricane’.
For more information see http://www.d-portal.org/news/2014-11-30-Interactive-IATI-data-d-portal-generator.html
The only time this can go wrong is if your website provider disallows you to include external widgets in your blog posts, ie. if you can embed a youtube video into your blog post, you can embed generator content in your post.
Use https://d-portal.org/ctrack.html#view=generator instead.
If you are unsure if this affects you, see if your company's website uses https:// instead of http:// in the website address.


tap: In the Toolkit, touching a {_control} for less than half a second, without moving. On a {_button} or help {_link_help link}, this invokes the normal function. On other controls, it can do other things.
Also: {_long-press}, {_drag}, {_long-drag}


This is a Web Implementation of ArdiMasters Stellaris Stat Viewer. You can get going by uploading a gamestate file
Nobody here but us chickens.


Overview
We develop statistical models to assess trends and differentials in key indicators in reproductive, maternal and child health and demography, focusing on two main areas:
- Family planning ~ contraceptive use and need, unintended pregnancies, abortions, fertility
- Child and maternal mortality
Model findings include population-level estimates of demographic and population-health trends as well as the description of global patterns of change, the identification of populations with outlying levels or trends, and comprehensive assessments of data quality.
We gratefully acknowledge funding for family planning research by the Bill and Melinda Gates Foundation (PI Alkema) and for data quality issues in sibling histories by NIH (PI: Helleringer). We are passionate about collaborating with substantive experts and making available improved estimation methods and resulting estimates to diverse international audiences. To that end, we collaborate with various United Nations groups and organizations in the areas of global health and family planning. The statistical methods that we have developed for child mortality, maternal mortality, and family planning indicators [preobably some refs in those] are in use by relevant UN agencies and WHO for monitoring. Child and maternal mortality methods [refs] have been used to produce estimates for the final assessment of UN Millennium Development Goals and are in use for monitoring progress toward the Sustainable Development Goals. Family planning estimation methods [refs] are being used for monitoring in the FP2020 initiative by Track20 using the web-based application FPET (see fpet.track20.org). Lastly, we collaborate with \textit{the Guttmacher Institute} (link) on the estimation of abortion incidence and pregnancy rates, with intention status.
Summary Table
| Area | Group/initiantive | Main methodological contributions |
|---|---|---|
| Family planning | UNPD, FP2020 | The family planning estimation tool... |
| 2 | Jacob | Thornton |
| 3 | Larry | the Bird |
Details
Main methodological challenges in this area of research are: - \item How to best use diverse data types with varying levels of bias and/or random measurement errors? - \item How to capture data-driven trends in data-rich populations, while also producing reliable estimates for populations where data are limited, with appropriate levels of uncertainty? - \item How to identify populations with outlying levels or trends? - \item How to construct probabilistic projections that are unbiased and well-calibrated? Our research has addressed these methodological challenges for various indicators via the conceptualization, development, and validation of context-specific Bayesian models. Key components of these models are: - \item the parameterization of biases and measurement error variances to account for data quality issues. - \item the specification of models that capture systematic trends or relationships, - \item hierarchical submodels to allow for ``sharing'' of information across populations to inform the estimates and projections for data-sparse populations, and Model findings include population-level estimates of demographic and population-health trends as well as the description of global patterns of change, the identification of populations with outlying levels or trends, and comprehensive assessments of data quality. Additional methodological contributions to the field include guidance on model validation \ssp{}, population-specific target setting \target{}, as well as the reporting and interpretation of population estimates and associated uncertainty \mikkel{} \cdata{}. We also contributed to GATHER, a new set of consensus guidelines which have been adopted by leading journals. Some family planning paper


Aeris.js components can all be loaded using an AMD loader (eg. Require.js). Take a peek at src/app.js
for an example of how to configure Require.js and load Aeris.js AMD modules.
Bower.js can be used to automatically download all of the dependencies for Aeris.js. Running bower install
from command line will install dependencies in the bower_components
directory.
apikeys.js
file, with a valid Aeris client_id
and client_secret
.bower install
from the examples/amd
directory in order to download dependencies for Aeris.js


You are currently a member of the following groups.  The button at the end of the row allows you to leave a group.


We have been live since October 17th, 2013. Our current block number is 37000.
I hope everyone who finds value in Boinc will come on board and join us to make Gridcoin the best cryptocurrency.


--- layout: page ---
Hi, I'm Eduardo Nava. I'm a computer science undergrad at Georgia Institute of Technology. I like to tinker with new things and write code. Feel free to explore some of my posts about the things I've done. They're pretty cool, and you might learn something new.
socials:
LinkedIn
· GitHub
· Twitter
email:
navaem (at) gmail dot com


Tooling around with CloudFormation helpers
Pretty much everybody can agree that working with CloudFormation is a bit, shall we say, cumbersome. Dare I attempt to recite its shortcomings? No, I dare not. Okay, maybe just a few:
- Until relatively recently, everything had to be done in JSON, which is terrible and allows no comments. Then AWS added YAML, which is better, but still a bit unwieldy.
- Working with multiple stacks with dependencies is downright painful. There is support for cross-stack references, but that’s also quite painful.

- The workflow of creating a template, validating the syntax, submitting it to AWS, waiting for something to fail, rinse & repeat is horrendously time consuming.
All bets are off when you’re trying to manage dozens of CF templates.
 It’s so much better! It will solve all your problems!”
I have the utmost respect for Hashicorp and their tooling. They’ve improved the situation for sys admins & devs & DevAdmins and all the other cloud people more than I can ever hope to. I <3 most of their tools. I went to a talk by Mitchell H just the other week at Nike. I learned stuff about graph theory and terraform.
I have tried to like terraform. I tried pretty hard about a year ago. It had some rough edges, and I understand that it has changed a lot since. But I still don’t trust it. At the time of writing there are 1,613 open issues and 254 pull requests. One of the issues is mine. It has been open since Feb. 4 2016. It’s a pretty serious bug that causes EBS volumes to be unexpectedly deleted.  Beyond labels, nobody on the Terraform team has even acknowledged it. One of the biggest benefits is the multi-provider support, which I don’t need. I even checked it out again for this project and found that it doesn’t have VPC IPv6 support yet. And dammit, CloudFormation just works for me, so plz stop yelling at me about it.
Fixing CloudFormation
I have some ideas to make CF a bit easier to work with, but I figured that before I built Yet Another Tool I should take a quick peek at what’s out there already. Some searching revealed a few promising candidates:
- StackMaster
- sceptre from CloudReach
- Stacker from Remind
- qaz
At first glance, qaz was really appealing because it’s written in Go and attempts only a minimal abstraction over CloudFormation. But it’s the brand new kid on the block and I need to use this sucker to manage production infrastructure, so I didn’t kick the tires on it too much. It’s definitely promising.
StackMaster supports diffs, change sets, and, enticingly, GPG parameter encryption. I like that stuff. But it’s written in Ruby, and I get a higher dose of Ruby than I’d like with Chef. I tend to tune out when I see terms like SparkleFormation. I didn’t necessarily want to learn another DSL, and the Ruby tooling (RVM, gem) makes me want to break keyboards. But for people that are better at Ruby than I am, this is probably a good choice.
Sceptre is on the minimalist end. You still need to write your own CF templates (in JSON or YAML). It provides simple wrappers for CRUD operations and presents a reasonable CLI. I want something with a bit more programmatic control and more features.
Enter Stacker, the tools that fits the bill for me. Python with Troposphere-based “blueprints” provide the full power of Python. A very intuitive, simple CLI with most of the features of StackMaster but without the Ruby. Plus it’s from the team at Remind, and I’ve followed their work for some time now. (Check out Empire, you might like it.)
The rest of this post is about my experience getting started with Stacker.
First steps with Stacker
Stacker has some very nice docs and examples but nothing that serves as an introductory tutorial. The following is more or less my raw notes on getting using Stacker for the first time.
Obvious first step: install stacker and setup a work space
Now it’s time to dive in to the deep end. I need to implement my reference vpc architecture with just a few changes. Since that post, AWS released NAT gateways (beautiful, I’d been waiting for those suckers for years), and IPv6 support (even more beautiful because I want the Internet to continue to scale). I work best from examples, and I’m lazy, so I’m going to start with Remind’s VPC blueprint.
To achieve my reference VPC I’ll need 3 environments: Management, NonProd, and Prod. Stacker has support for environments, so this starts with simply touch config/environments/{management.env,nonprod.env,prod.env}
.
Perfect!
Time for some customizations, starting with config.
Configuring Stacker
The config from example.yaml
is a great place to start but I needed some edits.
I have this custom blueprints/
location for my stacks so I need to let Stacker know about that. I added
to the top of config.yaml
. This means I can only execute stacker
commands from that stacker/
directory I created above, a
limitation that I’m fine with.
I also wanted to customize the S3 bucket where the templates are uploaded. I added
after sys_path
in config.yaml
.
Finally time to customize the CF stacks that Stacker will manage. Without getting to deep in the woods, I ended up with a VPC stack that looks like this:
This differs from the example config in a few important ways:
- Stacker will look for the VPC class in
blueprints/vpc.py
(theclass_path
setting) - I’ve set several of the parameters (
${SshKeyName}
,${PublicSubnets}
,${PrivateSubnets}
, etc) to be variables that are read from environment files. This let’s me customize on a per-environment basis.
Setting up the environments takes no time at all. Each environment (not yaml, but it looks like it) file looks like this:
The namespace
sets up a naming convention to disambiguate stacks. Set up one namespace for each environment and customize accordingly.
Customizing the VPC blueprint
Stacker blueprints are python code using the excellent troposphere library which I’ve used quite a lot in the past. I had to make a few minor edits to the default VPC class for my needs. I won’t go in to all the detail about those edits because customizing the blueprints is straightforward if you understand troposphere. If you don’t, go learn it, you’ll never go back to manually editing templates.
I did need to drop in to the support Slack room to ask a quick question. The default VPC blueprint does not name the resources, and I wanted to names so that things are identifiable in the AWS console. For example, the management VPC needed
the Name: management
tag. @phobologic (Mike Barrett) was super helpful and gave me the answer right away. To refer to a namespace from within the blueprint,
use self.context.get_fqn(self.name)
. I modified the resource tags to include namespaces in the names like this:
Building the stack
The fun part arrives: running the tool. Stacker makes it really easy to launch stacks with support for diff’ing the current state of a stack with the new state, tailing the cloudformation logs as the stack is being updated/created, and, my personal favorite, change sets. I used this command to create the management stack:
Worked on the first try.
Summing it up
Stacker is a huge improvement over working with raw CloudFormation. I’m working with a new client to build an ECS-based infrastructure, and I’m pretty excited to implement it all without typing any JSON and a minimal amount of YAML. I intend to integrate the Stacker workflowy in a Jenkins pipeline just as we do for the application code. The Stacker team is preparing the 1.0 release which will come with a bunch of improvements and fixes. Can’t wait to see what they build next.