The p5.js canvas works like a piece of graph paper. When you want to draw something, you specify its coordinates on the graph. For example, we can draw a 100x100 pixel rectangle at the point (60, 60).
function setup() {
createCanvas(400, 400);
fill(0);
}
function draw() {
background(240);
drawGrid();
fill(0);
rect(60, 60, 100, 100);
}
function drawGrid() {
stroke(200);
fill(120);
for (var x=-width; x < width; x+=40) {
line(x, -height, x, height);
text(x, x+1, 12);
}
for (var y=-height; y < height; y+=40) {
line(-width, y, width, y);
text(y, 1, y+12);
}
}
If you want to move your rectangle 120 pixels to the right and 120 pixels down, you can add that to the x and y-coordinates of the rectangle and run rect(180, 180, 100, 100). But there is another way to do this: move the graph paper itself. If you move the graph paper 120 pixels right and 120 pixels down, you will get exactly the same visual result. Moving the coordinate system is called translation, and can be done using the translate() function.
function setup() {
createCanvas(400, 400);
fill(0);
}
function draw() {
background(240);
translate(120, 120);
drawGrid();
fill(0);
rect(60, 60, 100, 100);
}
function drawGrid() {
stroke(200);
fill(120);
for (var x=-width; x < width; x+=40) {
line(x, -height, x, height);
text(x, x+1, 12);
}
for (var y=-height; y < height; y+=40) {
line(-width, y, width, y);
text(y, 1, y+12);
}
}
As far as the rectangle is concerned, it hasn't moved at all. Its upper left corner is still at (60, 60). When you use transformations, the things you draw never change position; the coordinate system itself does. We will see how this can be useful in the next section.
Sometimes, a sketch will involve multiple translations to draw the same thing in different locations. If we want to draw the same rectangle in three separate locations -- say at (50, 50), (250, 150), and (140, 280) -- we can use push() and pop() to keep track of and revert our translations as we go. The function pop() undoes all translations made following the previous push(). push() and pop() also track and revert style and color changes -- fill(), stroke(), etc -- in the same way. Thus all translations and style/color commands made between a single push() and pop() apply only to the operations made between them. Take a look at the following example.
function setup() {
createCanvas(400, 400);
}
function draw() {
background(240);
push();
translate(50, 50);
fill(255, 0, 0);
rect(0, 0, 100, 100);
pop();
push();
translate(250, 150);
fill(0, 255, 0);
rect(0, 0, 100, 100);
pop();
push();
translate(140, 280);
fill(0, 0, 255);
rect(0, 0, 100, 100);
pop();
}
The first triangle in red is drawn at (0, 0) after the coordinate system has been translated to the right and down by 50 pixels each. After the first pop() is called, the first translation is undone and the coordinate (0, 0) reverts back to the top-left corner of the canvas. We call push() and translate(250, 150), now moving the the canvas 250 pixels right and 150 pixels done, and then draw the green rectangle. The second pop() undoes that translation as well. Finally, the blue triangle is done after making the same operation, this time translating 140 pixels right and 280 pixels down.
In the last sketch, we could have achieved the same visual output by simply drawing the rectangles at those exact points. Or we could have used translate() without the push() and pop() blocks by undoing each translation manually, e.g. translate(-50, -50) to undo the first translation of (50, 50).
For simple shapes like rectangles, it is easier to not use translations, but rather to specify the points manually. But when we begin to draw more complex shapes, translations become advantageous. Let's take a look at an example where translate(), push(), and pop() are really useful.
function setup(){
createCanvas(400, 400);
}
function draw(){
background(240);
for (var x = 5; x < 400; x = x+50){
for (var y = 5; y < 400; y = y+50){
push();
translate(x, y);
drawHouse();
pop();
}
}
}
function drawHouse() {
triangle(15, 0, 0, 15, 30, 15);
rect(0, 15, 30, 25);
rect(12, 30, 10, 10);
}
In this example, we draw a grid of houses. Instead of having to keep track of the coordinates for every vertex of every house, we simply have one function, drawHouse() which draws a house relative to the point (0, 0). We then create a loop which will translate to the correct point before drawing each house.
Using the transformations command allows us to use a single drawing function irrespective of where we intend to draw it on the canvas, while push() and pop() allow us to reserve transformations only to the relevant code blocks.
In addition to translating the grid, you can also rotate it with the rotate() function. This function takes one argument, the angle of rotation. It then rotates the canvas around the origin point (0, 0). It's as though you have a piece of graph paper, place your finger on the point (0, 0), and then rotate the paper around your finger.
Angle is measured clockwise with zero being at 3 o'clock. In p5.js, all the functions having to do with angles take radians. A single rotation or a full circle has 360° or 2π radians. Here is a diagram of how p5.js measures angles in degrees (black) and radians (red).
Since most people think in degrees, p5.js has a built-in radians() function which takes a number of degrees as its argument and converts it to radians for you. It also has a degrees() function that converts radians to degrees. Take a look at the following example which rotates the canvas by an angle measurement set by the mouseX position. The red square is drawn before rotation, and the green one after rotation.
function setup(){
createCanvas(400, 400);
}
function draw() {
var deg = mouseX;
var rad = radians(deg);
background(240);
// the red rectangle is drawn before the rotation so
// it will stay in place
fill(255, 0, 0);
rect(200, 200, 100, 100);
line(0, 0, 200, 200);
fill(0, 255, 0);
text("rotate "+floor(deg)+" degrees", 200, 50);
// rotation is done here. all subsequent drawing
// is done post-rotation
rotate(rad);
// draw the grid
drawGrid();
// the green rectangle and grid is drawn after rotating the canvas
rect(200, 200, 100, 100);
line(0, 0, 200, 200);
}
function drawGrid() {
stroke(200);
fill(120);
for (var x=-2*width; x < 2*width; x+=40) {
line(x, -2*height, x, 2*height);
text(x, x+1, 12);
}
for (var y=-2*height; y < 2*height; y+=40) {
line(-2*width, y, 2*width, y);
text(y, 1, y+12);
}
}
In the last example, the green square goes off the screen as we rotate the canvas more. Sometimes we want to be able to rotate a shape in place, rather than around the top-left corner of the screen. We can do this by combining translate() and rotate() together. Recall that translate() moves the origin (0,0) to another place in the canvas and that rotate() rotates the canvas around the origin. Thus if we want to draw a shape in the middle of the screen and then rotate it in place, we should first translate the canvas to the point we want to draw the shape, then rotate, then draw the shape at (0, 0). Take a look at the following example.
function setup(){
createCanvas(400, 400);
}
function draw(){
background(240);
// move the origin to the pivot point
translate(150, 150);
// then rotate the grid around the pivot point by a
// number of degrees equal to the frame count of the sketch
rotate(radians(frameCount));
// and draw the square at the origin
fill(0);
rect(0, 0, 100, 100);
}
If you want the rectangle to rotate around its own center point rather than by its corner, we can use the command rectMode(CENTER) to set p5.js to interpret the coordinates of rectangles to refer to the center of the rectangle rather than the top-left corner. The following example is the same as the previous one but with centered coordinates.
function setup(){
createCanvas(400, 400);
rectMode(CENTER); // now the first two arguments of a rect are its center point, not corner
}
function draw(){
background(240);
// move the origin to the pivot point
translate(150, 150);
// then rotate the grid around the pivot point by a
// number of degrees equal to the frame count of the sketch
rotate(radians(frameCount));
// and draw the square at the origin
fill(0);
rect(0, 0, 100, 100);
}
Here is a program that generates a wheel of colors by using rotation, and draws a strip every 25 frames.
function setup() {
createCanvas(400, 400);
background(240);
rectMode(CENTER);
}
function draw(){
// the modulo operator % calculates the remainder.
// example: 24 % 25 = 24, 25 % 25 = 0, 26 % 25 = 1, etc.
// thus this if statement will evaluate True every 25 frames.
if (frameCount % 25 == 0) {
fill(random(255), random(255), random(255));
push();
translate(200, 200);
rotate(radians(frameCount));
rect(0, 0, 300, 20);
pop();
}
}
Let's take a look at a more complicated example where we can use translate and rotate together to create interesting symmetries around the center of the canvas. Let's first build it by drawing 8 squares which are evenly distributed along a circle around the center of the canvas. What we do is we first translate to the center of the screen, translating the canvas by (width/2, height/2). We then rotate the canvas to the 8 angles evenly distributed between 0 and 2π radians. We then draw a circle 120 pixels away from the center, after each of these rotations.
function setup() {
createCanvas(400, 400);
rectMode(CENTER);
}
function draw(){
background(240);
translate(width/2, height/2);
for (var i = 0; i < 8; i++) {
push();
rotate(TWO_PI * i / 8);
rect(120, 0, 20, 20);
pop();
}
}
Look at the line rect(120, 0, 20, 20). Recall that you can draw a shape in the same place by translating to it first and then drawing it at (0, 0). So we can do another another translation after we've rotated, to translate along the right-pointing line after it's been rotated. So we can replace the statement rect(120, 0, 20, 20) with the following two: translate(120, 0) and rect(0, 0, 20, 20). By doing so, we bring the pivot point or origin of the canvas to each of the eight rectangles we just drew.
This is convenient because now we can do something even more interesting. Instead of simply drawing a rectangle at each of those points, let's create another loop which will draw six smaller squares evenly distributed around a small circle around each of these new pivot points. See the following.
function setup() {
createCanvas(400, 400);
rectMode(CENTER);
}
function draw(){
background(240);
translate(width/2, height/2);
for (var i = 0; i < 8; i++) {
push();
rotate(TWO_PI * i / 8);
translate(120, 0);
rect(0, 0, 20, 20);
for (var j = 0; j < 6; j++) {
push();
rotate(TWO_PI * j / 6);
rect(25, 0, 8, 8);
pop();
}
translate()
pop();
}
}
Now that we have achieved an interesting geometry, we can embellish this program by making a variable for the first translation amount and for the x-position of the rectangle, and modulating them in interesting ways. In the following example, we'll modulate those two values using Perlin noise. If you haven't used Perlin noise before, take a look at the tutorial on Perlin noise.
function setup() {
createCanvas(400, 400);
rectMode(CENTER);
}
function draw(){
background(240);
translate(width/2, height/2);
for (var i = 0; i < 8; i++) {
push();
rotate(TWO_PI * i / 8);
var tx = 200 * noise(0.01*frameCount);
translate(tx, 0);
rect(0, 0, 20, 20);
for (var j = 0; j < 6; j++) {
push();
rotate(TWO_PI * j / 6);
var rx = 60 * noise(0.01*frameCount + 10);
rect(rx, 0, 8, 8);
pop();
}
translate()
pop();
}
}
Now we're getting somewhere! Let's take the previous example and make the following additions to it to make it even more interesting.
function setup() {
createCanvas(400, 400);
rectMode(CENTER);
fill(0, 50);
stroke(255, 50);
}
function draw(){
background(240);
var ang1 = TWO_PI * noise(0.01*frameCount + 10);
var ang2 = TWO_PI * noise(0.01*frameCount + 20);
var ang3 = TWO_PI * noise(0.01*frameCount + 30);
var rx = 60 * noise(0.01*frameCount + 40);
var tx = 200 * noise(0.01*frameCount + 50);
var size1 = 200 * noise(0.01*frameCount + 60);
var size2 = 50 * noise(0.01*frameCount + 60);
translate(width/2, height/2);
for (var i = 0; i < 8; i++) {
push();
rotate(ang1 + TWO_PI * i / 8);
translate(tx, 0);
rect(0, 0, size1, size1);
for (var j = 0; j < 6; j++) {
push();
rotate(ang2 + TWO_PI * j / 6);
translate(rx, 0);
rotate(ang3);
rect(rx, 0, size2, size2);
pop();
}
translate()
pop();
}
}
There is one more function which transforms the coordinate system, and that is scale(). Scale changes the size of the grid by multiplying the coordinate system. See the following example.
function setup() {
createCanvas(400, 400);
}
function draw(){
background(240);
var scaleAmount = mouseX / 200;
scale(scaleAmount);
drawGrid();
fill(255, 255, 0);
rect(20, 20, 50, 50);
}
function drawGrid() {
fill(0);
stroke(120);
for (var x=-width; x < width; x+=40) {
line(x, -height, x, height);
text(x, x+1, 12);
}
for (var y=-height; y < height; y+=40) {
line(-width, y, width, y);
text(y, 1, y+12);
}
}
The scaling function can be useful for drawing multiple copies of the same thing at different sizes. Let's bring back our house drawing example, and draw three houses of three separate sizes. We first translate to the position of each house and then use scale to multiply the size of the house in-place.
function setup() {
createCanvas(400, 400);
}
function draw(){
background(240);
push();
translate(50, 100);
drawHouse();
pop();
push();
translate(150, 100);
scale(2.0);
drawHouse();
pop();
push();
translate(250, 100);
scale(3.0);
drawHouse();
pop();
}
function drawHouse() {
triangle(15, 0, 0, 15, 30, 15);
rect(0, 15, 30, 25);
rect(12, 30, 10, 10);
}
The scaling function can take two arguments instead of one, allowing you to scale by a different amount in the x and y directions. See the following example.
Note that order makes a difference. Translating and then scaling is not necessarily going to give you the same result as scaling and then translating. What happens in the following example if you switch the order of the scale and translate? Try for yourself and see.
function setup() {
createCanvas(400, 400);
}
function draw(){
background(240);
translate(width/2, height/2);
scale(mouseX / 300, mouseY / 300);
fill(255, 0, 0);
triangle(0, -100, -100, 100, 100, 100);
}
Interesting visual results can be found by combining translate, rotate, and scale. See the following example. Try changing some of the numbers in the sketch and see how the result is affected.
function setup() {
createCanvas(400, 400);
}
function draw(){
background(240);
fill(0, 10);
stroke(0, 160);
translate(width/2, height/2);
for (var i=0; i < 30; i++) {
push();
rotate(i / 5.0);
scale(i / 8.0);
triangle(0, -100, -10, 100, 10, 100);
pop();
}
}
In this tutorial, we have covered transformations of the coordinate space including translate(), rotate(), scale(), push(), and pop(). You've seen how they can be helpful in keeping track of coordinates when running repetitive drawing routines. Additionally, you can see how just a few lines of code can generate interesting visual results when transformations are applied multiple times iteratively. Mastering these commands will help you greatly scale the complexity of your programs and produce structured and visual compelling sketches.


Normalizing Flows - Introduction (Part 1)¶
This tutorial introduces Pyro’s normalizing flow library. It is independent of much of Pyro, but users may want to read about distribution shapes in the Tensor Shapes Tutorial.
Introduction¶
In standard probabilistic modeling practice, we represent our beliefs over unknown continuous quantities with simple parametric distributions like the normal, exponential, and Laplacian distributions. However, using such simple forms, which are commonly symmetric and unimodal (or have a fixed number of modes when we take a mixture of them), restricts the performance and flexibility of our methods. For instance, standard variational inference in the Variational Autoencoder uses independent univariate normal distributions to represent the variational family. The true posterior is neither independent nor normally distributed, which results in suboptimal inference and simplifies the model that is learnt. In other scenarios, we are likewise restricted by not being able to model multimodal distributions and heavy or light tails.
Normalizing Flows [1-4] are a family of methods for constructing flexible learnable probability distributions, often with neural networks, which allow us to surpass the limitations of simple parametric forms. Pyro contains state-of-the-art normalizing flow implementations, and this tutorial explains how you can use this library for learning complex models and performing flexible variational inference. We introduce the main idea of Normalizing Flows (NFs) and demonstrate learning simple univariate distributions with element-wise, multivariate, and conditional flows.
Univariate Distributions¶
Background¶
Normalizing Flows are a family of methods for constructing flexible distributions. Let’s first restrict our attention to representing univariate distributions. The basic idea is that a simple source of noise, for example a variable with a standard normal distribution, \(X\sim\mathcal{N}(0,1)\), is passed through a bijective (i.e. invertible) function, \(g(\cdot)\) to produce a more complex transformed variable \(Y=g(X)\).
For a given random variable, we typically want to perform two operations: sampling and scoring. Sampling \(Y\) is trivial. First, we sample \(X=x\), then calculate \(y=g(x)\). Scoring \(Y\), or rather, evaluating the log-density \(\log(p_Y(y))\), is more involved. How does the density of \(Y\) relate to the density of \(X\)? We can use the substitution rule of integral calculus to answer this. Suppose we want to evaluate the expectation of some function of \(X\). Then,
\begin{align} \mathbb{E}_{p_X(\cdot)}\left[f(X)\right] &= \int_{\text{supp}(X)}f(x)p_X(x)dx\\ &= \int_{\text{supp}(Y)}f(g^{-1}(y))p_X(g^{-1}(y))\left|\frac{dx}{dy}\right|dy\\ &= \mathbb{E}_{p_Y(\cdot)}\left[f(g^{-1}(Y))\right], \end{align}
where \(\text{supp}(X)\) denotes the support of \(X\), which in this case is \((-\infty,\infty)\). Crucially, we used the fact that \(g\) is bijective to apply the substitution rule in going from the first to the second line. Equating the last two lines we get,
\begin{align} \log(p_Y(y)) &= \log(p_X(g^{-1}(y)))+\log\left(\left|\frac{dx}{dy}\right|\right)\\ &= \log(p_X(g^{-1}(y)))-\log\left(\left|\frac{dy}{dx}\right|\right). \end{align}
Inituitively, this equation says that the density of \(Y\) is equal to the density at the corresponding point in \(X\) plus a term that corrects for the warp in volume around an infinitesimally small length around \(Y\) caused by the transformation.
If \(g\) is cleverly constructed (and we will see several examples shortly), we can produce distributions that are more complex than standard normal noise and yet have easy sampling and computationally tractable scoring. Moreover, we can compose such bijective transformations to produce even more complex distributions. By an inductive argument, if we have \(L\) transforms \(g_{(0)}, g_{(1)},\ldots,g_{(L-1)}\), then the log-density of the transformed variable \(Y=(g_{(0)}\circ g_{(1)}\circ\cdots\circ g_{(L-1)})(X)\) is
\begin{align} \log(p_Y(y)) &= \log\left(p_X\left(\left(g_{(L-1)}^{-1}\circ\cdots\circ g_{(0)}^{-1}\right)\left(y\right)\right)\right)+\sum^{L-1}_{l=0}\log\left(\left|\frac{dg^{-1}_{(l)}(y_{(l)})}{dy'}\right|\right), %\left( g^{(l)}(y^{(l)}) %\right). \end{align}
where we’ve defined \(y_{(0)}=x\), \(y_{(L-1)}=y\) for convenience of notation.
In a latter section, we will see how to generalize this method to multivariate \(X\). The field of Normalizing Flows aims to construct such \(g\) for multivariate \(X\) to transform simple i.i.d. standard normal noise into complex, learnable, high-dimensional distributions. The methods have been applied to such diverse applications as image modeling, text-to-speech, unsupervised language induction, data compression, and modeling molecular structures. As probability distributions are the most fundamental component of probabilistic modeling we will likely see many more exciting state-of-the-art applications in the near future.
Fixed Univariate Transforms in Pyro¶
PyTorch contains classes for representing fixed univariate bijective transformations, and sampling/scoring from transformed distributions derived from these. Pyro extends this with a comprehensive library of learnable univariate and multivariate transformations using the latest developments in the field. As Pyro imports all of PyTorch’s distributions and transformations, we will work solely with Pyro. We also note that the NF components in Pyro can be used independently of the probabilistic programming functionality of Pyro, which is what we will be doing in the first two tutorials.
Let us begin by showing how to represent and manipulate a simple transformed distribution,
\begin{align} X &\sim \mathcal{N}(0,1)\\ Y &= \text{exp}(X). \end{align}
You may have recognized that this is by definition, \(Y\sim\text{LogNormal}(0,1)\).
We begin by importing the relevant libraries:
[1]:
import torch
import pyro
import pyro.distributions as dist
import pyro.distributions.transforms as T
import matplotlib.pyplot as plt
import seaborn as sns
import os
smoke_test = ('CI' in os.environ)
A variety of bijective transformations live in the pyro.distributions.transforms module, and the classes to define transformed distributions live in pyro.distributions. We first create the base distribution of \(X\) and the class encapsulating the transform \(\text{exp}(\cdot)\):
[2]:
dist_x = dist.Normal(torch.zeros(1), torch.ones(1))
exp_transform = T.ExpTransform()
The class ExpTransform derives from Transform and defines the forward, inverse, and log-absolute-derivative operations for this transform,
\begin{align} g(x) &= \text{exp(x)}\\ g^{-1}(y) &= \log(y)\\ \log\left(\left|\frac{dg}{dx}\right|\right) &= x. \end{align}
In general, a transform class defines these three operations, from which it is sufficient to perform sampling and scoring.
The class TransformedDistribution takes a base distribution of simple noise and a list of transforms, and encapsulates the distribution formed by applying these transformations in sequence. We use it as:
[3]:
dist_y = dist.TransformedDistribution(dist_x, [exp_transform])
Now, plotting samples from both to verify that we that have produced the log-normal distribution:
[4]:
plt.subplot(1, 2, 1)
plt.hist(dist_x.sample([1000]).numpy(), bins=50)
plt.title('Standard Normal')
plt.subplot(1, 2, 2)
plt.hist(dist_y.sample([1000]).numpy(), bins=50)
plt.title('Standard Log-Normal')
plt.show()
Our example uses a single transform. However, we can compose transforms to produce more expressive distributions. For instance, if we apply an affine transformation we can produce the general log-normal distribution,
\begin{align} X &\sim \mathcal{N}(0,1)\\ Y &= \text{exp}(\mu+\sigma X). \end{align}
or rather, \(Y\sim\text{LogNormal}(\mu,\sigma^2)\). In Pyro this is accomplished, e.g. for \(\mu=3, \sigma=0.5\), as follows:
[5]:
dist_x = dist.Normal(torch.zeros(1), torch.ones(1))
affine_transform = T.AffineTransform(loc=3, scale=0.5)
exp_transform = T.ExpTransform()
dist_y = dist.TransformedDistribution(dist_x, [affine_transform, exp_transform])
plt.subplot(1, 2, 1)
plt.hist(dist_x.sample([1000]).numpy(), bins=50)
plt.title('Standard Normal')
plt.subplot(1, 2, 2)
plt.hist(dist_y.sample([1000]).numpy(), bins=50)
plt.title('Log-Normal')
plt.show()
For the forward operation, transformations are applied in the order of the list that is the second argument to TransformedDistribution. In this case, first AffineTransform is applied to the base distribution and then ExpTransform.
Learnable Univariate Distributions in Pyro¶
Having introduced the interface for invertible transforms and transformed distributions, we now show how to represent learnable transforms and use them for density estimation. Our dataset in this section and the next will comprise samples along two concentric circles. Examining the joint and marginal distributions:
[6]:
import numpy as np
from sklearn import datasets
from sklearn.preprocessing import StandardScaler
n_samples = 1000
X, y = datasets.make_circles(n_samples=n_samples, factor=0.5, noise=0.05)
X = StandardScaler().fit_transform(X)
plt.title(r'Samples from $p(x_1,x_2)$')
plt.xlabel(r'$x_1$')
plt.ylabel(r'$x_2$')
plt.scatter(X[:,0], X[:,1], alpha=0.5)
plt.show()
plt.subplot(1, 2, 1)
sns.distplot(X[:,0], hist=False, kde=True,
bins=None,
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 2})
plt.title(r'$p(x_1)$')
plt.subplot(1, 2, 2)
sns.distplot(X[:,1], hist=False, kde=True,
bins=None,
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 2})
plt.title(r'$p(x_2)$')
plt.show()
Standard transforms derive from the Transform class and are not designed to contain learnable parameters. Learnable transforms, on the other hand, derive from TransformModule, which is a torch.nn.Module and registers parameters with the object.
We will learn the marginals of the above distribution using such a transform, Spline [5,6], defined on a two-dimensional input:
[7]:
base_dist = dist.Normal(torch.zeros(2), torch.ones(2))
spline_transform = T.Spline(2, count_bins=16)
flow_dist = dist.TransformedDistribution(base_dist, [spline_transform])
This transform passes each dimension of its input through a separate monotonically increasing function known as a spline. From a high-level, a spline is a complex parametrizable curve for which we can define specific points known as knots that it passes through and the derivatives at the knots. The knots and their derivatives are parameters that can be learnt, e.g., through stochastic gradient descent on a maximum likelihood objective, as we now demonstrate:
[8]:
%%time
steps = 1 if smoke_test else 1001
dataset = torch.tensor(X, dtype=torch.float)
optimizer = torch.optim.Adam(spline_transform.parameters(), lr=1e-2)
for step in range(steps):
optimizer.zero_grad()
loss = -flow_dist.log_prob(dataset).mean()
loss.backward()
optimizer.step()
flow_dist.clear_cache()
if step % 200 == 0:
print('step: {}, loss: {}'.format(step, loss.item()))
step: 0, loss: 2.682476758956909
step: 200, loss: 1.278384804725647
step: 400, loss: 1.2647961378097534
step: 600, loss: 1.2601449489593506
step: 800, loss: 1.2561875581741333
step: 1000, loss: 1.2545257806777954
CPU times: user 4.92 s, sys: 69.3 ms, total: 4.99 s
Wall time: 5.01 s
Note that we call flow_dist.clear_cache()
after each optimization step to clear the transform’s forward-inverse cache. This is required because flow_dist
’s spline_transform
is a stateful TransformModule rather than a purely stateless Transform object. Purely functional Pyro code typically creates
Transform
objects each model execution, then discards them after .backward()
, effectively clearing the transform caches. By contrast in this tutorial we create stateful module objects and need to manually clear their cache after update.
Plotting samples drawn from the transformed distribution after learning:
[9]:
X_flow = flow_dist.sample(torch.Size([1000,])).detach().numpy()
plt.title(r'Joint Distribution')
plt.xlabel(r'$x_1$')
plt.ylabel(r'$x_2$')
plt.scatter(X[:,0], X[:,1], label='data', alpha=0.5)
plt.scatter(X_flow[:,0], X_flow[:,1], color='firebrick', label='flow', alpha=0.5)
plt.legend()
plt.show()
plt.subplot(1, 2, 1)
sns.distplot(X[:,0], hist=False, kde=True,
bins=None,
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 2},
label='data')
sns.distplot(X_flow[:,0], hist=False, kde=True,
bins=None, color='firebrick',
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 2},
label='flow')
plt.title(r'$p(x_1)$')
plt.subplot(1, 2, 2)
sns.distplot(X[:,1], hist=False, kde=True,
bins=None,
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 2},
label='data')
sns.distplot(X_flow[:,1], hist=False, kde=True,
bins=None, color='firebrick',
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 2},
label='flow')
plt.title(r'$p(x_2)$')
plt.show()
As we can see, we have learnt close approximations to the marginal distributions, \(p(x_1),p(x_2)\). It would have been challenging to fit the irregularly shaped marginals with standard methods, e.g., a mixture of normal distributions. As expected, since there is a dependency between the two dimensions, we do not learn a good representation of the joint, \(p(x_1,x_2)\). In the next section, we explain how to learn multivariate distributions whose dimensions are not independent.
Multivariate Distributions¶
Background¶
The fundamental idea of normalizing flows also applies to multivariate random variables, and this is where its value is clearly seen - representing complex high-dimensional distributions. In this case, a simple multivariate source of noise, for example a standard i.i.d. normal distribution, \(X\sim\mathcal{N}(\mathbf{0},I_{D\times D})\), is passed through a vector-valued bijection, \(g:\mathbb{R}^D\rightarrow\mathbb{R}^D\), to produce the more complex transformed variable \(Y=g(X)\).
Sampling \(Y\) is again trivial and involves evaluation of the forward pass of \(g\). We can score \(Y\) using the multivariate substitution rule of integral calculus,
\begin{align} \mathbb{E}_{p_X(\cdot)}\left[f(X)\right] &= \int_{\text{supp}(X)}f(\mathbf{x})p_X(\mathbf{x})d\mathbf{x}\\ &= \int_{\text{supp}(Y)}f(g^{-1}(\mathbf{y}))p_X(g^{-1}(\mathbf{y}))\det\left|\frac{d\mathbf{x}}{d\mathbf{y}}\right|d\mathbf{y}\\ &= \mathbb{E}_{p_Y(\cdot)}\left[f(g^{-1}(Y))\right], \end{align}
where \(d\mathbf{x}/d\mathbf{y}\) denotes the Jacobian matrix of \(g^{-1}(\mathbf{y})\). Equating the last two lines we get,
\begin{align} \log(p_Y(y)) &= \log(p_X(g^{-1}(y)))+\log\left(\det\left|\frac{d\mathbf{x}}{d\mathbf{y}}\right|\right)\\ &= \log(p_X(g^{-1}(y)))-\log\left(\det\left|\frac{d\mathbf{y}}{d\mathbf{x}}\right|\right). \end{align}
Inituitively, this equation says that the density of \(Y\) is equal to the density at the corresponding point in \(X\) plus a term that corrects for the warp in volume around an infinitesimally small volume around \(Y\) caused by the transformation. For instance, in \(2\)-dimensions, the geometric interpretation of the absolute value of the determinant of a Jacobian is that it represents the area of a parallelogram with edges defined by the columns of the Jacobian. In \(n\)-dimensions, the geometric interpretation of the absolute value of the determinant Jacobian is that is represents the hyper-volume of a parallelepiped with \(n\) edges defined by the columns of the Jacobian (see a calculus reference such as [7] for more details).
Similar to the univariate case, we can compose such bijective transformations to produce even more complex distributions. By an inductive argument, if we have \(L\) transforms \(g_{(0)}, g_{(1)},\ldots,g_{(L-1)}\), then the log-density of the transformed variable \(Y=(g_{(0)}\circ g_{(1)}\circ\cdots\circ g_{(L-1)})(X)\) is
\begin{align} \log(p_Y(y)) &= \log\left(p_X\left(\left(g_{(L-1)}^{-1}\circ\cdots\circ g_{(0)}^{-1}\right)\left(y\right)\right)\right)+\sum^{L-1}_{l=0}\log\left(\left|\frac{dg^{-1}_{(l)}(y_{(l)})}{dy'}\right|\right), %\left( g^{(l)}(y^{(l)}) %\right). \end{align}
where we’ve defined \(y_{(0)}=x\), \(y_{(L-1)}=y\) for convenience of notation.
The main challenge is in designing parametrizable multivariate bijections that have closed form expressions for both \(g\) and \(g^{-1}\), a tractable Jacobian whose calculation scales with \(O(D)\) rather than \(O(D^3)\), and can express a flexible class of functions.
Multivariate Transforms in Pyro¶
Up to this point we have used element-wise transforms in Pyro. These are indicated by having the property transform.event_dim == 0
set on the transform object. Such element-wise transforms can only be used to represent univariate distributions and multivariate distributions whose dimensions are independent (known in variational inference as the mean-field approximation).
The power of Normalizing Flow, however, is most apparent in their ability to model complex high-dimensional distributions with neural networks and Pyro contains several such flows for accomplishing this. Transforms that operate on vectors have the property transform.event_dim == 1
, transforms on matrices with transform.event_dim == 2
, and so on. In general, the event_dim
property of a transform indicates how many dependent dimensions there are in the output of a transform.
In this section, we show how to use SplineCoupling to learn the bivariate toy distribution from our running example. A coupling transform [8, 9] divides the input variable into two parts and applies an element-wise bijection to the section half whose parameters are a function of the first. Optionally, an element-wise bijection is also applied to the first half. Dividing the inputs at \(d\), the transform is,
\begin{align} \mathbf{y}_{1:d} &= g_\theta(\mathbf{x}_{1:d})\\ \mathbf{y}_{(d+1):D} &= h_\phi(\mathbf{x}_{(d+1):D};\mathbf{x}_{1:d}), \end{align}
where \(\mathbf{x}_{1:d}\) represents the first \(d\) elements of the inputs, \(g_\theta\) is either the identity function or an elementwise bijection parameters \(\theta\), and \(h_\phi\) is an element-wise bijection whose parameters are a function of \(\mathbf{x}_{1:d}\).
This type of transform is easily invertible. We invert the first half, \(\mathbf{y}_{1:d}\), then use the resulting \(\mathbf{x}_{1:d}\) to evaluate \(\phi\) and invert the second half,
\begin{align} \mathbf{x}_{1:d} &= g_\theta^{-1}(\mathbf{y}_{1:d})\\ \mathbf{x}_{(d+1):D} &= h_\phi^{-1}(\mathbf{y}_{(d+1):D};\mathbf{x}_{1:d}). \end{align}
Difference choices for \(g\) and \(h\) form different types of coupling transforms. When both are monotonic rational splines, the transform is the spline coupling layer of Neural Spline Flow [5,6], which is represented in Pyro by the SplineCoupling class. As shown in the references, when we combine a sequence of coupling layers sandwiched between random permutations so we introduce dependencies between all dimensions, we can model complex multivariate distributions.
Most of the learnable transforms in Pyro have a corresponding helper function that takes care of constructing a neural network for the transform with the correct output shape. This neural network outputs the parameters of the transform and is known as a hypernetwork [10]. The helper functions are represented by lower-case versions of the corresponding class name, and usually input at the very least the input-dimension or shape of the distribution to model. For instance, the helper function corresponding to SplineCoupling is spline_coupling. We create a bivariate flow with a single spline coupling layer as follows:
[10]:
base_dist = dist.Normal(torch.zeros(2), torch.ones(2))
spline_transform = T.spline_coupling(2, count_bins=16)
flow_dist = dist.TransformedDistribution(base_dist, [spline_transform])
Similarly to before, we train this distribution on the toy dataset and plot the results:
[11]:
%%time
steps = 1 if smoke_test else 5001
dataset = torch.tensor(X, dtype=torch.float)
optimizer = torch.optim.Adam(spline_transform.parameters(), lr=5e-3)
for step in range(steps+1):
optimizer.zero_grad()
loss = -flow_dist.log_prob(dataset).mean()
loss.backward()
optimizer.step()
flow_dist.clear_cache()
if step % 500 == 0:
print('step: {}, loss: {}'.format(step, loss.item()))
step: 0, loss: 8.446191787719727
step: 500, loss: 2.0197808742523193
step: 1000, loss: 1.794958472251892
step: 1500, loss: 1.73616361618042
step: 2000, loss: 1.7254879474639893
step: 2500, loss: 1.691617488861084
step: 3000, loss: 1.679549217224121
step: 3500, loss: 1.6967085599899292
step: 4000, loss: 1.6723777055740356
step: 4500, loss: 1.6505967378616333
step: 5000, loss: 1.8024061918258667
CPU times: user 10min 41s, sys: 14.7 s, total: 10min 56s
Wall time: 1min 39s
[12]:
X_flow = flow_dist.sample(torch.Size([1000,])).detach().numpy()
plt.title(r'Joint Distribution')
plt.xlabel(r'$x_1$')
plt.ylabel(r'$x_2$')
plt.scatter(X[:,0], X[:,1], label='data', alpha=0.5)
plt.scatter(X_flow[:,0], X_flow[:,1], color='firebrick', label='flow', alpha=0.5)
plt.legend()
plt.show()
plt.subplot(1, 2, 1)
sns.distplot(X[:,0], hist=False, kde=True,
bins=None,
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 2},
label='data')
sns.distplot(X_flow[:,0], hist=False, kde=True,
bins=None, color='firebrick',
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 2},
label='flow')
plt.title(r'$p(x_1)$')
plt.subplot(1, 2, 2)
sns.distplot(X[:,1], hist=False, kde=True,
bins=None,
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 2},
label='data')
sns.distplot(X_flow[:,1], hist=False, kde=True,
bins=None, color='firebrick',
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 2},
label='flow')
plt.title(r'$p(x_2)$')
plt.show()
We see from the output that this normalizing flow has successfully learnt both the univariate marginals and the bivariate distribution.
Conditional versus Joint Distributions¶
Background¶
In many cases, we wish to represent conditional rather than joint distributions. For instance, in performing variational inference, the variational family is a class of conditional distributions,
where \(\mathbf{z}\) is the latent variable and \(\mathbf{x}\) the observed one, that hopefully contains a member close to the true posterior of the model, \(p(\mathbf{z}\mid\mathbf{x})\). In other cases, we may wish to learn to generate an object \(\mathbf{x}\) conditioned on some context \(\mathbf{c}\) using \(p_\theta(\mathbf{x}\mid\mathbf{c})\) and observations \(\{(\mathbf{x}_n,\mathbf{c}_n)\}^N_{n=1}\). For instance, \(\mathbf{x}\) may be a spoken sentence and \(\mathbf{c}\) a number of speech features.
The theory of Normalizing Flows is easily generalized to conditional distributions. We denote the variable to condition on by \(C=\mathbf{c}\in\mathbb{R}^M\). A simple multivariate source of noise, for example a standard i.i.d. normal distribution, \(X\sim\mathcal{N}(\mathbf{0},I_{D\times D})\), is passed through a vector-valued bijection that also conditions on C, \(g:\mathbb{R}^D\times\mathbb{R}^M\rightarrow\mathbb{R}^D\), to produce the more complex transformed variable \(Y=g(X;C=\mathbf{c})\). In practice, this is usually accomplished by making the parameters for a known normalizing flow bijection \(g\) the output of a hypernet neural network that inputs \(\mathbf{c}\).
Sampling of conditional transforms simply involves evaluating \(Y=g(X; C=\mathbf{c})\). Conditioning the bijections on \(\mathbf{c}\), the same formula holds for scoring as for the joint multivariate case.
Conditional Transforms in Pyro¶
In Pyro, most learnable transforms have a corresponding conditional version that derives from ConditionalTransformModule. For instance, the conditional version of the spline transform is ConditionalSpline with helper function conditional_spline.
In this section, we will show how we can learn our toy dataset as the decomposition of the product of a conditional and a univariate distribution,
First, we create the univariate distribution for \(x_1\) as shown previously,
[13]:
dist_base = dist.Normal(torch.zeros(1), torch.ones(1))
x1_transform = T.spline(1)
dist_x1 = dist.TransformedDistribution(dist_base, [x1_transform])
A conditional transformed distribution is created by passing the base distribution and list of conditional and non-conditional transforms to the ConditionalTransformedDistribution class:
[14]:
x2_transform = T.conditional_spline(1, context_dim=1)
dist_x2_given_x1 = dist.ConditionalTransformedDistribution(dist_base, [x2_transform])
You will notice that we pass the dimension of the context variable, \(M=1\), to the conditional spline helper function.
Until we condition on a value of \(x_1\), the ConditionalTransformedDistribution object is merely a placeholder and cannot be used for sampling or scoring. By calling its .condition(context) method, we obtain a
TransformedDistribution for which all its conditional transforms have been conditioned on context
.
For example, to draw a sample from \(x_2\mid x_1=1\):
[15]:
x1 = torch.ones(1)
print(dist_x2_given_x1.condition(x1).sample())
tensor([-0.4529])
In general, the context variable may have batch dimensions and these dimensions must broadcast over the batch dimensions of the input variable.
Now, combining the two distributions and training it on the toy dataset:
[16]:
%%time
steps = 1 if smoke_test else 5001
modules = torch.nn.ModuleList([x1_transform, x2_transform])
optimizer = torch.optim.Adam(modules.parameters(), lr=3e-3)
x1 = dataset[:,0][:,None]
x2 = dataset[:,1][:,None]
for step in range(steps):
optimizer.zero_grad()
ln_p_x1 = dist_x1.log_prob(x1)
ln_p_x2_given_x1 = dist_x2_given_x1.condition(x1.detach()).log_prob(x2.detach())
loss = -(ln_p_x1 + ln_p_x2_given_x1).mean()
loss.backward()
optimizer.step()
dist_x1.clear_cache()
dist_x2_given_x1.clear_cache()
if step % 500 == 0:
print('step: {}, loss: {}'.format(step, loss.item()))
step: 0, loss: 5.663331031799316
step: 500, loss: 2.040316581726074
step: 1000, loss: 1.9603266716003418
step: 1500, loss: 1.8922736644744873
step: 2000, loss: 1.8509924411773682
step: 2500, loss: 1.8328033685684204
step: 3000, loss: 1.9166260957717896
step: 3500, loss: 1.877435326576233
step: 4000, loss: 1.8416743278503418
step: 4500, loss: 1.8379391431808472
step: 5000, loss: 1.824593424797058
CPU times: user 9min 24s, sys: 5.93 s, total: 9min 30s
Wall time: 1min 23s
[17]:
X = torch.cat((x1, x2), dim=-1)
x1_flow = dist_x1.sample(torch.Size([1000,]))
x2_flow = dist_x2_given_x1.condition(x1_flow).sample(torch.Size([1000,]))
X_flow = torch.cat((x1_flow, x2_flow), dim=-1)
plt.title(r'Joint Distribution')
plt.xlabel(r'$x_1$')
plt.ylabel(r'$x_2$')
plt.scatter(X[:,0], X[:,1], label='data', alpha=0.5)
plt.scatter(X_flow[:,0], X_flow[:,1], color='firebrick', label='flow', alpha=0.5)
plt.legend()
plt.show()
plt.subplot(1, 2, 1)
sns.distplot(X[:,0], hist=False, kde=True,
bins=None,
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 2},
label='data')
sns.distplot(X_flow[:,0], hist=False, kde=True,
bins=None, color='firebrick',
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 2},
label='flow')
plt.title(r'$p(x_1)$')
plt.subplot(1, 2, 2)
sns.distplot(X[:,1], hist=False, kde=True,
bins=None,
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 2},
label='data')
sns.distplot(X_flow[:,1], hist=False, kde=True,
bins=None, color='firebrick',
hist_kws={'edgecolor':'black'},
kde_kws={'linewidth': 2},
label='flow')
plt.title(r'$p(x_2)$')
plt.show()
Conclusions¶
In this tutorial, we have explained the basic idea behind normalizing flows and the Pyro interface to create flows to represent univariate, multivariate, and conditional distributions. It is useful to think of flows as a powerful general-purpose tool in your probabilistic modelling toolkit, and you can replace any existing distribution in your model with one to increase its flexibility and performance. We hope you have fun exploring the power of normalizing flows!
References¶
E.G. Tabak, Christina Turner. A Family of Nonparametric Density Estimation Algorithms. Communications on Pure and Applied Mathematics, 66(2):145–164, 2013.
Danilo Jimenez Rezende, Shakir Mohamed. Variational Inference with Normalizing Flows. ICML 2015.
Ivan Kobyzev, Simon J.D. Prince, and Marcus A. Brubaker. Normalizing Flows: An Introduction and Review of Current Methods. [arXiv:1908.09257] 2019.
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, Balaji Lakshminarayanan. Normalizing Flows for Probabilistic Modeling and Inference. [arXiv:1912.02762] 2019.
Conor Durkan, Artur Bekasov, Iain Murray, George Papamakarios. Neural Spline Flows. NeurIPS 2019.
Hadi M. Dolatabadi, Sarah Erfani, Christopher Leckie. Invertible Generative Modeling using Linear Rational Splines. AISTATS 2020.
James Stewart. Calculus. Cengage Learning. 9th Edition 2020.
Laurent Dinh, David Krueger, Yoshua Bengio. NICE: Non-linear Independent Components Estimation. Workshop contribution at ICLR 2015.
Laurent Dinh, Jascha Sohl-Dickstein, Samy Bengio. Density estimation using Real-NVP. Conference paper at ICLR 2017.
David Ha, Andrew Dai, Quoc V. Le. HyperNetworks. Workshop contribution at ICLR 2017.
[ ]:


class: center, middle, inverse, title-slide # Point Estimation ### Sebastian Hoyos-Torres --- # What we will talk about in the next two weeks: - General Concepts - Methods of point estimation --- # The Basis of Statistical Inference - The reason people use statistics is because we often want to know characteristics of a population from the characteristics of a sample. - Often, this results in us taking samples (for multiple reasons) in order to estimate the population. - This results in an **estimator** which is a function defined on a sample space. The resulting value is called an **estimate**. - Since we want our estimator to be a sensible rule and to give estimates which are close in some way to the population characteristic we want to determine. But how do we know if our estimates are sensible? --- # The "Sniff Test" - A simple way of choosing your estimator is to pick the sample version of the parameter to be estimated: - This means that when we want to find the expected value of a population, we should use the sample mean - Likewise, if we are interested in the population standard deviation, we can use the sample standard deviation to generate estimates of the population. --- # Example - Suppose we had the following random sample of observations on coating thickness for low viscosity paint. Assuming the distribution of coating thickness is normal;find the following properties: ``` ## [1] 0.84 0.88 0.88 1.04 1.10 1.15 1.29 1.31 1.48 1.49 1.60 1.65 1.75 1.71 ## [15] 1.86 ``` - Calculate the point estimate for the mean value of coating thickness. - Calculate a point estimate for the median. - Calculate a point value for the 95th percentile of coating thickness - Estimate P(X < 1.6) - **As always we can do this in R** --- # Doing it in R - For the first problem, we just take the sample mean of the values of the coating thickness (which is done through mean) ```r mean(vectors) ``` ``` ## [1] 1.335333 ``` - The median is simply calculated as follows ```r median(vectors) ``` ``` ## [1] 1.31 ``` - of course to speed up some of the processes, we can simply use summary from base R ```r summary(vectors) ``` ``` ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 0.840 1.070 1.310 1.335 1.625 1.860 ``` --- # Example Solution continued - the point value for the 95th percentile of coating thickness is simply: ```r quantile(vectors,.95) ``` ``` ## 95% ## 1.783 ``` - And finally, for our point estimates regarding the mean, we can do a little R subsetting to speed things up for us ```r length(vectors[vectors <= 1.5])/length(vectors) ``` ``` ## [1] 0.6666667 ``` - **OR** since we have already determined that the population is normal ```r pnorm(1.5, mean = mean(vectors), sd = sd(vectors)) ``` ``` ## [1] 0.6847211 ``` --- # General concepts of point estimation - A point estimate of a parameter `\(\theta\)` is a single number which can be sensibly regarded as a sensible value of `\(\theta\)` which is often written as `\(\hat{\theta}\)` - The corresponding random variable is called an estimator. This is written as `\(\Theta\)` - Often, there is a simple and obvious estimator. - For binomial data, the parameter is p, the probability of success, and the obvious estimator is `\(\hat{P}= X/n\)`, the proportion of successes. - For a sample of continuous measurements from a distribution with mean `\(\mu\)` and variance `\(\sigma^2\)` we use the estimator `\(\hat{\mu}= \bar{X} = \frac{1}{n}\Sigma{x_i}\)`. The usual estimator for `\(\sigma^2\)` is `\(\hat{\sigma^2} = S^2\)` with value `\(s^2\)` --- # Example - Let's look at another example on dielectric breakdown voltage for pieces of epoxy resin. ``` ## [1] 24.46 25.65 26.25 26.50 26.66 27.20 27.30 27.54 27.76 27.94 27.90 ## [12] 28.05 28.29 28.50 28.87 29.11 29.13 29.50 30.90 31.37 ``` - Assuming breakdown voltage has a normal distribution with an unknown mean of `\(\mu\)` - We have 20 independent, identically distributed (iid) normal random variables `\(X_1,...,X_{20}\)` with mean `\(\mu\)`. --- --- # Example continued: - The sample mean ```r mean(dielectric) ``` ``` ## [1] 27.944 ``` - The sample median: ```r median(dielectric) ``` ``` ## [1] 27.92 ``` - the average of the largest and smallest values ```r mean(dielectric[c(1,20)]) ``` ``` ## [1] 27.915 ``` - a trimmed mean: ```r mean(dielectric,trim = 0.1) ``` ``` ## [1] 27.90625 ``` --- # Things to think about: - Which of the prior estimates are best for handling a normal distribution? - Any of these rules could get a value close to the real value in any sample occurrence. - Ideally, we want our estimator to estimate the correct value on average and not have too much variation around that value. - Simulation time! --- # The Simulation - Let's try running 5000 simulations with each simulation having a sample size of 20 with mean 20 and standard deviation 5. ```r mn1 <- mn2 <- mn3 <- mn4 <- c() for (i in 1:5000) { x <- rnorm(20,20,5) mn1[i] <- mean(x) mn2[i] <- median(x) mn3[i] <- mean(x, trim = .1) mn4[i] <- min(x) + max(x)/2 meanlist <- list(sample_mean = mn1,sample_median= mn2, trimmed_mean = mn3,min_max_mean = mn4) } par(mfrow = c(2,2)) lapply(meanlist, hist) ```  ``` ## $sample_mean ## $breaks ## [1] 15.5 16.0 16.5 17.0 17.5 18.0 18.5 19.0 19.5 20.0 20.5 21.0 21.5 22.0 ## [15] 22.5 23.0 23.5 24.0 24.5 25.0 ## ## $counts ## [1] 1 2 13 39 122 258 501 701 910 863 691 472 250 112 45 16 3 ## [18] 0 1 ## ## $density ## [1] 0.0004 0.0008 0.0052 0.0156 0.0488 0.1032 0.2004 0.2804 0.3640 0.3452 ## [11] 0.2764 0.1888 0.1000 0.0448 0.0180 0.0064 0.0012 0.0000 0.0004 ## ## $mids ## [1] 15.75 16.25 16.75 17.25 17.75 18.25 18.75 19.25 19.75 20.25 20.75 ## [12] 21.25 21.75 22.25 22.75 23.25 23.75 24.25 24.75 ## ## $xname ## [1] "X[[i]]" ## ## $equidist ## [1] TRUE ## ## attr(,"class") ## [1] "histogram" ## ## $sample_median ## $breaks ## [1] 14 15 16 17 18 19 20 21 22 23 24 25 26 ## ## $counts ## [1] 1 10 48<PHONE_NUMBER> 1372 781 248 63 12 1 ## ## $density ## [1] 0.0002 0.0020 0.0096 0.0634 0.1580 0.2714 0.2744 0.1562 0.0496 0.0126 ## [11] 0.0024 0.0002 ## ## $mids ## [1] 14.5 15.5 16.5 17.5 18.5 19.5 20.5 21.5 22.5 23.5 24.5 25.5 ## ## $xname ## [1] "X[[i]]" ## ## $equidist ## [1] TRUE ## ## attr(,"class") ## [1] "histogram" ## ## $trimmed_mean ## $breaks ## [1] 15.5 16.0 16.5 17.0 17.5 18.0 18.5 19.0 19.5 20.0 20.5 21.0 21.5 22.0 ## [15] 22.5 23.0 23.5 24.0 24.5 25.0 ## ## $counts ## [1] 2 2 21 50 138 252 503 686 879 869 656 478 259 135 44 19 5 ## [18] 0 2 ## ## $density ## [1] 0.0008 0.0008 0.0084 0.0200 0.0552 0.1008 0.2012 0.2744 0.3516 0.3476 ## [11] 0.2624 0.1912 0.1036 0.0540 0.0176 0.0076 0.0020 0.0000 0.0008 ## ## $mids ## [1] 15.75 16.25 16.75 17.25 17.75 18.25 18.75 19.25 19.75 20.25 20.75 ## [12] 21.25 21.75 22.25 22.75 23.25 23.75 24.25 24.75 ## ## $xname ## [1] "X[[i]]" ## ## $equidist ## [1] TRUE ## ## attr(,"class") ## [1] "histogram" ## ## $min_max_mean ## $breaks ## [1] 12 14 16 18 20 22 24 26 28 30 32 34 36 ## ## $counts ## [1] 2 15 47 148<PHONE_NUMBER> 1225 696 194 23 2 ## ## $density ## [1] 0.0002 0.0015 0.0047 0.0148 0.0431 0.0904 0.1313 0.1225 0.0696 0.0194 ## [11] 0.0023 0.0002 ## ## $mids ## [1] 13 15 17 19 21 23 25 27 29 31 33 35 ## ## $xname ## [1] "X[[i]]" ## ## $equidist ## [1] TRUE ## ## attr(,"class") ## [1] "histogram" ``` --- # Unbiasedness: - An estimator `\(\Theta\)` for the parameter `\(\theta\)` is said to be unbiased if `$$E[\hat{\Theta}] = \theta$$` - The sample mean `\(\bar{X}\)` is an unbiased estimator for `\(\mu\)` if we can assume that `\(X_1,...,X_n\)` are a random sample (independent and identically distributed) - The sample median is an unbiased estimator for `\(\mu\)` if we can assume that `\(X_1,...,X_n\)` are a random sample and the distribution of the `\(X_i's\)` is continuous and symmetrical. - The sample variance `\(S^2 = \frac{(\Sigma{X_i - \bar{X})^2}}{n-1}\)` is an unbiased estimator for `\(\sigma^2\)` with a random sample from a normal population. - **NOTE** even when `\(S^2\)` is an unbiased estimator for `\(\sigma^2\)`, `\(\sqrt{S^2}\)` is **NOT** an unbiased estimator for `\(\sigma\)`. --- # More Notes on Unbiasedness - Every time we take a random sample from a population we get different values. We use those to compute an estimate `\(\hat{\Theta_i}\)` of some population parameter `\(\theta\)`. We will almost never know the value of `\(\theta\)`. - `\(\hat{\Theta_i}\)` is calculated from DATA so it in turn also has its own probability distribution. - Usually, we want as unbiased an estimator as we can get. - formally, if our `\(\hat{\Theta_i}\)` is not systematically over or underestimating `\(\theta\)`, then we typically refer to it as an unbiased estimator. --- # Variance of estimators: - Given two unbiased estimators, we generally prefer the one with the smallest variance. - Occasionally, it is possible to prove mathematically that an estimator is a minimum variance unbiased estimator. This means it has the minimum variance among the class of unbiased estimators so it should be good to use. - Generally, the desirability of an estimator depends on the form of the underlying distribution. When working with actual data, we often don't know the distribution though. - Let's try it out! --- # Simulation example: - Suppose we have multiple populations of interest following the cauchy, normal and uniform distributions as follows (normal is red, cauchy is blue).  --- # The Simulation continued: Let's continue with a for loop ```r mns1 <- mns2 <- mns3 <- c() for(i in seq_along(1:5000)){ mns1[i] <- mean(rnorm(200, 6,1)) mns2[i] <- mean(rcauchy(200,6,1)) mns3[i] <- mean(runif(200,3,9)) meanslist <- list(normal_means = mns1,cauchy_means = mns2,unif_means = mns3) } par(mfrow = c(1,3)) lapply(meanslist,hist) ```  ``` ## $normal_means ## $breaks ## [1] 5.75 5.80 5.85 5.90 5.95 6.00 6.05 6.10 6.15 6.20 6.25 6.30 ## ## $counts ## [1] 9 61<PHONE_NUMBER> 1302 787 338 60 14 1 ## ## $density ## [1] 0.036 0.244 1.308 3.192 5.212 5.208 3.148 1.352 0.240 0.056 0.004 ## ## $mids ## [1] 5.775 5.825 5.875 5.925 5.975 6.025 6.075 6.125 6.175 6.225 6.275 ## ## $xname ## [1] "X[[i]]" ## ## $equidist ## [1] TRUE ## ## attr(,"class") ## [1] "histogram" ## ## $cauchy_means ## $breaks ## [1] -1600 -1400 -1200 -1000 -800 -600 -400 -200 0 200 400 ## [12]<PHONE_NUMBER> 1200 1400 ## ## $counts ## [1] 1 0 0 0 1 0 4 252 4735 4 1 1 0 0 ## [15] 1 ## ## $density ## [1] 0.000001 0.000000 0.000000 0.000000 0.000001 0.000000 0.000004 ## [8] 0.000252 0.004735 0.000004 0.000001 0.000001 0.000000 0.000000 ## [15] 0.000001 ## ## $mids ## [1] -1500 -1300 -1100 -900 -700 -500 -300 -100 100 300 500 ## [12]<PHONE_NUMBER> 1300 ## ## $xname ## [1] "X[[i]]" ## ## $equidist ## [1] TRUE ## ## attr(,"class") ## [1] "histogram" ## ## $unif_means ## $breaks ## [1] 5.55 5.60 5.65 5.70 5.75 5.80 5.85 5.90 5.95 6.00 6.05 6.10 6.15 6.20 ## [15] 6.25 6.30 6.35 6.40 6.45 ## ## $counts ## [1] 2 8 25 53 131 301 469 666 844 773 683 472 316 151 63 30 11 ## [18] 2 ## ## $density ## [1] 0.008 0.032 0.100 0.212 0.524 1.204 1.876 2.664 3.376 3.092 2.732 ## [12] 1.888 1.264 0.604 0.252 0.120 0.044 0.008 ## ## $mids ## [1] 5.575 5.625 5.675 5.725 5.775 5.825 5.875 5.925 5.975 6.025 6.075 ## [12] 6.125 6.175 6.225 6.275 6.325 6.375 6.425 ## ## $xname ## [1] "X[[i]]" ## ## $equidist ## [1] TRUE ## ## attr(,"class") ## [1] "histogram" ``` --- # Further look at the distributions and their estimates: ```r fnctlist <- list(sample_mean = mean,sample_sd = sd) unlist(lapply(fnctlist,function(f){lapply(meanslist,f)})) ``` ``` ## sample_mean.normal_means sample_mean.cauchy_means sample_mean.unif_means ## 6.0005443 5.9078670 6.0023537 ## sample_sd.normal_means sample_sd.cauchy_means sample_sd.unif_means ## 0.0705207 36.0687311 0.1209933 ``` From the prior plot, we should begin to notice that the mean is a horrendous estimator for the cauchy distribution. --- --- # Standard Error of an Estimator - Once we compute a point estimator, often, we are interested in how precise that estimate would be from sample to sample - The standard deviation of the estimator is a reasonable measure to use as it measures the dispersion surrounding the distribution. The standard deviation is called the standard error or the estimator. - For a binomial model, the estimator of success probability ,p, `\(\hat{P} = \frac{X}{n}\)` has a standard deviation `\(\sqrt{\frac{p(1-p)}{n}}\)`. This depends on p which we often don't know and are trying to estimate. - For a normal (or near normal model), to estimate `\(\mu\)`, we use estimator `\(\bar{X}\)` whose standard deviation is `\(\frac{\sigma}{\sqrt{n}}\)` - To address unknown parameters in the population, we often rely on the estimated standard error of the estimator which is calculated by: For binomially distributed data `$$\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$` or for normally distributed data: `\(\frac{s}{\sqrt{n}}\)` --- # The Formulation of Estimators - We have discussed how to compare different estimators based off of a parameter based on their expected values and variances. - However, knowing how to compare estimators does not help us in formulating an estimator. - Two methods : maximum likelihood estimate and method of moments --- # Method of Moments - Formal Definition Let `\(X_1,...,X_n\)` be a random sample from a pmf or pdf. For k = 1,2,3,...,the kth population moment, is `\(E(X^k)\)`. The sample moment is `\(\frac{1}{n}\Sigma{X_i^k}\)` Let `\(X_1,...,X_n\)` be a random sample from a pmf/ pdf `\(f(x,\Theta_1,...\Theta_m)\)` are parameters whose values are unknown. Then the moment estimators are obtained by equating the first m sample moments and solving for `\(\Theta_1,...,\Theta_m\)` --- # An example from the gamma distribution ```r x <- c(152,115,109,94,88,137,152,77,160,165,125,40,128,123,136,101,62,153, 83,69) (n <- length(x)) ``` ``` ## [1] 20 ``` ```r (EX <- sum(x)/n) #1st sample moment ``` ``` ## [1] 113.45 ``` ```r (EX2 <- sum(x^2)/n) ``` ``` ## [1] 14087.75 ``` ```r (alphahat <- -(EX^2/(EX^2-EX2))) ``` ``` ## [1] 10.57725 ``` ```r (betahat <- -(-EX2 + EX^2)/EX) ``` ``` ## [1] 10.72585 ``` --- # Problems with Methods of Moments - Problems that arise with Method of moments estimators is the math can get complicated pretty quickly. For example; let's look at the weibull distribution. Starting with the equations `$$\mu = \beta \Gamma{(1 + \frac{1}{\alpha})}$$` `$$\sigma^2 = \beta^2[\Gamma(1+\frac{2}{\alpha})-[\Gamma(1 + \frac{1}{\alpha})^2]]$$` - Methods of moments typically use a few quantiles from the distribution and the data to formulate estimators. The values of the estimators do not have to be consistent with the distribution. --- # The Maximum Likelihood Estimation: - The essence of the maximum likelihood estimate is that we want to find the value of `\(\theta\)` which maximizes the probability/likelihood of getting the data observed. - R.A. Fisher suggested that to avoid some of the problems involved in maximum likelihood estimation, we can consider the joint density of the responses as a function of the parameters with the data fixed. `$$L(\theta|y) = f(y|\theta)$$` where L is the likelihood, f is the probability density, y is the vector of responses, and `\(\theta\)` is the vector of parameters for the distribution. - Fisher argued that we should choose estimates of parameters which provided the greatest likelihood of seeing the data that we did. In other words `$$\hat{\theta} = argmaxL(\theta|y)$$` --- # Maximum Likelihood - Likelihood function: Let f(x,$\theta$) be the probability density function for a discrete or continuous probability distribution for random variable X. Suppose `\(X_1,...,X_n\)` are the actual observed values from a random sample of size n from X. Then the likelihood function is `$$L(x_1,x_2,...,x_n,\theta) = L(\theta|x_1,x_2,...,x_n) = \Pi_{i = 1}^nf(x_i,\theta)$$` considered as a function of `\(\theta\)` with the `\(x_i's\)` fixed. `\(\theta\)` can be multidimensional. --- # Example from (https://onlinecourses.science.psu.edu/stat414/node/191/) Suppose the weights of randomly selected American female college students are normally distributed with unknown mean μ and standard deviation σ. A random sample of 10 American female college students yielded the following weights (in pounds): ```r weights <- c(115,122,130,127,149,160,152,138,149,180) ``` Based on the definitions given above, identify the likelihood function and the maximum likelihood estimator of μ, the mean weight of all American female college students. Using the given sample, find a maximum likelihood estimate of μ as well. --- # Example worked out - In chapter 4 we talked briefly about normal random variables and touched briefly upon their pdfs. - If we recall, the probability density function for the normal random variable was simply `$$f(x;\mu,\sigma) = \frac{1}{\sigma\sqrt{2\pi}}e^{\frac{(x-\mu)^2}{2\sigma^2}}$$` The Likelihood function then becomes: `$$L(\theta_1,\theta_2) = \sigma^{-n}2(\pi)^{-n/2}exp[-\frac{1}{2\sigma^2}\Sigma(x_i-\mu)^2]$$` Which ultimately results in: `$$\frac{1}{n}\Sigma{x_i}$$` or ```r sum(weights)/length(weights) ``` ``` ## [1] 142.2 ``` --- # Properties of the Maximum Likelihood Estimate: - The Likelihood or log likelihood measures the suitability of parameters for the probability model applied to the data. - For distributions with one or two parameters, we can examine the log-likelihood function or the contours of the function to get an idea surrounding the precision of the estimates. - There is typically a way to associate a confidence level with particular contours of the log-likelihood. - The invariance principle (see p 260 of the text.) - MLE's are the best estimators which can be formulated.


Precedence¶ ↑
From highest to lowest, this is the precedence table for ruby. High precedence operations happen before low precedence operations.
!, ~, unary + ** unary - *, /, % +, - <<, >> & |, ^ >, >=, <, <= <=>, ==, ===, !=, =~, !~ && || .., ... ?, : modifier-rescue =, +=, -=, etc. defined? not or, and modifier-if, modifier-unless, modifier-while, modifier-until { } blocks
Unary +
and unary -
are for +1
,
-1
or -(a + b)
.
Modifier-if, modifier-unless, etc. are for the modifier versions of those keywords. For example, this is a modifier-unless expression:
a += 1 unless a.zero?
{ ... }
blocks have priority below all listed operations, but
do ... end
blocks have lower priority.
All other words in the precedence table above are keywords.


I am a resilient, innovative, and passionate software engineer with a strong desire to learn and grow. My goal is to make a positive impact on the world by building solutions to real-life problems and contributing to open-source.
As a full-stack developer and cloud engineer, I have experience in a variety of programming languages and tools. I am always eager to expand my skillset and learn new technologies. In my free time, I enjoy automating tasks and exploring new software development trends.
Currently, I am pursuing a Master's Degree in Computer Science at San Diego State University, where I also work as a Research Assistant for Dr. Homayouni and Dr. Shirazi on phishing detection and analysis. I conducted data analysis and research to improve phishing detection techniques and built an image scanning deep learning model using Tensorflow.
I have designed this website to mimic the experience of using an Ubuntu shell. As a back-end developer, I wanted to create a CLI-based site that allows you to run commands and learn about me and my work through the terminal. With this site, you can interact with the interface and access information about my experience, skills, and projects in a unique and engaging way.
Thank you for visiting my website, and please feel free to reach out if you have any questions or just want to say hello!


👋 Hey y'all
I'm Gabriel
Web Developer and Passionate programmer 🛠
🖥 Programming
🎯 Development
📚 Learning Web
🔥 Front-end
🛠 GitHub


|
I love the sense of control and order that lists offer. My notebook is full of them; to-do lists and grocery lists, sure, but also lists of qualities I look for in a partner, books people have recommended to me, reasons I’m not interested in getting back together with my ex, how I want my ideal job to make me feel, dinner party guest lists, home improvement projects, newsletter topic ideas.  And when my fried attention span won’t let me focus on writing for more than a half hour at a time, it feels good to work on a list of things that feel good.
- The brightly painted cornice on the building at 304 West Evergreen
- The feeling of pressing a hot mug of tea against my face
- Ordering a book without knowing anything about it just because I love the author and then reading the synopsis and thinking “wow that sounds so good!”
- When someone uses an emoji so often, particularly in Slack reactions, that it becomes “theirs”
- Related: the odd, specific things you learn about coworkers just from spending so much time with them, like how Zach Breakstone always scooped out the inside of the baguette roll that came with the meatball lunch he got every day from the Italian place in the mall
- The blooming redbud outside my window, some cousin of the blooming redbud outside my dad’s home office window. We compare redbuds every time we talk
- Henna in my hair
- Podcast episodes so beautiful they make me cry every time I listen to them
- A song I think of every time I have a song stuck in my head and want to get it out.  It has served this purpose so long that I now think of it when I want to stop a line of thinking, like a painful or embarrassing memory or an anxiety spiral
- The feeling of putting on earrings with cool hands
- Stretching. God it’s so good. I know we talk a lot about how stretching is good, but I don’t think we talk about it enough
- My weekly breakfast burrito and coconut shake from the diner around the corner from my apartment (they didn’t make me spell my name when I called in my order today, so I think we’re well on our way to “the usual” status, which I’ve never had at any restaurant and delights me to think of)
- Water
- Watering plants
- The way my cat sits up tall, arching her neck and stretching on her front legs, before tipping over heavily onto me, reminding me of the comforting weight of a friend using me as a pillow on a long bus ride
- The ritual of making a drink (any drink)
- Bellflowers!! I didn’t know these existed until this week but they might be my favorite flower?? I lent a friend a book and she gave me a bouquet of these in exchange
This list was partially inspired by Hanif Abdurraqib’s essay on collage poetry and Sarah Mirk's essay on joyful participation in a world of sorrows. It was not inspired by Danny Caine's list of delights, but it harmonizes nicely. What feels good to you right now?
|
|
❧ ❧ ❧
|
|
"Really any description of Detectorists will make you think that watching paint dry would be more fun, but it's a charming, carefully observed gem of slow-paced comedy. And if you can deal with some (very) thick Cork accents, The Young Offenders is similarly great. Maybe try the movie first if it's easily available to you. It's hard to do good comedy about the lives of young working-class delinquents, but this is perfect.
What both of these have in common is a sense of warm-hearted optimism and a complete lack of cynicism or irony in treating these people as actual people and not just the butt of lazy jokes." —Michael Ashbridge
"Warm-hearted optimism" is maybe the most ideal descriptor of the genre of TV and movies that I gravitate toward so this is extremely on-brand for me. I loved the first few episodes of Detectorists and I'm excited to brave the Cork accents to check out Young Offenders!
|
|
❧ ❧ ❧
|
|
Got a recommendation of your own? Hit reply; I’d love to hear from you.
Subscribe, unsubscribe, or learn more on the website.
This edition of May I Recommend was written from an apartment strewn with craft supplies and half-finished projects in Chicago, Illinois.
|
|
|


Hi Charlie! It was good meeting you for coffee the other day, I would love to talk more about your autonomous car start up! Here's my email<EMAIL_ADDRESS>Thanks, Mygel! It was a pleasure chatting with you as well. You may also be interested in learning more about my new cryptocurrency start up, Goldhill LLC!
I recently started a cryptocurrency start up as well! I'll definitely read through the whitepaper posted on the Goldhill website and get back to you!
Sounds good to me! Looking forward to hearing from you - Charlie.


Press n or j to go to the next uncovered block, b, p or k for the previous block.
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 | /* * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with * this work for additional information regarding copyright ownership. * The ASF licenses this file to You under the Apache License, Version 2.0 * (the "License"); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ export default { service: 'Service', serviceSearch: 'Search service name', serviceGovernance: 'Service Governance', routingRule: 'Condition Rule', tagRule: 'Tag Rule', meshRule: 'Mesh Rule', dynamicConfig: 'Dynamic Config', accessControl: 'Black White List', weightAdjust: 'Weight Adjust', loadBalance: 'Load Balance', serviceTest: 'Service Test', serviceMock: 'Service Mock', serviceMetrics: 'Service Metrics', serviceRelation: 'Service Relation', metrics: 'Metrics', relation: 'Relation', group: 'Group', serviceInfo: 'Service Info', providers: 'Providers', consumers: 'Consumers', version: 'Version', app: 'Application', ip: 'IP', qps: 'qps', rt: 'rt', successRate: 'success rate', port: 'PORT', timeout: 'timeout(ms)', serialization: 'serialization', appName: 'Application Name', serviceName: 'Service Name', registrySource: 'Registry Source', instanceRegistry: 'Instance Registry', interfaceRegistry: 'Interface Registry', allRegistry: 'Instance / Interface Registry', operation: 'Operation', searchResult: 'Search Result', search: 'Search', methodName: 'Method Name', enabled: 'Enabled', disabled: 'Disabled', method: 'Method', weight: 'Weight', create: 'CREATE', save: 'SAVE', cancel: 'CANCEL', close: 'CLOSE', confirm: 'CONFIRM', ruleContent: 'RULE CONTENT', createNewRoutingRule: 'Create New Routing Rule', createNewTagRule: 'Create New Tag Rule', createNewMeshRule: 'Create New Mesh Rule', createNewDynamicConfigRule: 'Create New Dynamic Config Rule', createNewWeightRule: 'Create New Weight Rule', createNewLoadBalanceRule: 'Create new load balancing rule', serviceIdHint: 'Service ID', view: 'View', edit: 'Edit', delete: 'Delete', searchRoutingRule: 'Search Routing Rule', searchAccess: 'Search Access Rule', searchWeightRule: 'Search Weight Adjust Rule', dataIdClassHint: 'Complete package path of service interface class', dataIdVersionHint: 'The version of the service interface, which can be filled in according to the actual situation of the interface', dataIdGroupHint: 'The group of the service interface, which can be filled in according to the actual situation of the interface', agree: 'Agree', disagree: 'Disagree', searchDynamicConfig: 'Search Dynamic Config', appNameHint: 'Application name the service belongs to', basicInfo: 'BasicInfo', metaData: 'MetaData', methodMetrics: 'Method Statistics', searchDubboService: 'Search Dubbo Services or applications', serviceSearchHint: 'Service ID, org.apache.dubbo.demo.api.DemoService, * for all services', ipSearchHint: 'Find all services provided by the target server on the specified IP address', appSearchHint: 'Input an application name to find all services provided by one particular application, * for all', searchTagRule: 'Search Tag Rule by application name', searchMeshRule: 'Search Mesh Rule by application name', searchSingleMetrics: 'Search Metrics by IP', searchBalanceRule: 'Search Balancing Rule', noMetadataHint: 'There is no metadata available, please update to Dubbo2.7, or check your config center configuration in application.properties, please check ', parameterList: 'parameterList', returnType: 'returnType', here: 'here', configAddress: 'https://github.com/apache/incubator-dubbo-admin/wiki/Dubbo-Admin-configuration', whiteList: 'White List', whiteListHint: 'White list IP address, divided by comma: <IP_ADDRESS>,<IP_ADDRESS>', blackList: 'Black List', blackListHint: 'Black list IP address, divided by comma: <IP_ADDRESS>,<IP_ADDRESS>', address: 'Address', weightAddressHint: 'IP addresses to set this weight, divided by comma: <IP_ADDRESS>,<IP_ADDRESS>', weightHint: 'weight value, default is 100', methodHint: 'choose method of load balancing, * for all methods', strategy: 'Strategy', balanceStrategyHint: 'load balancing strategy', goIndex: 'Go To Index', releaseLater: 'will release later', later: { metrics: 'Metrics will release later', serviceTest: 'Service Test will release later', serviceMock: 'Service Mock will release later' }, by: 'by ', $vuetify: { dataIterator: { rowsPerPageText: 'Items per page:', rowsPerPageAll: 'All', pageText: '{0}-{1} of {2}', noResultsText: 'No matching records found', nextPage: 'Next page', prevPage: 'Previous page' }, dataTable: { rowsPerPageText: 'Rows per page:' }, noDataText: 'No data available' }, configManage: 'Configuration Management', configCenterAddress: 'ConfigCenter Address', searchDubboConfig: 'Search Dubbo Config', createNewDubboConfig: 'Create New Dubbo Config', scope: 'Scope', name: 'Name', warnDeleteConfig: ' Are you sure to Delete Dubbo Config: ', warnDeleteRouteRule: 'Are you sure to Delete routing rule', warnDeleteDynamicConfig: 'Are you sure to Delete dynamic config', warnDeleteBalancing: 'Are you sure to Delete load balancing', warnDeleteAccessControl: 'Are you sure to Delete access control', warnDeleteTagRule: 'Are you sure to Delete tag rule', warnDeleteMeshRule: 'Are you sure to Delete mesh rule', warnDeleteWeightAdjust: 'Are you sure to Delete weight adjust', configNameHint: "Application name the config belongs to, use 'global'(without quotes) for global config", configContent: 'Config Content', testMethod: 'Test Method', execute: 'EXECUTE', result: 'Result: ', success: 'SUCCESS', fail: 'FAIL', detail: 'Detail', more: 'More', copyUrl: 'Copy URL', copy: 'Copy', url: 'URL', copySuccessfully: 'Copied', test: 'Test', placeholders: { searchService: 'Search by service name' }, methods: 'Methods', testModule: { searchServiceHint: 'Entire service ID, org.apache.dubbo.demo.api.DemoService, press Enter to search' }, userName: 'User Name', password: 'Password', login: 'Login', apiDocs: 'API Docs', apiDocsRes: { dubboProviderIP: 'Dubbo Provider Ip', dubboProviderPort: 'Dubbo Provider Port', loadApiList: 'Load Api List', apiListText: 'Api List', apiForm: { missingInterfaceInfo: 'Missing interface information', getApiInfoErr: 'Exception in obtaining interface information', api404Err: 'Interface name is incorrect, interface parameters and response information are not found', apiRespDecShowLabel: 'Response Description', apiNameShowLabel: 'Api Name', apiPathShowLabel: 'Api Path', apiMethodParamInfoLabel: 'Api method parameters', apiVersionShowLabel: 'Api Version', apiGroupShowLabel: 'Api Group', apiDescriptionShowLabel: 'Api Description', isAsyncFormLabel: 'Whether to call asynchronously (this parameter cannot be modified, according to whether to display asynchronously defined by the interface)', apiModuleFormLabel: 'Api module (this parameter cannot be modified)', apiFunctionNameFormLabel: 'Api function name(this parameter cannot be modified)', registryCenterUrlFormLabel: 'Registry address. If it is empty, Dubbo provider IP and port will be used for direct connection', paramNameLabel: 'Parameter name', paramPathLabel: 'Parameter path', paramDescriptionLabel: 'Description', paramRequiredLabel: 'This parameter is required', doTestBtn: 'Do Test', responseLabel: 'Response', responseExampleLabel: 'Response Example', apiResponseLabel: 'Api Response', LoadingLabel: 'Loading...', requireTip: 'There are required items not filled in', requireItemTip: 'This field is required', requestApiErrorTip: 'There is an exception in the request interface. Please check the submitted data, especially the JSON class data and the enumeration part', unsupportedHtmlTypeTip: 'Temporarily unsupported form type', none: 'none' } }, authFailed: 'Authorized failed,please login.', ruleList: 'Rule List', mockRule: 'Mock Rule', mockData: 'Mock Data', globalDisable: 'Global Disable', globalEnable: 'Global Enable', saveRuleSuccess: 'Save Rule Successfully', deleteRuleSuccess: 'Delete Rule Successfully', disableRuleSuccess: 'Disable Rule Successfully', enableRuleSuccess: 'Enable Rule Successfully', methodNameHint: 'The method name of Service', createMockRule: 'Create Mock Rule', editMockRule: 'Edit Mock Rule', deleteRuleTitle: 'Are you sure to delete this mock rule?' } |


Good example for Leaflet - Russia. It it so big, that goes beyond 180 degrees. As result you cannot display Russia without .toLeaflet (or .toUnboundedCoordinates) command.


--- layout: "영화" title: "[영화] 나잇앤데이 (Knight and day, 2010)" date: '2010-12-11' comments: true adsense: true ignored: true tags: - knight and day - movie - "나잇앤데이" - "미국영화" - "영화" ---
이미지출처 : www.bscreview.com


--- layout: default ---
Ultimate LI is a DIY implementation of the Lenz Interface. It is a fully operable XpressNET to PC converter.
Ultimate LI is a simple, opensource and openhardware implementation of the original Lenz Interface developed in KMZ Brno I (Model Railroader Club in Brno, Czech Republic). This project constists of a PCB layout design and a firmware to the main CPU on the board. uLI connects to any command station supprosting XpressNET 3.0 or 3.6. On the other side, uLI is connected directly to the PC via simple USB cable. After installation of a simple driver, PC sees uLI simply as a serial port. This allows common application such as JMRI to connect to you command station. uLI is available for free and for everyone. All materials are in public repositories and everyone is welcome to contribute to this project!
Ultimate LI fits into 7×5×3 cm case. Just connect your uLI to the XpressNET and to the PC. Yes, that's all. Such simple.
Every application supporting LI101 of LI101F is also compatible with uLI. We heavily use uLI with JMRI and hJOP for both normal traffic and programming.
All the source codes and other materials are available in public GIT repositories.


Home
Hello
Boot Security
Invalid username and password.
You have been logged out.
Login
User Name
Password
Sign in


Pennyworth
Pennyworth follows Bruce Wayne's legendary butler, Alfred Pennyworth, a former British SAS soldier who forms a security company and goes to work with Thomas Wayne, Bruce's billionaire father, in 1960s London.


{% META %}
{% TOC %}
{% BODY %}
{% NOTES %}
This content was automatically generated by parsing
a Markdown original content
.


$CD36/52534: Read block to buffer Jump from $CD42, $CDA6: CD36: 20 F2 CD JSR $CDF2 ; open channel, set parameters CD39: 4C 60 D4 JMP $D460 ; read block from disk Previous Page / Next Page


The two samples below demonstrate a Calendar HTC component implemented using two variations of Element Behaviors. The first page implements a regular Element Behavior and the second implements an Element Behavior with a ViewLink. The samples show how a ViewLink encapsulates document content and permits greater control over CSS Style inheritance.
Control: Clicking the button below will apply a new stylesheet rule to all of the <TD> elements on page 1 and 2. Notice that the <TD> elements in the Element Behavior sample are changed, but the ViewLink elements stay the same.


FATHERLAND (1994) A Film Review Copyright Dragan Antulov 1999
Almost half a century after World War Two, there are still many voices, especially in some countries of Eastern Europe, who lament the outcome of that global conflict. For them, the wrong side has lost and the world would have been much better place if it happened otherwise. For others, Hitler's victory in WW2 is a matter of pure speculation, and Sci-Fi novels from the alternative history sub-genre used that concept very often (most notably in 1962 novel THE MAN IN THE HIGH CASTLE by Philip K. Dick). However, first such novel to reach the mainstream audience was Robert Harris' FATHERLAND, bestseller who later served as basis for 1994 HBO movie, directed by Christopher Menaul.
The plot of this film is set in the alternative 1964, in a world where Hitler managed to successfully finish conquest of Europe, while USA was too busy making war with Japan. The result is Germania, grand Nazi empire that controls the entire Old Continent, with 75-year old Adolf Hitler still at its helm. After the decades of cold war and still frustrated with the inability to finish the bitter guerrilla war with the remnants of Soviet Union, Hitler's regime wants to improve relations with the other superpower, USA, led by anti-Semitic and pro-German President Joseph Kennedy. The summit in Berlin is set, and among many invited journalists is Charlie Maguire (played by Miranda Richardson). She would stumble on Xavier March (Rutger Hauer), SS officer who is investigating the mysterious death of former Nazi official. The murder is somewhat connected with the "Jewish question" conference that took place during the war, and almost all participants are dying. While trying to find the motive for the killings, March and Maguire become targets of Gestapo.
The entire concept of alternative history is really fascinating. On one hand, the created worlds are so different from the world we live in, and on the other hand, they are in the same time closer and more recognisable than fantastic worlds of distant galaxies or distant futures. The world of FATHERLAND, although some may question its plausibility (most of the today's historians are almost unanimous in their opinion that Hitler could never win WW2, at least not in the way indicated in the film), is very realistic one. The most intriguing idea is to show the most evil and universally despised political system in the world - German Nazism - slowly evolving into its "gentler and kinder" form, in a manner very similar to the path Soviet Union took from Stalin to Gorbachev era.
Unfortunately, like many good novels, FATHERLAND suffered a terrible fate in its version as feature film. Instead of putting an effort to show the fascinating and intriguing details of Robert Harris' world, screenplay by Ron Hutchinson and Stanley Weiser concentrated almost exclusively on the lame, predictable and boring crime investigation. Such subplots, as the example of much better GORKY PARK shows us, simply don't work in totalitarian settings, as long as the all-powerful government is one of culprits. And with Nazi Germany, even in its "reformed" version, that is definitely the case. Things are even made worse with the poorly drawn characters, and actors who don't put much effort in playing, which is especially the case with Rutger Hauer. Terrible scripting culminates with hardly believable, unoriginal finale and corny epilogue that would have to establish the parallels between the fall of alternative Nazism and our world's Communism.
It would be unfair to say that FATHERLAND doesn't have its good sides. Director Menaul put some real effort in portraying some interesting details while portraying Robert Harris' world (like the posters of "Die Beatles" band). The locations and some of the artifacts from Communist era Czehoslovakia provided good hints how would latter dazy Third Reich would look like. Unfortunately, all those details are catching our attention only in the first part of the film. The rest is one great disappointment - one of the most tragic examples of great movies ideas being wasted in mediocre screen effort.
RATING: 4/10 (+)
Review written on May 24th 1999
Dragan Antulov a.k.a. Drax Fido: 2:381/100 E-mail<EMAIL_ADDRESS>E-mail<EMAIL_ADDRESS>E-mail<EMAIL_ADDRESS>The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


RECENT ACTIVITIES
CS5180098  Dipen Kumar joined BeMotivated Network.
CS1180396  Utsav joined BeMotivated Network.
CS1180334  Varsha joined BeMotivated Network.
CE1180075  Abhishek Parewa joined BeMotivated Network.
CE1180082  Aniket Jain joined BeMotivated Network.
CE1180102  Jatin Meena joined BeMotivated Network.
CE1180108  Harsha Vardhan joined BeMotivated Network.
CE1180123  Nishant Gupta joined BeMotivated Network.
CS1180330  Diwakar Prajapati joined BeMotivated Network.
CS1180343  Jatin Munjal joined BeMotivated Network.
CS5180411  Manoj Kumar joined BeMotivated Network.
CS5180413  Nikita Bhamu joined BeMotivated Network.
ME1180585  Akhil Bajiya joined BeMotivated Network.
PH1180832  Not to be Motivated joined BeMotivated Network.
PH1180840  Ruhanshi Barad joined BeMotivated Network.
PH1180841  Sakshi Mittal joined BeMotivated Network.
BB1180022  Kavya Parnami joined BeMotivated Network.
CS1180343  Jatin Munjal posted a new Story.
CS5180098  Dipen Kumar posted a new Story.
CS5180098  Dipen Kumar posted a new Story.
CE1180082  Aniket Jain posted a new Story.
CS5180411  Manoj Kumar posted a new Story.
CS5180411  Manoj Kumar posted a new Story.
CS5180411  Manoj Kumar posted a new Story.
CS5170422  Yash Jain joined BeMotivated Network.
ME2180678  Bhanu Pratap Asiwal joined BeMotivated Network.
CS1180386  Saurabh Kumar joined BeMotivated Network.
CS1180381  Sarang Chaudhari joined BeMotivated Network.
CS1180369  PULA JASWANTH RAM joined BeMotivated Network.
CS1180391  Siddhant Choudhary joined BeMotivated Network.
MT1180736  Aditi joined BeMotivated Network.
CS1180348  Kapil verma joined BeMotivated Network.
CS1180330  Diwakar Prajapati posted a new Story.
CS1180348  Kapil verma posted a new Story.
EE1180477  Mehtab Alam joined BeMotivated Network.
PH1180622  Mayank Jha joined BeMotivated Network.
EE1180603  DEDEEPYO RAY joined BeMotivated Network.
PH1180854  Kishore Kumar joined BeMotivated Network.
ME1180602  Chhavi Choudhary joined BeMotivated Network.
CE1180082  Aniket Jain posted a new Story.
ME2180685  Harshdeep Singh joined BeMotivated Network.
CS5180424  Umesh Parmar joined BeMotivated Network.
CS5180410  Manav Bansal joined BeMotivated Network.
CS5180411  Manoj Kumar posted a new Story.
CS5180411  Manoj Kumar posted a new Story.
CS5180411  Manoj Kumar posted a new Story.
CS1180374  Rishabh Kumar joined BeMotivated Network.
CS1180374  Rishabh Kumar posted a new Story.
CE1180082  Aniket Jain posted a new Story.
CS5180098  Dipen Kumar posted a new Story.
CE1180082  Aniket Jain posted a new Story.
CS5180098  Dipen Kumar posted a new Story.
CS5180098  Dipen Kumar posted a new Story.
CS1180334  Varsha Goalla posted a new Story.
CS1180334  Varsha Goalla posted a new Story.
CS5180411  Manoj Kumar posted a new Story.
CS1180350  Kavya joined BeMotivated Network.
CE1180082  Aniket Jain posted a new Story.


My New Hugo Site
Jekyll
March 13, 2017
Welcome to Jekyll!
You’ll find this post in your _posts directory. Go ahead and edit it and re-build the site to see your changes. You can rebuild the site in many different ways, but the most common way is to run jekyll serve, which launches a web server and auto-regenerates your site when a file is updated. To add new posts, simply add a file in the _posts directory that follows the convention YYYY-MM-DD-name-of-post.
Home
About


WITNESS LIST Appropriations-S/C on Govt. Efficiency & Oper Committee February 8, 2005-5:00P DPS Employee Compensation On: Davis, Colonel Tommy (Texas Department of Public Safety) McHugh, Colleen (Texas Public Safety Commission) State Employee Compensation For: Hawthorne, Sergeant Brian (Department of Public Safety Officers Association) O'Connor, Caroline (Texas State Employees Union) On: Hymel, Ray (Texas Public Employees Association) State Employee Issues For: Anderson, Gary (Texas Public Employees Association)


Unique
Graphics
Home
About
Services
Gallery
Contact us
Contact Us
UniqueGraphics
CONTACT
Currently Contact form functionality is disabled Kindly send an email.
Submit
UniqueGraphics
(+91)9027676843
(+91)9027676843
<EMAIL_ADDRESS>Uttarakhand Kotdwara
India


Bill Dueber, University of Michigan
z30_rec_key_3
.substr
hellThank you, Margaret Kelly and Talend software
All records with fixed field lang=eng (008/35-37) and with “&” in 245 $a or
245 $b that also have one or more 246 fields, none of which contain the
word “and.”
I would like to request a report for all items in:
Sublib = AAEL,
Collection = FUN
Please include
Title /
Author /
Call Number /
Barcode /
Loans /
Date last returned /
Date item created
Bill Dueber
University of Michigan
<EMAIL_ADDRESS>billdueber on twitter/github/irc


Cytotoxicity assay of A. citriodora essential oils from different Moroccan regions on PBMCs. AG: Agadir. BM: Beni Mellal. BE: Berkane. DE: Demnate. MA: Marrakech. DMSO: control.
Figure 6


Approved
| Reference | 98HGS3F |
| Added by | Hackney Skills and Training Ltd |
| User | Alice Jones | Date | Thursday 18 May 2017 |
| Apprentices | 23 Mechatronics Level 2 Engineers |
| Reference | 98HGS3F |
| Added by | Hackney Skills and Training Ltd |
| User | Alice Jones | Date | Thursday 18 May 2017 |
| Apprentices | 23 Mechatronics Level 2 Engineers |


| Test |
Duration |
Result |
| add and remove added vertex and check if vertex exists in graph() |
0.004s |
passed |
| addEdge() - add edge between existing vertices() |
0.002s |
passed |
| addEdge() - updates edge epoch time if edge is added and edge direction is reversed() |
0.005s |
passed |
| addEdge() - updates edge timestamp when two edges are added out of order() |
0.001s |
passed |
| addVertex() - add same vertex twice but keeps the one with latest time() |
0.004s |
passed |
| addVertex() - an old vertex operation cannot overwrite a newer vertex() |
0.003s |
passed |
| addVertex() - trivial add and check if exists in graph() |
0.002s |
passed |
| addVertex() - updates epochtime if adding same vertex twice() |
0.004s |
passed |
| exists() - return false if vertex does not exists in graph() |
0.001s |
passed |
| exists() - return true if vertex exists in graph() |
0.001s |
passed |
| findPath() - returns list of vertices in path to reach a vertex() |
0.002s |
passed |
| findPath() - returns list of vertices in path to reach a vertex, not shortest one proved() |
0.002s |
passed |
| findPath() - returns list when the first edge is the target vertex() |
0.002s |
passed |
| findPath() - when a vertex is removed and no path exists, returns empty list() |
0.003s |
passed |
| findPath() - when operations were received out of order, returns expected path list() |
0.003s |
passed |
| getAllConnectedVertices() - returns empty if no vertices found() |
0.002s |
passed |
| getAllConnectedVertices() - returns list of all vertices connect to a vertex() |
0.001s |
passed |
| getAllVertices() - given an empty graph with an added edge, should return empty() |
0.003s |
passed |
| getAllVertices() - returns empty if graph is empty() |
0.005s |
passed |
| getAllVertices() - returns list of vertices in graph() |
0.002s |
passed |
| given a vertex has been removed, a new added vertex can exist with same value() |
0.002s |
passed |
| merge() - can merge a graph with target graph for adding a vertex() |
0.001s |
passed |
| merge() - can merge a graph with target graph for adding an edge() |
0.002s |
passed |
| merge() - can merge with empty graph() |
0.025s |
passed |
| merge() - throws an error if not same graph type() |
0.082s |
passed |
| removeEdge() - remove edge between existing vertices when operations are out of order() |
0.003s |
passed |
| removeEdge() - when two remove edge operations are received, latest one wins() |
0.002s |
passed |
| removeVertex() - add vertex and remove added vertex and check if value exists in graph() |
0.001s |
passed |
| removeVertex() - given a vertex exists, can remove vertex() |
0.004s |
passed |
| removeVertex() - when two remove operations are received, the latest timestamp operation wins() |
0.002s |
passed |
| when two operations with same time are received, the first one takes priority() |
0.063s |
passed |


Exxon Valdez Matlab Simulation
(Spring - 2014)
This simulation was the final project in my computer engineering applications (ie, Matlab) class. I was just following an algorithm and some initial conditions provided so I really don't know anything about the oil spill itself or if this is an accurate simulation, but I think it demonstrates a couple of useful techniques for graph generation & visualization. Please see the Source Code (GPL License)
Result of Default Parameters:
This image shows the oil spill stemming from the center-left of the grid. The oil spreads almost immediately into the east, but the number of captures is set too low to see any cells become oil-free.
Capturing Simulation into a Movie
Matlab provides the very convenient movie() function, enabling visualizations of graph changes over time. This simulation uses a grid-overlayed scatterplot to show oil propagation and ties a number of these state captures together.
None of the default visualizations are really intended for spatial data. Conveniently, the simulation already works on a grid; After drawing a grid for each frame, the mapping from array data to spatial data is apparent.
I really wanted to use such toys as the Google Earth toolbox or the Google Maps loader to overlay the simulation grid on top of the real location in the Prince William Sound. I decided against it only because of the unknown accuracy of the simulation; It seemed a little pointless since I don't actually know which way the wind really blows.
Saving/Loading Figures
Drawing a grid is actually a rather painful process, and attempting to do so on-the-fly would make the runtime unreasonable.
This script generates and saves a grid figure to serve as the field, then uses load(file, '-mat') to load the figure. I elected to use load instead of hgload in order to avoid opening the figure while drawing. Ironically, after going through all the effort of 'silently' drawing each frame, I discovered that movie() forces a frame to become visible briefly in order to capture it.
load(file, '-mat') loads the figure data into a matlab structure; The relevant figure data (a nested structure) is hiding inside of the hgS_070000 key. This is then converted into a figure with struct2handle(figure.hgS_070000, 0) -- the zero corresponding to the parent figure. ('root figure': the Matlab environment.)
Usage & Limitations:
- Script was created and tested on R2011A. I have no idea about other versions, but it
should work. - You cannot execute this script using cell mode because it doesn't set the PATH correctly.
- If you find yourself playing with the parameters a lot, you might want to check out the console simulation mode. It skips the lengthy generation of images for the 'movie' and just prints out the current state on the console.
Parameter Notes:
- Gallons Spilled: The maximum value, 100k, is still 110 times smaller than the accepted estimate for the real spill.
- Wind Velocity: Any value of significance quickly blows the oil out of the field; Wind speeds can reach as high as 40 miles per hour, which yields depressing results.
- Iterations: More captures will make not only for a richer simulation, but also a longer one. Too long of a simulation and you may be watching the open ocean for a while. (And waiting a long time to watch anything.)
Result of Higher Parameters:
With evaporation and wind velocity set to higher values, oil can be seen being blown out/evaporated out of cells. Most of the oil is just being blown around, though, probably mostly onto Bligh Island. (Directly to the east of the spill location. )