A Thorough Introduction to Boltzmann Machines
Strategies for Confronting the Partition Function (Blog! + code)
Approximating the Softmax (Ruder)
Confronting the partition function (Slides)
Graphical Models, Exponential Families, and Variational Inference (M Jordan)
Introduction - The Partition Function
-
The Partition Function:
The Partition Function is the normalization constant of an unnormalized probability distribution \(\tilde{p}(\mathbf{x} ; \boldsymbol{\theta})\).Formally, it is the (possibly infinite) sum over the unnormalized probability \(\tilde{p}(\mathbf{x} ; \boldsymbol{\theta})\) of all the states/events \(\boldsymbol{x} \in X\),
- Discrete Variables:
$$Z(\boldsymbol{\theta}) = \sum_{\boldsymbol{x}} \tilde{p}(\boldsymbol{x})$$
- Continuous Variables:
$$Z(\boldsymbol{\theta}) = \int \tilde{p}(\boldsymbol{x}) d \boldsymbol{x}$$
It is defined such that:
$$\sum_\mathbf{x} p(\mathbf{x} ; \boldsymbol{\theta}) = \sum_\mathbf{x} \dfrac{\tilde{p}(\mathbf{x} ; \boldsymbol{\theta})}{Z(\boldsymbol{\theta})} = 1$$
Notes:
- The Partition Function contains an explicit Temperature
- The Partition Function is a generating function
- Statistical Mechanics of Learning from Examples
Sompolinsky et al. confront the partition function for a Perceptron using statistical mechanics methods developed for spin glasses and simple nets (Garder, Derrida) and applied it to Perceptrons and, later, to something like MLPs. - Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes
Uses old techniques from non-equilibrium statistical mechanics to address the modern problems of inference.
- Discrete Variables:
- Handling the Partition Function - Motivation:
Many Undirected Probabilistic Graphical Models (PGMs) are defined by an unnormalized probability distribution \(\tilde{p}(\mathbf{x} ; \boldsymbol{\theta})\).
To obtain a valid probability distribution, we need to normalize \(\tilde{p}\) by dividing by a partition function \(Z(\boldsymbol{\theta})\):$$p(\mathbf{x} ; \boldsymbol{\theta})=\dfrac{1}{Z(\boldsymbol{\theta})} \tilde{p}(\mathbf{x} ; \boldsymbol{\theta})$$
Calculating the partition function can be intractable for many interesting models.
The Partition Function in Deep Probabilistic Models:
Deep Probabilistic Models are usually designed with the partition function in mind. There a few approaches taken in the designs:- Some models are designed to have a tractable normalizing constant.
- Others are designed to be used in ways (training/inference) that avoid computing the normalized probability altogether.
- Yet, other models directly confront the challenge of intractable partition functions.
They use techniques, described below, for training and evaluating models with intractable \(Z\).
Handling the Partition Function:
There are a few approaches to handle the (intractable) partition function:- Estimate the partition function as a learned parameter; Noise-Contrastive Estimation.
- Estimate the gradient of the partition function directly; Stochastic MLE, Contrastive-Divergence.
- Avoid computing quantities related to the partition function altogether; Score Matching, Pseudolikelihood.
- Estimate the partition function (itself) explicitly: Annealed IS, Bridge Sampling, Linked IS.
- The Log-Likelihood Gradient:
Phase Decomposition of Learning:
Learning using MLE requires computing the gradient of the NLL, \(\nabla_{\boldsymbol{\theta}} \log p(\mathbf{x} ; \boldsymbol{\theta})\).
What makes learning undirected models by maximum likelihood particularly difficult is that the partition function depends on the parameters; thus, the gradient of the NLL wrt the parameters involves computing the gradient of \(Z(\mathbf{\theta})\).
In undirected models, this gradient can be written as:$$\nabla_{\boldsymbol{\theta}} \log p(\mathbf{x} ; \boldsymbol{\theta})=\nabla_{\boldsymbol{\theta}} \log \tilde{p}(\mathbf{x} ; \boldsymbol{\theta})-\nabla_{\boldsymbol{\theta}} \log Z(\boldsymbol{\theta})$$
which decomposes the gradient (learning) into a positive phase and a negative phase.
Difficulties in Learning wrt the Decomposition:
- Difficulty in the Negative Phase:
- For most undirected models of interest, the negative phase is difficult to compute. This is usually due to having to compute the unnormalized probability for all the states.
- Directed models define many “implicit” conditional independencies between the variables, making it easier to compute the normalization due to many terms canceling out.- Example - RBMs:
The quintessential example of a model with a straightforward positive phase and a difficult negative phase is the RBM.
It has hidden units that are conditionally independent from each other given the visible units.
Word2vec is another example.
- Example - RBMs:
- Difficulty in the Positive Phase:
- Latent Variable Models, generally, have intractable positive phase.
- Models with no latent variables or with few interactions between latent variables typically have a tractable positive phase.- Example - VAEs:
VAEs define a continuous distribution (over the data) with latent variable \(z\):$$p_{\theta}(x)=\int p_{\theta}(z) p_{\theta}(x \vert z) d z$$
which is intractable to compute for every \(z\).
Due to complicated interactions between latent variables, this integral requires exponential time to compute as it needs to be evaluated over all configurations of latent variable.
(all \(z_i\) variables are dependent on each other.)
- Example - VAEs:
Positive and Negative Phases:
The terms positive and negative do not refer to the sign of each term in the equation, but rather reflect their effect on the probability density defined by the model.
- The positive phase increases the probability of training data (by reducing the corresponding free energy)
- The negative phase decreases the probability of samples generated by the model.Monte Carlo Methods for Approximate LL Maximization:
To use MC methods for approximate learning, we need to rewrite the gradient of the partition function \(\nabla_{\boldsymbol{\theta}} \log Z\) as an expectation of the unnormalized probability \(\tilde{p}\):$$\nabla_{\boldsymbol{\theta}} \log Z=\mathbb{E}_{\mathbf{x} \sim p(\mathbf{x})} \nabla_{\boldsymbol{\theta}} \log \tilde{p}(\mathbf{x})$$
This identity is the basis for a variety of Monte Carlo methods for approximately maximizing the likelihood of models with intractable partition functions.
-
- Decomposing the gradient of \(\log Z\):
$$\begin{aligned} \nabla_{\boldsymbol{\theta}} \log Z =& \frac{\nabla_{\boldsymbol{\theta}} Z}{Z} \\=& \frac{\nabla_{\boldsymbol{\theta}} \sum_{\mathbf{x}} \tilde{p}(\mathbf{x})}{Z} \\=& \frac{\sum_{\mathbf{x}} \nabla_{\boldsymbol{\theta} \tilde{p}(\mathbf{x})}}{Z} \end{aligned}$$
- For models that guarantee \(p(\mathbf{x})>0\) for all \(\mathbf{x},\) we can substitute \(\exp (\log \tilde{p}(\mathbf{x}))\) for \(\tilde{p}(\mathbf{x})\):
$$\begin{aligned} \frac{\sum_{\mathbf{x}} \nabla_{\boldsymbol{\theta}} \exp (\log \tilde{p}(\mathbf{x}))}{Z} &= \frac{\sum_{\mathbf{x}} \exp (\log \tilde{p}(\mathbf{x})) \nabla_{\boldsymbol{\theta}} \log \tilde{p}(\mathbf{x})}{Z} \\ &=\frac{\sum_{\mathbf{x}} \tilde{p}(\mathbf{x}) \nabla_{\boldsymbol{\theta}} \log \tilde{p}(\mathbf{x})}{Z} \\ &=\sum_{\mathbf{x}} p(\mathbf{x}) \nabla_{\boldsymbol{\theta}} \log \tilde{p}(\mathbf{x}) \\ &= \mathbb{E}_{\mathbf{x} \sim p(\mathbf{x})} \nabla_{\boldsymbol{\theta}} \log \tilde{p}(\mathbf{x}) \end{aligned}$$
- Decomposing the gradient of \(\log Z\):
-
- We use Leibniz’s rule for diﬀerentiation under the integral sign to obtain the identity:
$$\nabla_{\boldsymbol{\theta}} \int \tilde{p}(\mathbf{x}) d \boldsymbol{x}=\int \nabla_{\boldsymbol{\theta}} \tilde{p}(\mathbf{x}) d \boldsymbol{x}$$
- Applicability - Measure Theory:
This identity is applicable only under certain regularity conditions on \(\tilde{p}\) and \(\nabla_{\boldsymbol{\theta}} \tilde{p}(\mathbf{x})\).
In measure theoretic terms, the conditions are:- The unnormalized distribution \(\tilde{p}\) must be a Lebesgue-integrable function of \(\boldsymbol{x} for every value of\)\boldsymbol{\theta}$$.
- The gradient \(\nabla_{\boldsymbol{\theta}} \tilde{p}(\mathbf{x})\) must exist for all \(\boldsymbol{\theta}\) and almost all \(\boldsymbol{x}\).
- There must exist an integrable function \(R(\boldsymbol{x})\) that bounds \(\nabla_{\boldsymbol{\theta}} \tilde{p}(\mathbf{x})\) in the sense that \(\max_{i}\left\vert\frac{\partial}{\partial \theta_{\theta}} \tilde{p}(\mathbf{x})\right\vert \leq R(\boldsymbol{x})\) for all \(\boldsymbol{\theta}\) and almost all \(\boldsymbol{x}\).
Fortunately, most machine learning models of interest have these properties..
- Applicability - Measure Theory:
- We use Leibniz’s rule for diﬀerentiation under the integral sign to obtain the identity:
Intuition:
The Monte Carlo approach to learning provides an intuitive framework in terms of the phases of the learning decomposition:- Positive Phase:
In the positive phase, we increase \(\log \tilde{p}(\mathbf{x})\) for \(\boldsymbol{x}\) drawn from the data.- Parametrize \(\log \tilde{p}\) in terms of an Energy Function:
We interpret the positive phase as pushing down on the energy of training examples.
- Parametrize \(\log \tilde{p}\) in terms of an Energy Function:
- Negative Phase:
In the negative phase, we decrease the partition function by decreasing \(\log \tilde{p}(\mathbf{x})\) drawn from the model distribution.- Parametrize \(\log \tilde{p}\) in terms of an Energy Function:
We interpret the negative phase as pushing up on the energy of samples drawn from the models.
- Parametrize \(\log \tilde{p}\) in terms of an Energy Function:
- Difficulty in the Negative Phase:
-
Stochastic Maximum Likelihood and Contrastive Divergence:
To approximately maximize the log likelihood, using the identity derived above, we need to use MCMC methods.Motivation - The Naive Approach:
The naive way to compute the identity above, is to approximate it by burning in a set of Markov Chains from a random initialization everytime the gradient is needed.
When learning is performed using stochastic gradient descent, this means the chains must be burned in once per gradient step.
The high cost of burning in the Markov chains in the inner loop makes this procedure computationally infeasible.Learning Intuition (from naive algorithm):
- We can view the MCMC approach to maximum likelihood as trying to achieve balance between two forces:
- One pushing up on the model distribution where the data occurs
Corresponds to maximizing \(\log \tilde{p}\). - Another pushing down on the model distribution where the model samples occur.
Corresponds to minimizing \(\log Z\).
- One pushing up on the model distribution where the data occurs
- There are several approximations to the negative phase.
Each of these approximations can be understood as making the negative phase computationally cheaper but also making it push down in the wrong locations. - Negative Phase Intuition:
- Because the negative phase involves drawing samples from the model’s distribution, we can think of it as finding points that the model believes in strongly.
- Because the negative phase acts to reduce the probability of those points, they are generally considered to represent the model’s incorrect beliefs about the world.
Referred to, in literature, as “hallucinations” or “fantasy particles”.-
- In fact, the negative phase has been proposed as a possible explanation for dreaming in humans and other animals (Crick and Mitchison, 1983), the idea being that the brain maintains a probabilistic model of the world and follows the gradient of \(\log \tilde{p}\) when experiencing real events while awake and follows the negative gradient of \(\log \tilde{p}\) to minimize \(\log Z\) while sleeping and experiencing events sampled from the current model. This view explains much of the language used to describe algorithms with a positive and a negative phase, but it has not been proved to be correct with neuroscientific experiments.
In machine learning models, it is usually necessary to use the positive and negative phase simultaneously, rather than in separate periods of wakefulness and REM sleep.
As we will see in section 19.5, other machine learning algorithms draw samples from the model distribution for other purposes, and such algorithms could also provide an account for the function of dream sleep.
- In fact, the negative phase has been proposed as a possible explanation for dreaming in humans and other animals (Crick and Mitchison, 1983), the idea being that the brain maintains a probabilistic model of the world and follows the gradient of \(\log \tilde{p}\) when experiencing real events while awake and follows the negative gradient of \(\log \tilde{p}\) to minimize \(\log Z\) while sleeping and experiencing events sampled from the current model. This view explains much of the language used to describe algorithms with a positive and a negative phase, but it has not been proved to be correct with neuroscientific experiments.
-
Summary:
The main cost of the naive MCMC algorithm is the cost of burning in the Markov chains from a random initialization at each step.
Contrastive Divergence:
One way to avoid the high cost in Naive MCMC, is to initialize the Markov chains from a distribution that is very close to the model distribution, so that the burn in operation does not take as many steps.
The Contrastive Divergence (CD) (or CD-\(k\) to indicate CD with \(k\) Gibbs steps) algorithm initializes the Markov chain at each step with samples from the data distribution (Hinton, 2000, 2010).
- Obtaining samples from the data distribution is free, because they are already available in the dataset.
- Initially, the data distribution is not close to the model distribution, so the negative phase is not very accurate.
- Fortunately, the positive phase can still accurately increase the model’s probability of the data.
- After the positive phase has had some time to act, the model distribution is closer to the data distribution, and the negative phase starts to become accurate.
Drawbacks:
- Spurious Modes:
Since CD is still an approximation to the correct negative phase, it results in spurious modes; i.e. fails to suppress regions of high probability that are far from actual training examples.
Spurious Modes: are those regions that have high probability under the model but low probability under the data-generating distribution.
- Modes in the distribution that are far from the data distribution will not be visited by Markov chains initialized at training points, unless \(k\) is very large. - CD as a Biased Estimator in RBMs and Boltzmann Machines:
- Carreira-Perpiñan and Hinton (2005) showed experimentally that the CD estimator is biased for RBMs and fully visible Boltzmann machines, in that it converges to different points than the maximum likelihood estimator.
- They argue that because the bias is small, CD could be used as an inexpensive way to initialize a model that could later be fine-tuned via more expensive MCMC methods.- Interpretation: Bengio and Delalleau (2009) show that CD can be interpreted as discarding the smallest terms of the correct MCMC update gradient, which explains the bias.
- Random Gradients:
Sutskever and Tieleman (2010) showed that the CD update direction is not the gradient of any function.
This allows for situations where CD could cycle forever, but in practice this is not a serious problem. - Difficulty for Deep Models:
- CD is useful for training shallow models like RBMs.
- The RBMs can be stacked to initialize deeper models like DBNs or DBMs.
- However, CD does NOT provide much help for training deeper models directly.
- This is because it is difficult to obtain samples of the hidden units given samples of the visible units.
- Since the hidden units are not included in the data, initializing from training points cannot solve the problem.
- Even if we initialize the visible units from the data, we will still need to burn in a Markov chain sampling from the distribution over the hidden units conditioned on those visible samples.
- This is because it is difficult to obtain samples of the hidden units given samples of the visible units.
Relation to Autoencoder Training:
- The CD algorithm can be thought of as penalizing the model for having a Markov chain that changes the input rapidly when the input comes from the data.
- This means training with CD somewhat resembles autoencoder training.
- Even though CD is more biased than some of the other training methods, it can be useful for pretraining shallow models that will later be stacked.- This is because the earliest models in the stack are encouraged to copy more information up to their latent variables, thereby making it available to the later models.
This should be thought of more as an often-exploitable side effect of CD training rather than a principled design advantage.
Stochastic Maximum Likelihood (SML) - Persistent Contrastive Divergence (PCD, PCD-\(k\)):
SML AKA PCD is a method that initializes the Markov Chains, in CD, at each gradient step with their states from the previous gradient step.
This strategy resolves many of the problems with CD.
Idea:- The basic idea of this approach is that, as long as the steps taken by the stochastic gradient algorithm are small, the model from the previous step will be similar to the model from the current step.
- It follows that the samples from the previous model’s distribution will be very close to being fair samples from the current model’s distribution, so a Markov chain initialized with these samples will not require much time to mix.
Advantages:
- SML is considerably more resistant to forming models with spurious modes than CD is:
Because each Markov chain is continually updated throughout the learning process, rather than restarted at each gradient step, the chains are free to wander far enough to find all the model’s modes. - SML is able to train deep models efficiently:
- SML provides an initialization point for both the hidden and the visible units:
Because it is possible to store the state of all the sampled variables, whether visible or latent. - CD is only able to provide an initialization for the visible units, and therefore requires burn-in for deep models.
- SML provides an initialization point for both the hidden and the visible units:
- Performance/Results - In-Practice:
Marlinet al. (2010) compared SML to many other criteria presented in this section. They found that:- SML results in the best test set log-likelihood for an RBM, and that
- if the RBM’s hidden units are used as features for an SVM classifier, SML results in the best classification accuracy.
Mixing Evaluation:
Sample Evaluation:
Care must be taken when evaluating the samples from a model trained with SML. It is necessary to draw the samples starting from a fresh Markov chain initialized from a random starting point after the model is done training. The samples present in the persistent negative chains used for training have been influenced by several recent versions of the model, and thus can make the model appear to have greater capacity than it actually does.Bias-Variance of CD and SML:
Berglund and Raiko (2013) performed experiments to examine the bias and variance in the estimate of the gradient provided by CD and SML:- CD proves to have lower variance than the estimator based on exact sampling.
The cause of CD’s low variance is its use of the same training points in both the positive and negative phase.
If the negative phase is initialized from different training points, the variance rises above that of the estimator based on exact sampling. - SML has higher variance.
Improving CD & SML:
- MCMC Algorithms:
All these methods based on using MCMC to draw samples from the model canin principle be used with almost any variant of MCMC. This means that techniques such as SML can be improved by using any of the enhanced MCMC techniques described in chapter 17, such as parallel tempering (Desjardins et al., 2010; Choet al., 2010). - Fast PCD (FPCD):
Another approach to accelerating mixing during learning relies not on changing the Monte Carlo sampling technology but rather on changing the parametrization of the model and the cost function.
FPCD is such a method that involves replacing the parameters \(\boldsymbol{\theta}\) of a traditional model with an expression:$$\boldsymbol{\theta}=\boldsymbol{\theta}^{(\mathrm{slow})}+\boldsymbol{\theta}^{(\mathrm{fast})}$$
Training with Positive Phase Estimators (bound-based, variational methods):
One key benefit to the MCMC-based methods described in this section is that they provide an estimate of the gradient of \(\log Z,\) and thus we can essentially decompose the problem into the \(\log \tilde{p}\) contribution and the \(\log Z\) contribution.
We can then use any other method to tackle \(\log \tilde{p}(\mathbf{x})\) and just add our negative phase gradient onto the other method’s gradient.
In particular, this means that our positive phase can make use of methods that provide only a lower bound on \(\tilde{p}\).
Most of the other methods of dealing with \(\log Z\) presented in this chapter are incompatible with bound-based positive phase methods.
- We can view the MCMC approach to maximum likelihood as trying to achieve balance between two forces:
-
Pseudolikelihood:
Motivation:
We can sidestep the issue of approximating the intractable partition function by training the model without computing it at all.Idea:
Most of these approaches are based on the observation that it is easy to compute ratios of probabilities in an undirected model.
This is because the partition function appears in both the numerator and the denominator of the ratio and cancels out:$$\frac{p(\mathbf{x})}{p(\mathbf{y})}=\frac{\frac{1}{Z} \tilde{p}(\mathbf{x})}{\frac{1}{Z} \tilde{p}(\mathbf{y})}=\frac{\tilde{p}(\mathbf{x})}{\tilde{p}(\mathbf{y})}$$
Pseudolikelihood:
The Pseudolikelihood is an objective function, based on predicting the value of feature \(x _ {i}\) given all the other features \(\boldsymbol{x}_ {-i}\):$$\sum_{i=1}^{n} \log p\left(x_{i} \vert \boldsymbol{x}_ {-i}\right)$$
Derivation:
- The pseudolikelihood is based on the observation that conditional probabilities take this ratio-based form and thus can be computed without knowledge of the partition function.
Computational Cost:
- If each random variable has \(k\) different values, this requires only \(k \times n\) evaluations of \(\tilde{p}\) to compute,
- as opposed to the \(k^{n}\) evaluations needed to compute the partition function.
Justification:
Estimation by maximizing the pseudolikelihood is asymptotically consistent (Mase, 1995).
When the datasets do not approach the large sample limit, pseudolikelihood may display different behavior from the maximum likelihood estimator.Generalized Pseudolikelihood Estimator:
The Generalized Pseudolikelihood Estimator gives us a way to trade-off computational complexity for deviation from maximum likelihood behavior.
The GPE objective function:$$\sum_{i=1}^{m} \log p\left(\mathbf{x}_{\mathbb{S}^{(i)}} \vert \mathbf{x}_{-\mathbb{S}^{(i)}}\right)$$
Complexity-Consistency Tradeoff:
It uses \(m\) different sets \(\mathbb{S}^{(i)}, i=1, \ldots, m\) of indices of variables that appear together on the left side of the conditioning bar:- In the extreme case of \(m=1\) and \(\mathbb{S}^{(1)}=1, \ldots, n,\) the generalized pseudolikelihood recovers the log-likelihood.
- In the extreme case of \(m=n\) and \(\mathbb{S}^{(i)}=\{i\},\) the generalized pseudolikelihood recovers the pseudolikelihood.
Performance:
The performance of pseudolikelihood-based approaches depends largely on how the model will be used:- Pseudolikelihood tends to perform poorly on tasks that require a good model of the full joint \(p(\mathbf{x}),\) such as density estimation and sampling.
- It can perform better than maximum likelihood for tasks that require only the conditional distributions used during training, such as filling in small amounts of missing values.
- Generalized pseudolikelihood techniques are especially powerful if the data has regular structure that allows the \(\mathbb{S}\) index sets to be designed to capture the most important correlations while leaving out groups of variables that have only negligible correlation.
For example, in natural images, pixels that are widely separated in space also have weak correlation, so the generalized pseudolikelihood can be applied with each \(\mathbb{S}\) set being a small, spatially localized window.
Drawbacks - Training with Lower-Bound Maximization Methods:
- One weakness of the pseudolikelihood estimator is that it cannot be used with other approximations that provide only a lower bound on \(\tilde{p}(\mathbf{x}),\) e.g. variational inference.
- This is because \(\tilde{p}\) appears in the denominator.
A lower bound on the denominator provides only an upper bound on the expression as a whole, and there is no benefit to maximizing an upper bound.
This makes it difficult to apply pseudolikelihood approaches to deep models such as deep Boltzmann machines, since variational methods are one of the dominant approaches to approximately marginalizing out the many layers of hidden variables that interact with each other.
- This is because \(\tilde{p}\) appears in the denominator.
- Nonetheless, pseudolikelihood is still useful for deep learning, because it can be used to train single-layer models or deep models using approximate inference methods that are not based on lower bounds.
Pseudolikelihood vs SML/PCD - Computational Cost:
Pseudolikelihood has a much greater cost per gradient step than SML, due to its explicit computation of all the conditionals.
But generalized pseudolikelihood and similar criteria can still perform well if only one randomly selected conditional is computed per example (Goodfellow et al., 2013b), thereby bringing the computational cost down to match that of SML.Relation to the Negative Phase:
Though the pseudolikelihood estimator does not explicitly minimize \(\log Z\), it can still be thought of as having something resembling a negative phase.
The denominators of each conditional distribution result in the learning algorithm suppressing the probability of all states that have only one variable differing from a training example.Asymptotic Efficiency:
See Marlin and de Freitas (2011) for a theoretical analysis of the asymptotic efficiency of pseudolikelihood.
- Score-Matching and Ratio-Matching:
Denoising Score Matching:
- Noise-Contrastive Estimation (NCE):
Noise-Contrastive Estimation (NCE) is a method for computing the partition function as a learned parameter in the model; where the probability distribution estimated by the model is represented explicitly as:$$\log p_{\text {model }}(\mathbf{x})=\log \tilde{p}_ {\text {model}}(\mathbf{x} ; \boldsymbol{\theta})+c$$
where \(c\) is explicitly introduced as an approximation of \(-\log Z(\boldsymbol{\theta})\).
Rather than estimating only \(\boldsymbol{\theta}\), the noise contrastive estimation procedure treats \(c\) as just another parameter and estimates \(\boldsymbol{\theta}\) and \(c\) simultaneously, using the same algorithm for both.
The resulting \(\log p_{\text {model}}(\mathbf{x})\) thus may not correspond exactly to a valid probability distribution, but it will become closer and closer to being valid as the estimate of \(c\) improves.1Derivation:
- Problem with Maximum Likelihood Criterion:
Such an approach would not be possible using maximum likelihood as the criterion for the estimator.
The maximum likelihood criterion would choose to set \(c\) arbitrarily high, rather than setting \(c\) to create a valid probability distribution. -
Solution - New Estimator of the original problem:
NCE works by reducing the unsupervised learning problem of estimating \(p(\mathrm{x})\) to that of learning a probabilistic binary classifier in which one of the categories corresponds to the data generated by the model.
This supervised learning problem is constructed in such a way that maximum likelihood estimation defines an asymptotically consistent estimator of the original problem.Specifically,
- Posit two distributions: the model, and a noise distribution.
- The Noise Distribution \(p_{\text{noise}}(\mathbf{x})\):
We introduce a new distribution \(p_{\text{noise}}(\mathbf{x})\) over the noise.
The noise distribution should be tractable to evaluate and to sample from.
- The Noise Distribution \(p_{\text{noise}}(\mathbf{x})\):
- Construct a new joint model over both \(\boldsymbol{x}\) and a binary variable \(y\):
- We can now construct a model over both \(\mathbf{x}\) and a new, binary class variable \(y\). In the new joint model, we specify that
(1) \(p_{\mathrm{joint}}(y=1)=\frac{1}{2}\)
(2) \(p_{\mathrm{joint}}(\mathbf{x} \vert y=1)=p_{\mathrm{model}}(\mathbf{x})\)
(3) \(p_{\mathrm{joint}}(\mathbf{x} \vert y=0)=p_{\mathrm{noise}}(\mathbf{x})\)
In other words, \(y\) is a switch variable that determines whether we will generate \(\mathbf{x}\) from the model or from the noise distribution. - Equivalently, We can construct a similar joint model of training data.
Formally, (1) \(p_{\text {train}}(y=1)=\frac{1}{2}\)
(2) \(p_{\text {train}}(\mathbf{x} \vert y=1)=p_{\text {data }}(\mathbf{x}),\)
(3) \(p_{\text {train}}(\mathbf{x} \vert y=0)=p_{\text {noise}}(\mathbf{x})\)
In this case, the switch variable determines whether we draw \(\mathbf{x}\) from the data or from the noise distribution.
- We can now construct a model over both \(\mathbf{x}\) and a new, binary class variable \(y\). In the new joint model, we specify that
- Construct the new supervised Binary Classification Task - fitting \(p_{\text {joint}}\) to \(p_{\text {train}}\):
We can now just use standard maximum likelihood learning on the supervised learning problem of fitting \(p_{\text {joint}}\) to \(p_{\text {train}}\), by swapping \(p_{\text {model}}\) with \(p_{\text {joint}}\):$$\boldsymbol{\theta}, c=\underset{\boldsymbol{\theta}, c}{\arg \max } \mathbb{E}_{\mathbf{x}, \mathbf{y} \sim p_{\text {train}}} \log p_{\text {joint}}(y \vert \mathbf{x})$$
- Expanding \(p_{\text{joint}}(y \vert x)\):
The distribution \(p_{\text{joint}}\) is essentially a logistic regression model applied to the difference in log probabilities of the model and the noise distribution:$$\begin{aligned} p_{\text {joint}}(y=1 \vert \mathbf{x}) &= \frac{p_{\text {model }}(\mathbf{x})}{p_{\text {model }}(\mathbf{x})+p_{\text {noise}}(\mathbf{x})} \\ &= \frac{1}{1+\frac{p_{\text {noise}}(\mathbf{x})}{p_{\text {model}} (\mathbf{x})}} \\ &= \frac{1}{1+\exp \left(\log \frac{p_{\text {noise}}(\mathbf{x})}{p_{\text {model }}(\mathbf{x})}\right)} \\ &= \sigma\left(-\log \frac{p_{\text {noise}}(\mathbf{x})}{p_{\text {model }}(\mathbf{x})}\right) \\ &= \sigma\left(\log p_{\text {model }}(\mathbf{x})-\log p_{\text {noise}}(\mathbf{x})\right) \end{aligned}$$
- Expanding \(p_{\text{joint}}(y \vert x)\):
- Posit two distributions: the model, and a noise distribution.
Summary:
- Posit two distributions: the model, and a noise distribution.
- Given a data point, predict from which distribution this point was generated.
NCE is thus simple to apply as long as \(\log \tilde{p}_{\text {model}}\) is easy to back-propagate through, and, as specified above, \(p_{\text {noise}}\) is easy to evaluate (in order to evaluate \(p_{\text {joint}}\) and sample from (to generate the training data).
The Noise Distribution:
- Practical Implications and Complexity:
- Better Distributions - Parametric \(p_{\text{noise}}\):
The noise distribution is generally non-parametric.
However, there is nothing stopping us from evolving this distribution and giving it trainable parameters, then updating these parameters such that it generates increasingly “optimal” samples.- Optimality:
Of course, we would have to design what “optimal” means.- Adversarial Contrastive Estimation:
One interesting approach is called Adversarial Contrastive Estimation, wherein the authors adapt the noise distribution to generate increasingly “harder negative examples, which forces the main model to learn a better representation of the data.
- Adversarial Contrastive Estimation:
- Optimality:
Weaknesses/Drawbacks:
- Problems with Many RVs:
When NCE is applied to problems with many random variables, it becomes less efficient.- The logistic regression classifier can reject a noise sample by identifying any one variable whose value is unlikely.
This means that learning slows down greatly after \(p_{\text {model}}\) has learned the basic marginal statistics. - Imagine learning a model of images of faces, using unstructured Gaussian noise as \(p_{\text {noise}}\).
If \(p_{\text {model }}\) learns about eyes, it can reject almost all unstructured noise samples without having learned anything about other facial features, such as mouths.
- The logistic regression classifier can reject a noise sample by identifying any one variable whose value is unlikely.
- Noise Distribution Complexity:
The constraint that \(p_{\text {noise}}\) must be easy to evaluate and easy to sample from can be overly restrictive:- For our training data, we require the ability to sample from our noise distribution..
- For our target, we require the ability to compute the likelihood of some data under our noise distribution.
When \(p_{\text {noise}}\) is simple, most samples are likely to be too obviously distinct from the data to force \(p_{\text {model}}\) to improve noticeably.
- Training with Lower-Bound Maximizing Methods:
NCE does not work if only a lower bound on \(\tilde{p}\) is available.
Such a lower bound could be used to construct a lower bound on \(p_{\text {joint}}(y=1 \vert \mathbf{x}),\) but it can only be used to construct an upper bound on \(p_{\text {joint}}(y=0 \vert \mathbf{x}),\) which appears in half the terms of the NCE objective.
Likewise, a lower bound on \(p_{\text {noise}}\) is not useful, because it provides only an upper bound on \(p_{\text {joint}}(y=1 \vert \mathbf{x})\).
Self-Contrastive Estimation:
When the model distribution is copied to define a new noise distribution before each gradient step, NCE defines a procedure called self-contrastive estimation, whose expected gradient is equivalent to the expected gradient of maximum likelihood (Goodfellow, 2014).
Interpretation:- Self-Contrastive Estimation:
The special case of NCE where the noise samples are those generated by the model suggests that maximum likelihood can be interpreted as a procedure that forces a model to constantly learn to distinguish reality from its own evolving beliefs, - NCE:
However, NCE achieves some reduced computational cost by only forcing the model to distinguish reality from a fixed baseline (noise model).
Connection to Importance Sampling:
Jozefowicz et al. (2016) show that NCE and IS are not only similar as both are sampling-based approaches, but are strongly connected.
While NCE uses a binary classification task, they show that IS can be described similarly using a surrogate loss function: Instead of performing binary classification with a logistic loss function like NCE, IS then optimises a multi-class classification problem with a softmax and cross-entropy loss function.
They observe that as IS performs multi-class classification, it may be a better choice for language modeling, as the loss leads to tied updates between the data and noise samples rather than independent updates as with NCE.
Indeed, Jozefowicz et al. (2016) use IS for language modeling and obtain state-of-the-art performance on the 1B Word benchmark.Relation to Generative Adversarial Networks (GANs):
Noise contrastive estimation is based on the idea that a good generative model should be able to distinguish data from noise.
A closely related idea is that a good generative model should be able to generate samples that no classifier can distinguish from data.
This idea yields generative adversarial networks.Self-Normalization:
Mnih and Teh (2012) and Vaswani et al. (2013) fix \(c = 1\).
They report does not affect the model’s performance.
This assumption has the nice side-effect of reducing the model’s parameters, while ensuring that the model self-normalises by not depending on the explicit normalisation in \(c\).
Indeed, Zoph et al. (2016) find that even when learned, \(c\) is close to \(1\) and has low variance.
- Problem with Maximum Likelihood Criterion:
-
Negative Sampling:
Negative Sampling (NEG) can be seen as an approximation to NCE.
As we have mentioned above, NCE can be shown to approximate the loss of \(\log p_{\text{model}}\) as the number of samples \(k\) increase.
NEG simplifies NCE and does away with this guarantee, as the objective of NEG is to learn high-quality word representations rather than achieving low perplexity on a test set, as is the goal in language modeling.The key difference to NCE is that NEG only approximates this probability by making it as easy to compute as possible.
It simplifies NCE as follows:- Considers noise distributions whose likelihood we cannot evaluate
- To accommodate, it simply set the most expensive term \(p_{\text {noise}}(x)=1\)
Equivalently, \(k\:p_{\text {noise}}(x)=1\)-
- Thus, Expanding \(p_{\text{joint}}(y \vert x)\):
The distribution \(p_{\text{joint}}\) is essentially a logistic regression model applied to the difference in log probabilities of the model and the noise distribution:$$\begin{aligned} p_{\text {joint}}(y=1 \vert \mathbf{x}) &= \frac{p_{\text {model }}(\mathbf{x})}{p_{\text {model }}(\mathbf{x})+p_{\text {noise}}(\mathbf{x})} \\ &= \frac{1}{1+\frac{p_{\text {noise}}(\mathbf{x})}{p_{\text {model}} (\mathbf{x})}} \\ &= \frac{1}{1+\exp \left(\log \frac{p_{\text {noise}}(\mathbf{x})}{p_{\text {model }}(\mathbf{x})}\right)} \\ &= \sigma\left(-\log \frac{p_{\text {noise}}(\mathbf{x})}{p_{\text {model }}(\mathbf{x})}\right) \\ &= \sigma\left(-\log \frac{1}{p_{\text {model}}(\mathbf{x})}\right) \\ &= \sigma\left(\log p_{\text {model}}(\mathbf{x})\right) \end{aligned}$$
- Thus, Expanding \(p_{\text{joint}}(y \vert x)\):
-
Equivalence with NCE:
- \(k p_{\text {noise}}=1\) is exactly then true, when (discrete):
- \(k=\vert X\vert\) and
- \(p_{\text {noise}}\) is a uniform distribution.
In this case, NEG is equivalent to NCE.
- The reason we set \(k p_{\text {noise}}=1\) and not to some other constant can be seen by rewriting the equation, as \(P(y=1 \vert \mathbf{x})\) can simplify the sigmoid function.
- In all other cases, NEG only approximates NCE, which means that it will not directly optimize the likelihood \(\log p_{\text {model}}(\mathbf{x})\).
- Asymptotic Consistency:
Since NEG only approximates NCE, it lacks any asymptotic consistency guarantees.
Application - Language Modeling and Word Embeddings:
NEG only approximates NCE, which means that it will not directly optimise the likelihood of correct words, which is key for language modelling. While NEG may thus be useful for learning word embeddings, its lack of asymptotic consistency guarantees makes it inappropriate for language modelling.
- Self-Normalization:
Remember from NCE that we decomposed the log likelihood of the model as:$$\log p_{\text {model }}(\mathbf{x})=\log \tilde{p}_ {\text {model}}(\mathbf{x} ; \boldsymbol{\theta})+c$$
where \(c\) is explicitly introduced as an approximation of \(-\log Z(\boldsymbol{\theta})\).
If we are able to constrain our model so that it sets \(c=0\) (i.e. \(e^c = 1\)), then we can avoid computing the normalization in \(c\) altogether.
Devlin et al. (2014) thus propose to add a squared error penalty term to the loss function that encourages the model to keep \(c\) as close as possible to \(0\):$$\tilde{J} = J + \lambda (c-0)^{2}$$
where \(\lambda\) allows us to trade-off between model accuracy and mean self-normalisation.
At inference time, we set
$$p_{\text {model }}(\mathbf{x})=\dfrac{\tilde{p}_ {\text {model}}(\mathbf{x} ; \boldsymbol{\theta})}{Z(\boldsymbol{\theta})} \approx \dfrac{\tilde{p}_ {\text {model}}(\mathbf{x} ; \boldsymbol{\theta})}{1} = \tilde{p}_ {\text {model}}(\mathbf{x} ; \boldsymbol{\theta})$$
Results - MT:
They report that self-normalisation achieves a speed-up factor of about 15, while only resulting in a small degradation of BLEU scores compared to a regular non-self-normalizing neural language model.Notes:
Estimating the Partition Function
- Estimating the Partition Function:
- Annealed Importance Sampling (AIS):
- Bridge Sampling:
- Linked Importance Sampling (LIS):
- Estimating the Partition Function while Training:
-
NCE is also applicable to problems with a tractable partition function, where there is no need to introduce the extra parameter \(c\). However, it has generated the most interest as a means of estimating models with difficult partition functions. ↩


Nice collection of cheat sheets for many languages with links to documentation.
Nice collection of cheat sheets for many languages with links to documentation.
Online book with interactive coding examples
This is a great intermediate level JS video course. You will benefit most if you have already gone through some kind of beginner JS course. The full course is available on Udemy
Michael Chan (videos in reverse order, start at bottom right)
tutorial series on youtube very easy to follow
This site compares a long list of code snippets showing the jQuery and vanilla JavaScript way to do it.
Welcome to your GitHub training. The course you are about to experience will interact with your GitHub account to provide real time feedback on activities.
command line tutorial This is the one on FCC. It uses the command line but gives you step by step instructions
the simple guide quick run-through of the most common commands
If you've started using Git but need a nice reference, this pretty nice. Clean design and to the point.
Have you ever wondered how Facebook is able to automatically display your Instagram photos? How about how Evernote syncs notes between your computer and smartphone? If so, then it’s time to get excited!
How to create a custom WordPress theme from scratch(22-video series)
Free Microsoft Training Delivered by Experts. There's so much here you should just use the search.
A collection of awesome React tools, resources, videos and shiny things.
Huge list of node tutorials and more
This is a great intermediate level JS video course. You will benefit most if you have already gone through some kind of beginner JS course. (Regularly $149, this course goes on sale all the time. Don't pay full price for it. You should be able to get it for $20 or less. And you can watch the first 3.5 hrs free on YouTube. youtube link listed above)
In this WordPress CMS tutorial we'll learn how to download and install WordPress on your computer to start creating WordPress sites locally.


Being fit is a neccesity for strong health and it doesn’t suck, get back into it today. Befit has got you covered for your fitness
Befit is dedicated to her customers to ensure their fitness and their strong health. Being fit doesn’t suck, instead it gives one a stronger lifespan and health. Get back into the business of befit. Join our community and coaches in the mission to help bring you back to a healthier lifestyle.
Befit is dedicated to her customers to ensure their fitness and ong health. Being fit doesn’t suck, instead it gives one a stronger lifespan and health. Get back into the business of befit. Join our community and coaches in the mission to help bring you back to a healthier lifestyle.
Befit is dedicated to her customers to ensure their fitness and their strong health. Being fit doesn’t suck, instead it gives one a stronger lifespan and health. Get back into the business of befit. Join our community and coaches in the mission to hel
Befit is dedicated to her customers to ensure their fitness and their strong health. Being fit doesn’t suck, instead it gives one a stronger lifespan and health. Get back into the business of befit. Join our community and coaches in the mission to hel
Befit is dedicated to her customers to ensure their fitness and their strong health. Being fit doesn’t suck, instead it gives one a stronger lifespan and health. Get back into the business of befit. Join our community and coaches in the mission to help bring you back to a healthier lifestyle.
Consultation
Best Trainers
1248 Working Hours
600 Success Stories
1500 Happy Clients


Description
When people don’t express themselves, they die one piece at a time. And what’s a better way
to express than to Roar, Roar so high that the world is forced to sit and take notice!
Nukkad Natak serves as an excellent platform to express one’s views and is the most energetic
event at VIVACITY. This year, the rules for the competition are as follows:
JUDGING CRITERIA:
1. Loudness: Voice and impact on the audience.
2. Constraint: Uniqueness, innovation and improvisation in terms of executing the
transformation.
3. Time Limit: Exceeding the time limit will result in the deduction of marks.
4. Team: Music and narration, coordination, relevance to the theme and overall performance.
5. Use of props.
6. Crowd Interaction: Participation with crowd in the play.
RULES:
1. Prior online registration is required.
2. It is a team event. Team size limit to 22 members, including performers, musicians,
drummers etc.
3. Each member of the team is required to carry their ID cards.
4. Theme selection is open. All the teams can have a theme of their own choice.
5. Each team will be allotted 30 minutes for their performances (including stage setup).
Points will be deducted for exceeding the time limit.
6. Props should be arranged by the teams on their own. Use of fire or any hazardous material is strictly prohibited.
7. Vulgarity of any kind would lead to disqualification of team from the event. Hence if the team
feels that any kind of act, expression or dialogue design can be deemed as vulgar, it is
strongly advised to consult the organizers than performing it directly on stage.
8. Use of electronic musical instruments is not allowed.
9. The organizers take the liberty to check on any sort of misconduct during the stay of the
teams and would take necessary actions whenever required. Please maintain the integrity of
the competition.
10. The decision of the judges and the organizing committee will be final and binding.


| 035:011 Khan | : |
And Allah did create you (Adam) from dust, then from Nutfah (male and female discharge semen drops i.e. Adam's offspring), then He made you pairs (male and female). And no female conceives or gives birth, but with His Knowledge. And no aged man is granted a length of life, nor is a part cut off from his life (or another man's life), but is in a Book (Al-Lauh Al-Mahfuz) Surely, that is easy for Allah. |
| 035:011 Maulana | : |
And Allah created you from dust, then from the life-germ, then He made you pairs. And no females bears, nor brings forth, except with His knowledge. And no one living long is granted a long life, nor is aught diminished of one's life, but it is all in a book. Surely this is easy to Allah. |
| 035:011 Pickthal | : |
Allah created you from dust, then from a little fluid, then He made you pairs (the male and female). No female beareth or bringeth forth save with His knowledge. And no-one groweth old who groweth old, nor is aught lessened of his life, but it is recorded in a Book, Lo! that is easy for Allah. |
| 035:011 Rashad | : |
GOD created you from dust, then from a tiny drop, then He causes you to reproduce through your spouses. No female becomes pregnant, nor gives birth, without His knowledge. No one survives for a long life, and no one's life is snapped short, except in accordance with a pre-existing record. This is easy for GOD. |
| 035:011 Sarwar | : |
God created you from clay which He then turned into a living germ and made you into pairs. No female conceives or delivers without His knowledge. No one grows older nor can anything be reduced from one's life without having its record in the Book. This is not at all difficult for God. |
| 035:011 Shakir | : |
And Allah created you of dust, then of the life-germ, then He made you pairs; and no female bears, nor does she bring forth, except with His knowledge; and no one whose life is lengthened has his life lengthened, nor is aught diminished of one's life, but it is all in a book; surely this is easy to Allah. |
| 035:011 Sherali | : |
And ALLAH created you from dust, then from a sperm drop, then HE made you pairs. And no female conceives, nor does she bring forth a child save with HIS knowledge. And no one is granted long life who is granted long life, nor is anything diminished of his life, but it is all recorded in a Book. That surely is easy for ALLAH. |
| 035:011 Yusufali | : |
And Allah did create you from dust; then from a sperm-drop; then He made you in pairs. And no female conceives, or lays down (her load), but with His knowledge.  All this is easy to Allah. |
Concepts/Themes Covered: أركان الإسلام > التوحيد > توحيد الله تعالي > علمه جلّ شأنه Pillars of Islam> Islamic> Oneness of Allah> His Glory's Knowledge أركان الإسلام > التوحيد > توحيد الله تعالي > وحدانيته Pillars of Islam> Islamic> Oneness of Allah> Singularity and Being Unique الإيمان > الغيب > الغيب النفسي > القضاء والقدر Faith> The Unseen World> The Psychological Secret> Fate and Divine Decree الإنسان والعلاقات الاجتماعية > الإنسان > خلقه Man and The Social Relations>Man>Being Created الإنسان والعلاقات الاجتماعية > الإنسان > شرفه ودنوه Man and The Social Relations>Man>When Honourable and When Unhonourable الإنسان والعلاقات الاجتماعية > الإنسان > طول عمره يضعفه ويعجزه Man and The Social Relations>Man>His Long Life Weakens him and Making him Incapable الإنسان والعلاقات الاجتماعية > الرجل والمرأة Man and The Social Relations>Man and Woman الإنسان والعلاقات الاجتماعية > المجتمعات > خلقهم من نفس واحدة Man and The Social Relations>The Societies>Their Creation from One Self الإنسان والعلاقات الاجتماعية > النساء > المرأة Man and The Social Relations>Women>The Woman العلوم والفنون > الحقائق العلمية و الإشاة إلى وقائع أيدتهاالإكتشافات العلمية > الإنسان وخلقه Science and Art> The Scientific Facts and the Indication to Facts which have been supported by the Scientific Discoveries>Man and his Creation
This tool uses eight English parallel translations for the Holy Quran imported from Mr. Clay Smith. You can also visit Qurany Concepts for browsing the Quran themes or topics. If you want to search the Holy Quran for keywords, please visit Qurany Keywords.


class
Spritean ASGE supported sprite.
Contents
Used to render and load textures. Includes the ability to resize, scale, flip, recolour etc. 2D games are made up of these.
Public types
Constructors, destructors, conversion operators
- ~Sprite() defaulted virtual
Public functions
- auto loadTexture(const std::string&) -> bool pure virtual
- auto getTexture() const -> const Texture2D* pure virtual
- auto xPos() const -> float noexcept
- void xPos(float x) noexcept
- auto yPos() const -> float noexcept
- void yPos(float y) noexcept
- auto width() const -> float noexcept
- void width(float width) noexcept
- auto height() const -> float noexcept
- void height(float height) noexcept
- void dimensions(float& width, float& height) const noexcept
- auto rotationInRadians() const -> float noexcept
- void rotationInRadians(float rotation_radians)
- auto scale() const -> float noexcept
- void scale(float scale_value) noexcept
- auto colour() const -> Colour noexcept
-
void colour(ASGE::
Colour sprite_colour) noexcept - auto isFlippedOnX() const -> bool noexcept
- auto isFlippedOnY() const -> bool noexcept
- void setFlipFlags(FlipFlags flags) noexcept
- void opacity(float alpha) noexcept
- auto opacity() const -> float noexcept
- auto srcRect() -> float* noexcept
- auto srcRect() const -> const float* noexcept
Protected variables
- float dims
- float src_rect
- float position
- float angle
- float scale_factor
- float alpha
- FlipFlags flip_flags
- Colour tint
Enum documentation
enum ASGE:: Sprite:: FlipFlags
flags to control the flipping of the sprite
Used to flip the texture's UV coordinates. The flip is performed on the CPU when calculating the UVs.
| Enumerators | |
|---|---|
| NORMAL |
texture is not flipped. |
| FLIP_X |
texture is flipped on the x axis. |
| FLIP_Y |
texture is flipped on the y axis. |
| FLIP_BOTH |
texture is flipped on both axis. |
Function documentation
ASGE:: Sprite:: ~Sprite() virtual defaulted
Default destructor. The destructor does not free the memory used on the GPU. This is handled inside the platform's specific implementation.
bool ASGE:: Sprite:: loadTexture(const std::string&) pure virtual
| Returns | The result of the load operation. |
|---|
Pure virtual function for loading texture on to the GPU. This is handled inside the platform's specific implementation.**
const Texture2D* ASGE:: Sprite:: getTexture() const pure virtual
| Returns | The result of the load operation. |
|---|
Pure virtual function for retrieving the loaded texture. This is handled inside the platform's specific implementation.**
float ASGE:: Sprite:: xPos() const noexcept
| Returns | The position on the x axis |
|---|
Returns the sprite's position on the X axis. Sprite positions are described using Cartesian Coordinates. The horizontal position in this engine is labelled X and runs from left to right. It is mapped directly to screen space.
void ASGE:: Sprite:: xPos(float x) noexcept
| Parameters | |
|---|---|
| x | The position on the x axis |
Updates the sprite's position on the X axis. Sprite positions are described using Cartesian Coordinates. The horizontal position in this engine is labelled X and runs from left to right. It is mapped directly to screen
float ASGE:: Sprite:: yPos() const noexcept
| Returns | The new position on the y axis |
|---|
Returns the sprite's position on the Y axis. Sprite positions are described using Cartesian Coordinates. The vertical position in this engine is labelled y and runs from top to bottom. It is mapped directly to screen space.
void ASGE:: Sprite:: yPos(float y) noexcept
| Parameters | |
|---|---|
| y | The new position on the y axis |
Returns the sprite's position on the Y axis. Sprite positions are described using Cartesian Coordinates. The vertical position in this engine is labelled y and runs from top to bottom. It is mapped directly to screen space.
float ASGE:: Sprite:: width() const noexcept
| Returns | The sprites width. |
|---|
Gets the width of the sprite. Width is used to determine how wide/long a sprite will be when it's rendered to the screen. It is mapped using pixels.
void ASGE:: Sprite:: width(float width) noexcept
| Parameters | |
|---|---|
| width in | The sprites width. |
Sets the width of the sprite. Width is used to determine how wide/long a sprite will be when it's rendered to the screen. It is mapped using pixels.
float ASGE:: Sprite:: height() const noexcept
| Returns | The sprites height. |
|---|
Gets the height of the sprite. Height is used to determine how tall a sprite will be when it's rendered to the screen. It is mapped using pixels.
void ASGE:: Sprite:: height(float height) noexcept
| Parameters | |
|---|---|
| height in | The sprites height. |
Sets the height of the sprite. Height is used to determine how tall a sprite will be when it's rendered to the screen. It is mapped using pixels.
void ASGE:: Sprite:: dimensions(float& width,
float& height) const noexcept
| Parameters | |
|---|---|
| width | The sprites width. |
| height | The sprites height. |
Gets the height and the width of the sprite. Updates the width and the height of two parameters passed in to the function. It does not return a value. The width and height is found in the resultant parameters.
float ASGE:: Sprite:: rotationInRadians() const noexcept
| Returns | The sprites angle of rotation in radians. |
|---|
Gets the rotation of the sprite in radians. The angle of rotation is used to rotate the sprite around it's origin. The origin defaults to the mid-point of the sprite. The angle of rotation is expressed in radians.
void ASGE:: Sprite:: rotationInRadians(float rotation_radians)
| Parameters | |
|---|---|
| rotation_radians | The angle of rotation in radians. |
Sets the rotation of the sprite in radians. The angle of rotation is used to rotate the sprite around it's origin. The origin defaults to the mid-point of the sprite. The angle of rotation is expressed in radians.
float ASGE:: Sprite:: scale() const noexcept
| Returns | The scale factor. |
|---|
Gets the scale of the sprite. The scale can be adjusted to automatically make the sprite bigger or smaller than it's original size, allowing for on the fly alterations withouth having to edit the sprite's true size. The scaling operation is performed in both axis.
void ASGE:: Sprite:: scale(float scale_value) noexcept
| Parameters | |
|---|---|
| scale_value | The scale factor. |
Sets the scale of the sprite. The scale can be adjusted to automatically make the sprite bigger or smaller than it's original size, allowing for on the fly alterations withouth having to edit the sprite's true size. The scaling operation is performed in both axis.
void ASGE:: Sprite:: colour(ASGE:: Colour sprite_colour) noexcept
| Parameters | |
|---|---|
| sprite_colour | The sprite's colour. |
Sets the colour of the sprite. The colour of the sprite can be influenced by providing a tint.
bool ASGE:: Sprite:: isFlippedOnX() const noexcept
| Returns | If the texture is flipped on the X axis. |
|---|
Checks to see if the texture is flipped on the X axis. It is possible to flip the UV coordinates used on the sprite's texture. This will check to see if the X axis is flipped.
bool ASGE:: Sprite:: isFlippedOnY() const noexcept
| Returns | If the texture is flipped on the Y axis. |
|---|
Checks to see if the texture is flipped on the Y axis. It is possible to flip the UV coordinates used on the sprite's texture. This will check to see if the Y axis is flipped.
void ASGE:: Sprite:: setFlipFlags(FlipFlags flags) noexcept
| Parameters | |
|---|---|
| flags | The flip flags to apply to the texture. |
Sets the flip state of the texture. It is possible to flip the UV coordinates used on the sprite's texture. This function allows setting of the flip flags. These are then used to control shader's UV mappings.
void ASGE:: Sprite:: opacity(float alpha) noexcept
| Parameters | |
|---|---|
| alpha in | The new value for the alpha channel. |
Sets the opacity of the sprite. This function let's you control the opacity of the sprite by modifying the alpha channel during the render cycle. 1.0f is opaque and 0.0f is completely transparent.
float ASGE:: Sprite:: opacity() const noexcept
| Returns | The value for the alpha channel. |
|---|
Retrieves the opacity of the sprite. This function let's you obtain the opacity of the sprite. The opacity is controlled by using the alpha channel. 1.0f is opaque and 0.0f is completely transparent.
float* ASGE:: Sprite:: srcRect() noexcept
| Returns | The rectangle used for the UV source. |
|---|
Gets the source rectangle used for rendering. It is possible to render a subset of a sprite using a rectangle. The rectangle will use [0] and [1] for the beginning x and y position in the texture and [2] and [3] for the width and height. This is useful when the texture is a sprite sheet.
const float* ASGE:: Sprite:: srcRect() const noexcept
| Returns | The rectangle (as a constant) used for the UV source. |
|---|
Gets a read-only source rectangle used for rendering. It is possible to render a subset of a sprite using a rectangle. The rectangle will use [0] and [1] for the beginning x and y position in the texture and [2] and [3] for the width and height. This is useful when the texture is a sprite sheet.
Variable documentation
float ASGE:: Sprite:: scale_factor protected
Sprite Scale. Scales the sprite equally in both dims.
FlipFlags ASGE:: Sprite:: flip_flags protected
Sprite Texture Flip. Flags to control UV mappings.


Total number of infected people as of today.
Number of newly infected people since yesterday :
Serial
Province/State
Country/Region
Latest Infected Count
Infected Since Yesterday
0
0
Return to home


?2^
PLANTS
OF
BAHAMA
ISLANDS
Panicum
gerainatum
Forsk.
Great
Extuna:
in
water
of
Basil's
Hole,
Hooper's
Bay,
coppice.
January
25,
1978
*new
YORj^
BOTANICAL
GARDEH
New
York
Botanical
Garden
ibosm?
D.
S.Correll
¤
H.B.Correll
49415
(with
Priscilla
Fawcett)
Herbarium
FAIRCHILD
TROPICAL
GARDEN
1605147
Legend -
Level of confidence that token is an accurately-transcribed word
extremely low
very low
low
undetermined
medium
high
very high


Resize
Grab the right or bottom border and drag to the desired width or height.
Resize random widget
0
1
2
3
4
5
6
7
8
9


pandas docstring guide¶
Note
Video tutorial: Pandas docstring guide by Frank Akogun.
About docstrings and standards¶
A Python docstring is a string used to document a Python module, class, function or method, so programmers can understand what it does without having to read the details of the implementation.
Also, it is a common practice to generate online (html) documentation automatically from docstrings. Sphinx serves this purpose.
Next example gives an idea on how a docstring looks like:
def add(num1, num2):
"""
Add up two integer numbers.
This function simply wraps the `+` operator, and does not
do anything interesting, except for illustrating what is
the docstring of a very simple function.
Parameters
----------
num1 : int
First number to add
num2 : int
Second number to add
Returns
-------
int
The sum of `num1` and `num2`
See Also
--------
subtract : Subtract one integer from another
Examples
--------
>>> add(2, 2)
4
>>> add(25, 0)
25
>>> add(10, -10)
0
"""
return num1 + num2
Some standards exist about docstrings, so they are easier to read, and they can be exported to other formats such as html or pdf.
The first conventions every Python docstring should follow are defined in PEP-257.
As PEP-257 is quite open, and some other standards exist on top of it. In the case of pandas, the numpy docstring convention is followed. The conventions is explained in this document:
- numpydoc docstring guide (which is based in the original Guide to NumPy/SciPy documentation)
numpydoc is a Sphinx extension to support the numpy docstring convention.
The standard uses reStructuredText (reST). reStructuredText is a markup language that allows encoding styles in plain text files. Documentation about reStructuredText can be found in:
The rest of this document will summarize all the above guides, and will provide additional convention specific to the pandas project.
Writing a docstring¶
General rules¶
Docstrings must be defined with three double-quotes. No blank lines should be left before or after the docstring. The text starts in the next line after the opening quotes. The closing quotes have their own line (meaning that they are not at the end of the last sentence).
In rare occasions reST styles like bold text or itallics will be used in docstrings, but is it common to have inline code, which is presented between backticks. It is considered inline code:
- The name of a parameter
- Python code, a module, function, built-in, type, literal… (e.g.
os
,list
,numpy.abs
,datetime.date
,True
) - A pandas class (in the form
:class:`pandas.Series`
) - A pandas method (in the form
:meth:`pandas.Series.sum`
) - A pandas function (in the form
:func:`pandas.to_datetime`
)
Note
To display only the last component of the linked class, method or
function, prefix it with ~
. For example, :class:`~pandas.Series`
will link to pandas.Series
but only display the last part, Series
as the link text. See Sphinx cross-referencing syntax
for details.
Good:
def add_values(arr):
"""
Add the values in `arr`.
This is equivalent to Python `sum` of :meth:`pandas.Series.sum`.
Some sections are omitted here for simplicity.
"""
return sum(arr)
Bad:
def func():
"""Some function.
With several mistakes in the docstring.
It has a blank like after the signature `def func():`.
The text 'Some function' should go in the line after the
opening quotes of the docstring, not in the same line.
There is a blank line between the docstring and the first line
of code `foo = 1`.
The closing quotes should be in the next line, not in this one."""
foo = 1
bar = 2
return foo + bar
Section 1: Short summary¶
The short summary is a single sentence that expresses what the function does in a concise way.
The short summary must start with a capital letter, end with a dot, and fit in a single line. It needs to express what the object does without providing details. For functions and methods, the short summary must start with an infinitive verb.
Good:
def astype(dtype):
"""
Cast Series type.
This section will provide further details.
"""
pass
Bad:
def astype(dtype):
"""
Casts Series type.
Verb in third-person of the present simple, should be infinitive.
"""
pass
def astype(dtype):
"""
Method to cast Series type.
Does not start with verb.
"""
pass
def astype(dtype):
"""
Cast Series type
Missing dot at the end.
"""
pass
def astype(dtype):
"""
Cast Series type from its current type to the new type defined in
the parameter dtype.
Summary is too verbose and doesn't fit in a single line.
"""
pass
Section 2: Extended summary¶
The extended summary provides details on what the function does. It should not go into the details of the parameters, or discuss implementation notes, which go in other sections.
A blank line is left between the short summary and the extended summary. And every paragraph in the extended summary is finished by a dot.
The extended summary should provide details on why the function is useful and their use cases, if it is not too generic.
def unstack():
"""
Pivot a row index to columns.
When using a multi-index, a level can be pivoted so each value in
the index becomes a column. This is especially useful when a subindex
is repeated for the main index, and data is easier to visualize as a
pivot table.
The index level will be automatically removed from the index when added
as columns.
"""
pass
Section 3: Parameters¶
The details of the parameters will be added in this section. This section has the title “Parameters”, followed by a line with a hyphen under each letter of the word “Parameters”. A blank line is left before the section title, but not after, and not between the line with the word “Parameters” and the one with the hyphens.
After the title, each parameter in the signature must be documented, including *args and **kwargs, but not self.
The parameters are defined by their name, followed by a space, a colon, another space, and the type (or types). Note that the space between the name and the colon is important. Types are not defined for *args and **kwargs, but must be defined for all other parameters. After the parameter definition, it is required to have a line with the parameter description, which is indented, and can have multiple lines. The description must start with a capital letter, and finish with a dot.
For keyword arguments with a default value, the default will be listed after a comma at the end of the type. The exact form of the type in this case will be “int, default 0”. In some cases it may be useful to explain what the default argument means, which can be added after a comma “int, default -1, meaning all cpus”.
In cases where the default value is None, meaning that the value will not be used. Instead of “str, default None”, it is preferred to write “str, optional”. When None is a value being used, we will keep the form “str, default None”. For example, in df.to_csv(compression=None), None is not a value being used, but means that compression is optional, and no compression is being used if not provided. In this case we will use str, optional. Only in cases like func(value=None) and None is being used in the same way as 0 or foo would be used, then we will specify “str, int or None, default None”.
Good:
class Series:
def plot(self, kind, color='blue', **kwargs):
"""
Generate a plot.
Render the data in the Series as a matplotlib plot of the
specified kind.
Parameters
----------
kind : str
Kind of matplotlib plot.
color : str, default 'blue'
Color name or rgb code.
**kwargs
These parameters will be passed to the matplotlib plotting
function.
"""
pass
Bad:
class Series:
def plot(self, kind, **kwargs):
"""
Generate a plot.
Render the data in the Series as a matplotlib plot of the
specified kind.
Note the blank line between the parameters title and the first
parameter. Also, note that after the name of the parameter `kind`
and before the colon, a space is missing.
Also, note that the parameter descriptions do not start with a
capital letter, and do not finish with a dot.
Finally, the `**kwargs` parameter is missing.
Parameters
----------
kind: str
kind of matplotlib plot
"""
pass
Parameter types¶
When specifying the parameter types, Python built-in data types can be used directly (the Python type is preferred to the more verbose string, integer, boolean, etc):
- int
- float
- str
- bool
For complex types, define the subtypes. For dict and tuple, as more than one type is present, we use the brackets to help read the type (curly brackets for dict and normal brackets for tuple):
- list of int
- dict of {str : int}
- tuple of (str, int, int)
- tuple of (str,)
- set of str
In case where there are just a set of values allowed, list them in curly brackets and separated by commas (followed by a space). If the values are ordinal and they have an order, list them in this order. Otherwise, list the default value first, if there is one:
- {0, 10, 25}
- {‘simple’, ‘advanced’}
- {‘low’, ‘medium’, ‘high’}
- {‘cat’, ‘dog’, ‘bird’}
If the type is defined in a Python module, the module must be specified:
- datetime.date
- datetime.datetime
- decimal.Decimal
If the type is in a package, the module must be also specified:
- numpy.ndarray
- scipy.sparse.coo_matrix
If the type is a pandas type, also specify pandas except for Series and DataFrame:
- Series
- DataFrame
- pandas.Index
- pandas.Categorical
- pandas.SparseArray
If the exact type is not relevant, but must be compatible with a numpy array, array-like can be specified. If Any type that can be iterated is accepted, iterable can be used:
- array-like
- iterable
If more than one type is accepted, separate them by commas, except the last two types, that need to be separated by the word ‘or’:
- int or float
- float, decimal.Decimal or None
- str or list of str
If None
is one of the accepted values, it always needs to be the last in
the list.
For axis, the convention is to use something like:
- axis : {0 or ‘index’, 1 or ‘columns’, None}, default None
Section 4: Returns or Yields¶
If the method returns a value, it will be documented in this section. Also if the method yields its output.
The title of the section will be defined in the same way as the “Parameters”. With the names “Returns” or “Yields” followed by a line with as many hyphens as the letters in the preceding word.
The documentation of the return is also similar to the parameters. But in this case, no name will be provided, unless the method returns or yields more than one value (a tuple of values).
The types for “Returns” and “Yields” are the same as the ones for the “Parameters”. Also, the description must finish with a dot.
For example, with a single value:
def sample():
"""
Generate and return a random number.
The value is sampled from a continuous uniform distribution between
0 and 1.
Returns
-------
float
Random number generated.
"""
return random.random()
With more than one value:
def random_letters():
"""
Generate and return a sequence of random letters.
The length of the returned string is also random, and is also
returned.
Returns
-------
length : int
Length of the returned string.
letters : str
String of random letters.
"""
length = random.randint(1, 10)
letters = ''.join(random.choice(string.ascii_lowercase)
for i in range(length))
return length, letters
If the method yields its value:
def sample_values():
"""
Generate an infinite sequence of random numbers.
The values are sampled from a continuous uniform distribution between
0 and 1.
Yields
------
float
Random number generated.
"""
while True:
yield random.random()
Section 5: See Also¶
This section is used to let users know about pandas functionality related to the one being documented. In rare cases, if no related methods or functions can be found at all, this section can be skipped.
An obvious example would be the head() and tail() methods. As tail() does the equivalent as head() but at the end of the Series or DataFrame instead of at the beginning, it is good to let the users know about it.
To give an intuition on what can be considered related, here there are some examples:
loc
andiloc
, as they do the same, but in one case providing indices and in the other positionsmax
andmin
, as they do the oppositeiterrows
,itertuples
anditeritems
, as it is easy that a user looking for the method to iterate over columns ends up in the method to iterate over rows, and vice-versafillna
anddropna
, as both methods are used to handle missing valuesread_csv
andto_csv
, as they are complementarymerge
andjoin
, as one is a generalization of the otherastype
andpandas.to_datetime
, as users may be reading the documentation ofastype
to know how to cast as a date, and the way to do it is withpandas.to_datetime
where
is related tonumpy.where
, as its functionality is based on it
When deciding what is related, you should mainly use your common sense and think about what can be useful for the users reading the documentation, especially the less experienced ones.
When relating to other libraries (mainly numpy
), use the name of the module
first (not an alias like np
). If the function is in a module which is not
the main one, like scipy.sparse
, list the full module (e.g.
scipy.sparse.coo_matrix
).
This section, as the previous, also has a header, “See Also” (note the capital S and A). Also followed by the line with hyphens, and preceded by a blank line.
After the header, we will add a line for each related method or function, followed by a space, a colon, another space, and a short description that illustrated what this method or function does, why is it relevant in this context, and what are the key differences between the documented function and the one referencing. The description must also finish with a dot.
Note that in “Returns” and “Yields”, the description is located in the following line than the type. But in this section it is located in the same line, with a colon in between. If the description does not fit in the same line, it can continue in the next ones, but it has to be indented in them.
For example:
class Series:
def head(self):
"""
Return the first 5 elements of the Series.
This function is mainly useful to preview the values of the
Series without displaying the whole of it.
Returns
-------
Series
Subset of the original series with the 5 first values.
See Also
--------
Series.tail : Return the last 5 elements of the Series.
Series.iloc : Return a slice of the elements in the Series,
which can also be used to return the first or last n.
"""
return self.iloc[:5]
Section 6: Notes¶
This is an optional section used for notes about the implementation of the algorithm. Or to document technical aspects of the function behavior.
Feel free to skip it, unless you are familiar with the implementation of the algorithm, or you discover some counter-intuitive behavior while writing the examples for the function.
This section follows the same format as the extended summary section.
Section 7: Examples¶
This is one of the most important sections of a docstring, even if it is placed in the last position. As often, people understand concepts better with examples, than with accurate explanations.
Examples in docstrings, besides illustrating the usage of the function or method, must be valid Python code, that in a deterministic way returns the presented output, and that can be copied and run by users.
They are presented as a session in the Python terminal. >>> is used to present code. … is used for code continuing from the previous line. Output is presented immediately after the last line of code generating the output (no blank lines in between). Comments describing the examples can be added with blank lines before and after them.
The way to present examples is as follows:
- Import required libraries (except
numpy
andpandas
) - Create the data required for the example
- Show a very basic example that gives an idea of the most common use case
- Add examples with explanations that illustrate how the parameters can be used for extended functionality
A simple example could be:
class Series:
def head(self, n=5):
"""
Return the first elements of the Series.
This function is mainly useful to preview the values of the
Series without displaying the whole of it.
Parameters
----------
n : int
Number of values to return.
Return
------
pandas.Series
Subset of the original series with the n first values.
See Also
--------
tail : Return the last n elements of the Series.
Examples
--------
>>> s = pd.Series(['Ant', 'Bear', 'Cow', 'Dog', 'Falcon',
... 'Lion', 'Monkey', 'Rabbit', 'Zebra'])
>>> s.head()
0 Ant
1 Bear
2 Cow
3 Dog
4 Falcon
dtype: object
With the `n` parameter, we can change the number of returned rows:
>>> s.head(n=3)
0 Ant
1 Bear
2 Cow
dtype: object
"""
return self.iloc[:n]
The examples should be as concise as possible. In cases where the complexity of
the function requires long examples, is recommended to use blocks with headers
in bold. Use double star **
to make a text bold, like in **this example**
.
Conventions for the examples¶
Code in examples is assumed to always start with these two lines which are not shown:
import numpy as np
import pandas as pd
Any other module used in the examples must be explicitly imported, one per line (as recommended in PEP-8) and avoiding aliases. Avoid excessive imports, but if needed, imports from the standard library go first, followed by third-party libraries (like matplotlib).
When illustrating examples with a single Series
use the name s
, and if
illustrating with a single DataFrame
use the name df
. For indices,
idx
is the preferred name. If a set of homogeneous Series
or
DataFrame
is used, name them s1
, s2
, s3
… or df1
,
df2
, df3
… If the data is not homogeneous, and more than one structure
is needed, name them with something meaningful, for example df_main
and
df_to_join
.
Data used in the example should be as compact as possible. The number of rows
is recommended to be around 4, but make it a number that makes sense for the
specific example. For example in the head
method, it requires to be higher
than 5, to show the example with the default values. If doing the mean
, we
could use something like [1, 2, 3]
, so it is easy to see that the value
returned is the mean.
For more complex examples (groupping for example), avoid using data without interpretation, like a matrix of random numbers with columns A, B, C, D… And instead use a meaningful example, which makes it easier to understand the concept. Unless required by the example, use names of animals, to keep examples consistent. And numerical properties of them.
When calling the method, keywords arguments head(n=3)
are preferred to
positional arguments head(3)
.
Good:
class Series:
def mean(self):
"""
Compute the mean of the input.
Examples
--------
>>> s = pd.Series([1, 2, 3])
>>> s.mean()
2
"""
pass
def fillna(self, value):
"""
Replace missing values by `value`.
Examples
--------
>>> s = pd.Series([1, np.nan, 3])
>>> s.fillna(0)
[1, 0, 3]
"""
pass
def groupby_mean(self):
"""
Group by index and return mean.
Examples
--------
>>> s = pd.Series([380., 370., 24., 26],
... name='max_speed',
... index=['falcon', 'falcon', 'parrot', 'parrot'])
>>> s.groupby_mean()
index
falcon 375.0
parrot 25.0
Name: max_speed, dtype: float64
"""
pass
def contains(self, pattern, case_sensitive=True, na=numpy.nan):
"""
Return whether each value contains `pattern`.
In this case, we are illustrating how to use sections, even
if the example is simple enough and does not require them.
Examples
--------
>>> s = pd.Series('Antelope', 'Lion', 'Zebra', numpy.nan)
>>> s.contains(pattern='a')
0 False
1 False
2 True
3 NaN
dtype: bool
**Case sensitivity**
With `case_sensitive` set to `False` we can match `a` with both
`a` and `A`:
>>> s.contains(pattern='a', case_sensitive=False)
0 True
1 False
2 True
3 NaN
dtype: bool
**Missing values**
We can fill missing values in the output using the `na` parameter:
>>> s.contains(pattern='a', na=False)
0 False
1 False
2 True
3 False
dtype: bool
"""
pass
Bad:
def method(foo=None, bar=None):
"""
A sample DataFrame method.
Do not import numpy and pandas.
Try to use meaningful data, when it makes the example easier
to understand.
Try to avoid positional arguments like in `df.method(1)`. They
can be all right if previously defined with a meaningful name,
like in `present_value(interest_rate)`, but avoid them otherwise.
When presenting the behavior with different parameters, do not place
all the calls one next to the other. Instead, add a short sentence
explaining what the example shows.
Examples
--------
>>> import numpy as np
>>> import pandas as pd
>>> df = pd.DataFrame(numpy.random.randn(3, 3),
... columns=('a', 'b', 'c'))
>>> df.method(1)
21
>>> df.method(bar=14)
123
"""
pass
Tips for getting your examples pass the doctests¶
Getting the examples pass the doctests in the validation script can sometimes be tricky. Here are some attention points:
Import all needed libraries (except for pandas and numpy, those are already imported as
import pandas as pd
andimport numpy as np
) and define all variables you use in the example.Try to avoid using random data. However random data might be OK in some cases, like if the function you are documenting deals with probability distributions, or if the amount of data needed to make the function result meaningful is too much, such that creating it manually is very cumbersome. In those cases, always use a fixed random seed to make the generated examples predictable. Example:
>>> np.random.seed(42) >>> df = pd.DataFrame({'normal': np.random.normal(100, 5, 20)})
If you have a code snippet that wraps multiple lines, you need to use ‘…’ on the continued lines:
>>> df = pd.DataFrame([[1, 2, 3], [4, 5, 6]], index=['a', 'b', 'c'], ... columns=['A', 'B'])
If you want to show a case where an exception is raised, you can do:
>>> pd.to_datetime(["712-01-01"]) Traceback (most recent call last): OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 712-01-01 00:00:00
It is essential to include the “Traceback (most recent call last):”, but for the actual error only the error name is sufficient.
If there is a small part of the result that can vary (e.g. a hash in an object represenation), you can use
...
to represent this part.If you want to show that
s.plot()
returns a matplotlib AxesSubplot object, this will fail the doctest>>> s.plot() <matplotlib.axes._subplots.AxesSubplot at 0x7efd0c0b0690>
However, you can do (notice the comment that needs to be added)
>>> s.plot() <matplotlib.axes._subplots.AxesSubplot at ...>
Plots in examples¶
There are some methods in pandas returning plots. To render the plots generated
by the examples in the documentation, the .. plot::
directive exists.
To use it, place the next code after the “Examples” header as shown below. The plot will be generated automatically when building the documentation.
class Series:
def plot(self):
"""
Generate a plot with the `Series` data.
Examples
--------
.. plot::
:context: close-figs
>>> s = pd.Series([1, 2, 3])
>>> s.plot()
"""
pass


190_ling-dian-32.py
class Solution(object):
def cherryPickup(self, grid):
"""
:type grid: List[List[int]]
:rtype: int
"""
w = len(grid[0])
dp = [[None]*w for _ in range(w)]
dp[0][-1] = grid[0][0]+grid[0][-1]
print(dp)
for row in grid[1:]:
ndp = [[None]*w for _ in range(w)]
for i in range(w):
for j in range(w):
if i == j:
for k in range(max(0,i-1),min(w,i+2)):
if not dp[i][j] is None:
if ndp[i][j] is None:
ndp[i][j] = dp[k][k] + row[i]
else:
ndp[i][j] = max(ndp[i][j], dp[k][k] + row[i])
else:
for ki in range(max(0,i-1),min(w,i+2)):
for kj in range(max(0,j-1),min(w,j+2)):
if not dp[ki][kj] is None:
if ndp[i][j] is None:
ndp[i][j] = dp[ki][kj] + row[i] + row[j]
else:
ndp[i][j] = max(ndp[i][j],dp[ki][kj] + row[i] + row[j])
dp = ndp
res = float('-inf')
for row in dp:
for e in row:
if not e is None:
res = max(res,e)
return res


Academic writing in plain text: useful resources
This post is more of a continuously updated note on possible workflows that are based on plain text and the principles of minimal computing.
Plain text and MarkDown
AcademicMarkdown
CriticMarkup
CriticMarkup: extension to MMD for editorial markup of texts. The project website is http://criticmarkup.com/ and the code is hosted on GitHub
- Integration into tools:
- Sublime Text 3’s AcademicMarkdown package supports syntax highlighting for CriticMarkup
- There is a CriticMarkup SL3 package. It is not managed by package control and a documentation can be found here
- The CriticMarkup toolkit contains processors for various programs but not for Sublime Text.
- Integration of CriticMarkup and Pandoc
- Basic Syntax
- Addition
{++ ++}
- Deletion
{-- --}
- Substitution
{~~ ~> ~~}
- Comment
{>> <<}
- Highlight
{== ==}{>> <<}
- Addition
KramDown
An extension of MarkDown with a strict, unambiguous syntax used in Jekyll. The documentation of the syntax can be found here.
Pandoc and Jekyll
Pandoc can be used as standard processor for markdown in jekyll with the help of small plugins. However, one cannot use any plug-ins if jekyll is run on GitHub-pages, which operate in safe-mode. There was a lengthy discussion on the jekyll forum whether or not to include pandoc as a markdown processor. Unfortunately this has been rejected by the jekyll team.
The advantage of pandoc as markdown processor would be support for image captions etc.
Another pandoc plugin for jekyll
jekyll-pandoc
To install it via terminal type
$ gem install jekyll-pandoc
It can then be referenced in one’s site’s _config.yml
as markdown: pandoc
after the following has been added:
gems:
- jekyll-pandoc
jekyll-pandoc-plugin
This was originally designed to work with jekyll 1.x. Some forks have been updated to provide compatibility with jekyll 2.x and 3.x
Git / GitHub
- Chad Black’s getting started with github and prose.io on using a combination of GitHub, jekyll, and prose.io for a collaborative seminar blog.
- GitHub, Academia, and Collaborative Writing
- Push, Pull, Fork: GitHub for Academics
- GitHub for Academics: the open-source way to host, create and curate knowledge
- Living in a Plain Text World (Tools We Use)
- This software for academic paper writing is inspired by Git
- How To: Use Git to Version Your Writing
- Using git in my writing workflow
Jekyll
- Tags in Jekyll / GitHub pages
- Pandoc
- Jekyll themes
- Poole, Jekyll’s buttler: editor for Jekyll themes, at the moment at least two themes are available and GO::DH Minimal Computing SIG used it to build its website
- Plug-ins: if hosted on GitHub, jekyll runs in safe mode and disables third-party plug-ins
- add image captions:
Liquid templating
Liquid templating was designed by and for the shopify website: “Liquid is an open-source, Ruby-based template language created by Shopify. It is the backbone of Shopify themes and is used to load dynamic content on storefronts.”
- Liquid template engine
- Great overview of syntax and commands
- Jekyll Date Formatting Examples using liquid
Images in Jekyll
the standard markdown syntax of ![caption, alt text](path to image file)
works with jekyll, but the alt text is not written to a caption element etc.
A comment on stackoverflow suggests to generate a new template file called image.html
in _includes

and then to call this template with
![My cat, Robert Downey Jr.](/assets/my-cat.jpg)
The regex to replace existing markdown image links with the new liquid ones is:
- find:
!\[(Page.+)\]\(\{\{site.baseurl\}\}(/assets/.+.png)\)
- replace>
Post preview in jekyll
add excerpt_separator:
to _config.yml or the YAML block of your individual post.


Syntax:
search sequence-1 sequence-2 &key from-end test test-not key start1 start2 end1 end2
=> position
Arguments and Values:
Sequence-1---a sequence.
Sequence-2---a sequence.
from-end---a generalized boolean. The default is false.
test---a designator for a function of two arguments that returns a generalized boolean.
test-not---a designator for a function of two arguments that returns a generalized boolean.
key---a designator for a function of one argument, or nil.
start1, end1---bounding index designators of sequence-1. The defaults for start1 and end1 are 0 and nil, respectively.
start2, end2---bounding index designators of sequence-2. The defaults for start2 and end2 are 0 and nil, respectively.
position---a bounding index of sequence-2, or nil.
Description:
Searches sequence-2 for a subsequence that matches sequence-1.
The implementation may choose to search sequence-2 in any order; there is no guarantee on the number of times the test is made. For example, when start-end is true, the sequence might actually be searched from left to right instead of from right to left (but in either case would return the rightmost matching subsequence). If the search succeeds, search returns the offset into sequence-2 of the first element of the leftmost or rightmost matching subsequence, depending on from-end; otherwise search returns nil.
If from-end is true, the index of the leftmost element of the rightmost matching subsequence is returned.
Examples:
(search "dog" "it's a dog's life") => 7 (search '(0 1) '(2 4 6 1 3 5) :key #'oddp) => 2
Side Effects: None.
Affected By: None.
Exceptional Situations: None.
See Also:
Section 3.6 (Traversal Rules and Side Effects)
Notes:
The :test-not argument is deprecated.


{% load i18n material_frontend viewflow viewflow_frontend %} {% flows_start_actions view.flows request.user as start_actions %} {% if start_actions %}


Table of Contents - RDoc Documentation
Pages
Classes and Modules
Methods
- ::connection_for — CachedCounts
- #add_association_counting_hooks — CachedCounts
- #add_count_attribute_methods — CachedCounts
- #add_counting_hooks — CachedCounts
- #add_scope_counting_hooks — CachedCounts
- #association_count_key — CachedCounts
- #caches_count_of — CachedCounts
- #caches_count_of! — CachedCounts
- #caches_count_where — CachedCounts
- #caches_count_where! — CachedCounts
- #default_race_condition_fallback_proc — CachedCounts
- #define_association_count_attribute — CachedCounts
- #define_scope_count_attribute — CachedCounts
- #generate_association_counting_hook_key_getter — CachedCounts
- #generate_association_relation_getter — CachedCounts
- #scope_count_key — CachedCounts


Aletheia testnet faucet
Aletheia test network faucet
Your address will be given access to perform actions on Aletheia
Your address
Please retype the letters in the box
Request 0.5 ETH


About Assertions 5
Google Truth and AssertJ delivered a new style of assertions, where you don’t need to remember a lot of method names and class names placed in packages of Hamcrest’s jar file.
It made possible to rely on your IDE for listing available methods, from which you can choose a desired one, instead of relying on your memory for it.
The only thing that you need to remember is the fact that the method assertThat
is what you need.
Once you type assertThat(yourValue)
, your IDE will suggest to do static import [Figure.1].
And then, list methods that can be imported statically from your classpath[Figure.2].
To render these images, I used the code found at [BaeldungAssertJ].
The Pain Points
Please look at the method names and their packages a bit carefully.
These are the custom supports available for the target class Dog
.
PersonAssert.assertThat(com.baeldung.assertj.custom) (1) Assertions.assertThat(com.baeldung.assertj.custom) (2) UserSubject.assertThat(com.baeldung.assertj.custom) (3)
-
Support for AssertJ
-
Support for Hamcrest
-
Support for Google Truth
For instance, the class PersonAssert
is implemented as follows.
This code is cited from [BaeldungAssertionTutorial] and I’ve annotated [1].
package com.baeldung.assertj.custom;
import org.assertj.core.api.AbstractAssert;
public class PersonAssert extends AbstractAssert<PersonAssert, Person> {
public PersonAssert(Person actual) {
super(actual, PersonAssert.class);
}
public static PersonAssert assertThat(Person actual) {
return new PersonAssert(actual);
}
public PersonAssert hasFullName(String fullName) {
isNotNull();
if (!actual.getFullName().equals(fullName)) {
failWithMessage("Expected person to have full name %s but was %s", fullName, actual.getFullName()); //(1)
}
return this;
}
public PersonAssert isAdult() {
isNotNull();
if (actual.getAge() < 18) {
failWithMessage("Expected person to be adult"); //(1)
}
return this;
}
public PersonAssert hasNickname(String nickName) {
isNotNull();
if (!actual.getNicknames().contains(nickName)) {
failWithMessage("Expected person to have nickname %s", nickName); //(1)
}
return this;
}
}
-
A message string that a human programmer needs to come up with.
This is the first paint point in the "fluent" assertion style in these days, I’d point out. As annotated, you need to come up with a good message for each verification method. Another pain is the fact itself that we need to create a custom support class for your class every time.
Next Post: Yet another pain point (or others?)
This time, I mentioned a couple of pain points of the modern fluent assertion style, but they are not all. In the next post, I will discuss yet another pain point, or two.
References
Dog
a person? Please don’t blame me.


--- layout: post title: Insecure Programming by example date: '2007-08-05T23:01:00.001-03:00' author: meleu tags: - C - link - programacao modified_time: '2007-08-07T22:14:53.647-03:00' blogger_id: tag:blogger.com,1999:blog-1613465446497619833.post-5199409294385917605 blogger_orig_url: http://mdicas.blogspot.com/2007/08/insecure-programming-by-exampe.html ---


The web is ready for a great graphic designer.
by Hidde de Vries
Hidde de Vries is a front-end developer who works at the World Wide Web Consortium (W3C). W3C is an international community that develops open standards to ensure the long-term growth of the Web. Hidde invents and develops new web techniques that must be available for multiple browsers. As a front-end web developer he specialize in HTML, CSS, JavaScript, living style guides and accessibility for everyone.
Modernism in graphic design
Where function is above form (beauty). And Asymmetric alignment is effective. Each problem has its own solution. Modernism is pretty rule-based, like the web, and it is a movement that responded to industrialisation.
Inspiration for current web design comes from Modernism. Websites nowadays look a lot alike. It feels like everyone uses the same template. That's why I regret that we almost always don't dare to design something outside of that fixed template. The web is a responsive medium and there is so much more we can do with it. Logos are a good example of how we design nowadays, many companies used to use graceful letters, now they have often become simple letters. Jan Tschichold ( one of the great modernist graphic designers) wrote the book: De Neue typography. Shows that by making effective use of different letter weights and sizes, the information could be transferred quickly and easily.
Grid and flexbox
Why is using grids effective?
Müller-Brockmann explains in his book over grids why we'd want grids in the first place. He explains that if you have a bunch of content that needs to go into a layout, a grid can give solutions that are functional, logical, and just look better. When a grid is applied to it, a surface can more easily be organised in a way that is rational.
Something that we see the great graphic designers do a lot, is to use mathematical proportions to lay things out. Both flexbox and Grid Layout let us do this. Neither is better than the other. Flexbox lets us layout things from items, Grid Layout from the container.
.container{
display: flex;
}
.item-1{
flex: 2;
}
.item-2 {
flex: 1;
}
.container{
display: grid;
grid-template-columns: 1fr, 3fr, 2fr, 1fr;.
}
Take away
-
Leave maths to the browser.
For examples:
.container{
display: flex;
}
.item-1{
flex: 2;
}
.item-2 {
flex: 1;
}
.container{
display: grid;
grid-template-columns: 1fr, 3fr, 2fr, 1fr;.
}
- Use vh and vw units so you can design for each device screen.
- Use z-index to overlap the content.
- Check of the browser supports grid don't forget to write a fallback style for the browser that does not supports grid and your fallback has to be as simple as possible
@supports(display: grid){
body {
display: grid
}
}
- Consulting is better than having bugs.
- Stick to the standards of the web.
-
Embrace the flexibility of the web.
For example:
- 50c which means 50 characters
- minmax(... , ...) between two values - The web is interactive so make use of it.


Your browser does not support JavaScript!
Score:
0
Score:
0
StarTrek
Simulator
Great battle crew!
Best player was
Gary
Go again
StarTrek
Simulator
Your ship has been destroyed.
Go again
Join Room
Create Room
Non Binary
Female
Male
Starship
USS Entreprise
Miranda
Bird of Prey
IKS Devisor
Klingon
Federation
Join!
Create!


Admin page for vrock.it/index.html (dynamoDB)
Delete All
Delete All but BIM-IQ Data
Delete Creator (prompt will appear)
Upload button


Endless Possibilities
The smart piano works with your favorite third-party apps, from audio playlists to
music composition.
3 Steps to Start Using the App
Download, Connect, Start Playing
The smart piano works with your favorite third-party apps, from audio playlists to
music composition.


--- title: Getting Data to Shapes --- November 1, 2016 {{ page.title }} Let's Look at Your Progress http://www.dave-landry.com/classes/artg4552-2016/random/ Work in Class Get Data to Shapes Homework Have at least 1 SVG shape encoded with data.
Have at least 1 SVG shape encoded with data.


Parameter locale
The following game is shown on some boards with all possible
languages. Hover over the buttons to see the translated tooltips as well:
- the default locale en (not set) == English.
- the locale cs (explictly set) == Czech.
- the locale da (explictly set) == Danish.
- the locale de (explictly set) == German.
- the locale en (explictly set) == English.
- the locale es (explictly set) == Spanish.
- the locale et (explictly set) == Estonian.
- the locale fi (explictly set) == Finnish.
- the locale fr (explictly set) == French.
- the locale hu (explictly set) == Hungarian.
- the locale is (explictly set) == Icelandic.
- the locale it (explictly set) == Italian.
- the locale nb (explictly set) == Norvegian.
- the locale nl (explictly set) == Dutch.
- the locale pl (explictly set) == Polish.
- the locale pt (explictly set) == Portuguese.
- the locale ro (explictly set) == Romanian.
- the locale sv (explictly set) == Swedish.