Smile's classification algorithms are in the package
smile.classification
and all algorithms implement the interface
Classifier
that has a method predict
to predict the class label of an instance.
An overloaded version in SoftClassifier
can also calculate the a posteriori
probabilities besides the class label.
For all algorithms, the model can be trained through
the constructor. Meanwhile, each algorithm has a Trainer
companion
class that can hold model hyper-parameters and be applied to multiple training
datasets.
Some algorithms with online learning capability also implement the interface
OnlineClassifier
Online learning is a model of induction that learns one instance at a time.
The method learn
updates the model with a new instance.
The high-level operators are defined in Scala trait
smile.classification.Operators
and also in the package object of smile.classification
.
In what follows, we will discuss each algorithm, their high-level Scala API, and examples.
In machine learning and pattern recognition, classification refers to an algorithmic procedure for assigning a given input object into one of a given number of categories. The input object is formally termed an instance, and the categories are termed classes.
The instance is usually described by a vector of features, which together constitute a description of all known characteristics of the instance. Typically, features are either categorical (also known as nominal, i.e. consisting of one of a set of unordered items, such as a gender of "male" or "female", or a blood type of "A", "B", "AB" or "O"), ordinal (consisting of one of a set of ordered items, e.g. "large", "medium" or "small"), integer-valued (e.g. a count of the number of occurrences of a particular word in an email) or real-valued (e.g. a measurement of blood pressure).
Classification normally refers to a supervised procedure, i.e. a procedure that produces an inferred function to predict the output value of new instances based on a training set of pairs consisting of an input object and a desired output value. The inferred function is called a classifier if the output is discrete or a regression function if the output is continuous.
The inferred function should predict the correct output value for any valid input object. This requires the learning algorithm to generalize from the training data to unseen situations in a "reasonable" way.
A wide range of supervised learning algorithms is available, each with its strengths and weaknesses. There is no single learning algorithm that works best on all supervised learning problems. The most widely used learning algorithms are AdaBoost and gradient boosting, support vector machines, linear regression, linear discriminant analysis, logistic regression, naive Bayes, decision trees, k-nearest neighbor algorithm, and neural networks (multilayer perceptron).
If the feature vectors include features of many different kinds (discrete, discrete ordered, counts, continuous values), some algorithms cannot be easily applied. Many algorithms, including linear regression, logistic regression, neural networks, and nearest neighbor methods, require that the input features be numerical and scaled to similar ranges (e.g., to the [-1,1] interval). Methods that employ a distance function, such as nearest neighbor methods and support vector machines with Gaussian kernels, are particularly sensitive to this. An advantage of decision trees (and boosting algorithms based on decision trees) is that they easily handle heterogeneous data.
If the input features contain redundant information (e.g., highly correlated features), some learning algorithms (e.g., linear regression, logistic regression, and distance based methods) will perform poorly because of numerical instabilities. These problems can often be solved by imposing some form of regularization.
If each of the features makes an independent contribution to the output, then algorithms based on linear functions (e.g., linear regression, logistic regression, linear support vector machines, naive Bayes) generally perform well. However, if there are complex interactions among features, then algorithms such as nonlinear support vector machines, decision trees and neural networks work better. Linear methods can also be applied, but the engineer must manually specify the interactions when using them.
There are several major issues to consider in supervised learning:
The accuracy of the inferred function depends strongly on how the input object is represented. Typically, the input object is transformed into a feature vector, which contains a number of features that are descriptive of the object. The number of features should not be too large, because of the curse of dimensionality; but should contain enough information to accurately predict the output.
There are many algorithms for feature selection that seek to identify the relevant features and discard the irrelevant ones. More generally, dimensionality reduction may seek to map the input data into a lower dimensional space prior to running the supervised learning algorithm.
Overfitting occurs when a statistical model describes random error or noise instead of the underlying relationship. Overfitting generally occurs when a model is excessively complex, such as having too many parameters relative to the number of observations. A model which has been overfit will generally have poor predictive performance, as it can exaggerate minor fluctuations in the data.
The potential for overfitting depends not only on the number of parameters and data but also the conformability of the model structure with the data shape, and the magnitude of model error compared to the expected level of noise or error in the data.
In order to avoid overfitting, it is necessary to use additional techniques (e.g. cross-validation, regularization, early stopping, pruning, Bayesian priors on parameters or model comparison), that can indicate when further training is not resulting in better generalization. The basis of some techniques is either (1) to explicitly penalize overly complex models, or (2) to test the model's ability to generalize by evaluating its performance on a set of data not used for training, which is assumed to approximate the typical unseen data that a model will encounter.
Regularization involves introducing additional information in order to solve an ill-posed problem or to prevent over-fitting. This information is usually of the form of a penalty for complexity, such as restrictions for smoothness or bounds on the vector space norm. A theoretical justification for regularization is that it attempts to impose Occam's razor on the solution. From a Bayesian point of view, many regularization techniques correspond to imposing certain prior distributions on model parameters.
Mean squared error (MSE) can be broken down into two components: variance and squared bias, known as the bias-variance decomposition. Thus in order to minimize the MSE, we need to minimize both the bias and the variance. However, this is not trivial. Therefore, there is a tradeoff between bias and variance.
The k-nearest neighbor algorithm (k-NN) is a method for classifying objects by a majority vote of its neighbors, with the object being assigned to the class most common amongst its k nearest neighbors (k is typically small). k-NN is a type of instance-based learning, or lazy learning where the function is only approximated locally and all computation is deferred until classification.
def knn(x: Array[Array[Double]], y: Array[Int], k: Int): KNN[Array[Double]]
The simplest k-NN method takes a data set of feature vectors and labels
with Euclidean distance as the similarity measure. When applied to the
Iris datset, the accuracy of 10-fold cross validation is 96% for k = 3
.
val iris = readArff("data/weka/iris.arff", 4)
val (x, y) = iris.unzipInt
cv(x, y, 10) { case (x, y) => knn(x, y, 3) }
The best choice of k depends upon the data; generally, larger values of k reduce the effect of noise on the classification, but make boundaries between classes less distinct. A good k can be selected by various heuristic techniques, e.g. cross-validation. In binary problems, it is helpful to choose k to be an odd number as this avoids tied votes.
The nearest neighbor algorithm has some strong consistency results. As the amount of data approaches infinity, the algorithm is guaranteed to yield an error rate no worse than twice the Bayes error rate (the minimum achievable error rate given the distribution of the data). k-NN is guaranteed to approach the Bayes error rate, for some value of k (where k increases as a function of the number of data points).
The user can also provide a customized distance function.
def knn[T](x: Array[T], y: Array[Int], distance: Distance[T], k: Int): KNN[T]
Often, the classification accuracy of k-NN can be improved significantly if the distance metric is learned with specialized algorithms such as Large Margin Nearest Neighbor or Neighborhood Components Analysis.
Alternatively, the user may provide a k-nearest neighbor search data structure. Besides the simple linear search, Smile provides KD-Tree, Cover Tree, and LSH (Locality-Sensitive Hashing) for efficient k-nearest neighbor search.
def knn[T](x: KNNSearch[T, T], y: Array[Int], k: Int): KNN[T]
A KD-tree (short for k-dimensional tree) is a space-partitioning dataset structure for organizing points in a k-dimensional space. Cover tree is a data structure for generic nearest neighbor search (with a metric), which is especially efficient in spaces with small intrinsic dimension. The cover tree has a theoretical bound that is based on the dataset's doubling constant. LSH is an efficient algorithm for approximate nearest neighbor search in high dimensional spaces by performing probabilistic dimension reduction of data.
Nearest neighbor rules in effect compute the decision boundary in an
implicit manner. In the following example, we show the implicit decision
boundary of k-NN on a 2-dimensional toy data. Please try different k
,
and you will observe the change of decision boundary. In general, the larger
k
, the smoother the boundary.
val toy = readTable("classification/toy/toy-train.txt", Some((new NominalAttribute("class"), 0)))
val (x, y) = toy.unzipInt
plot(x, y, knn(x, y, 3)
In what follows, we will discuss the classification methods that compute the decision boundary explicitly. The computational complexity in the prediction phase is therefore a function of the boundary complexity. We will start with the simplest linear decision functions.
Linear discriminant analysis (LDA) is based on the Bayes decision theory and assumes that the conditional probability density functions are normally distributed. LDA also makes the simplifying homoscedastic assumption (i.e. that the class covariances are identical) and that the covariances have full rank. With these assumptions, the discriminant function of an input being in a class is purely a function of this linear combination of independent variables.
def lda(x: Array[Array[Double]], y: Array[Int], priori: Array[Double] = null, tol: Double = 0.0001): LDA
The parameter priori
is the priori probability of each class. If null, it will be
estimated from the training data. The parameter tol
is a tolerance to decide
if a covariance matrix is singular. The function will reject variables whose variance is
less than tol2.
As shown in the plot of toy data, the decision boundary of LDA is linear. Note that the decision boundary in the plot is the estimated contour on the testing grid (small dots) rather than the exact decision function. Therefore, there are the artifacts caused by discretized grid.
LDA is closely related to ANOVA (analysis of variance) and linear regression analysis, which also attempt to express one dependent variable as a linear combination of other features or measurements. In the other two methods, however, the dependent variable is a numerical quantity, while for LDA it is a categorical variable (i.e. the class label). Logistic regression and probit regression are more similar to LDA, as they also explain a categorical variable. These other methods are preferable in applications where it is not reasonable to assume that the independent variables are normally distributed, which is a fundamental assumption of the LDA method.
One complication in applying LDA (and Fisher's discriminant) to real data occurs when the number of variables/features does not exceed the number of samples. In this case, the covariance estimates do not have full rank, and so cannot be inverted. This is known as small sample size problem.
Fisher's linear discriminant (FLD) is another popular linear classifier. Fisher defined the separation between two distributions to be the ratio of the variance between the classes to the variance within the classes, which is, in some sense, a measure of the signal-to-noise ratio for the class labeling. FLD finds a linear combination of features which maximizes the separation after the projection. The resulting combination may be used for dimensionality reduction before later classification.
def fisher(x: Array[Array[Double]], y: Array[Int], L: Int = -1, tol: Double = 0.0001): FLD
The parameter L
is the dimensionality of mapped space.
The default value is the number of classes - 1.
In FLD, the features are mapped to the subspace spanned by the eigenvectors
corresponding to the C − 1
largest eigenvalues of
Σ-1Σb
,
where Σ
is the covariance matrix,
Σb
is the scatter matrix between class means,
and C
is the number of classes.
The terms Fisher's linear discriminant and LDA are often used interchangeably, although FLD actually describes a slightly different discriminant, which does not make some of the assumptions of LDA such as normally distributed classes or equal class covariances. When the assumptions of LDA are satisfied, FLD is equivalent to LDA.
FLD is also closely related to principal component analysis (PCA), which also looks for linear combinations of variables which best explain the data. As a supervised method, FLD explicitly attempts to model the difference between the classes of data. On the other hand, PCA is a unsupervised method and does not take into account any difference in class.
One complication in applying FLD (and LDA) to real data occurs when the number of variables/features does not exceed the number of samples. In this case, the covariance estimates do not have full rank, and so cannot be inverted. This is known as small sample size problem.
Quadratic discriminant analysis (QDA) is closely related to LDA. Like LDA, QDA models the conditional probability density functions as a Gaussian distribution, then uses the posterior distributions to estimate the class for a given test data.
def qda(x: Array[Array[Double]], y: Array[Int], priori: Array[Double] = null, tol: Double = 0.0001): QDA
Unlike LDA, however, in QDA there is no assumption that the covariance of each of the classes is identical. Therefore, the resulting separating surface between the classes is quadratic.
The Gaussian parameters for each class can be estimated from training data with maximum likelihood (ML) estimation. However, when the number of training instances is small compared to the dimension of input space, the ML covariance estimation can be ill-posed. One approach to resolve the ill-posed estimation is to regularize the covariance estimation as in regularized discriminant analysis.
In LDA and FLD, the eigenvectors corresponding to the smaller eigenvalues will tend to be very sensitive to the exact choice of training data, and it is often necessary to use regularization. In the regularized discriminant analysis (RDA), the regularized covariance matrices of each class is Σk(α) = α Σk + (1 - α) Σ.
def rda(x: Array[Array[Double]], y: Array[Int], alpha: Double, priori: Array[Double] = null, tol: Double = 0.0001): RDA
where the parameter alpha
is the regularization factor in [0, 1]
.
RDA is a compromise between LDA and QDA, which allows one to shrink the separate covariances of QDA toward a common variance as in LDA. This method is very similar in flavor to ridge regression. The quadratic discriminant function is defined using the shrunken covariance matrices Σk(α). The parameter α in [0, 1] controls the complexity of the model. When α is one, RDA becomes QDA. While α is zero, RDA is equivalent to LDA. Therefore, the regularization factor α allows a continuum of models between LDA and QDA.
Logistic regression (logit model) is a generalized linear model used for binomial regression. Logistic regression applies maximum likelihood estimation after transforming the dependent into a logit variable. A logit is the natural log of the odds of the dependent equaling a certain value or not (usually 1 in binary logistic models, the highest value in multinomial models). In this way, logistic regression estimates the odds of a certain event (value) occurring.
def logit(x: Array[Array[Double]], y: Array[Int], lambda: Double = 0.0, tol: Double = 1E-5, maxIter: Int = 500): LogisticRegression
where the parameter lambda (> 0)
gives a "regularized" estimate of linear
weights which often has superior generalization performance, especially
when the dimensionality is high.
Logistic regression has many analogies to ordinary least squares (OLS) regression. Unlike OLS regression, however, logistic regression does not assume linearity of relationship between the raw values of the independent variables and the dependent, does not require normally distributed variables, does not assume homoscedasticity, and in general has less stringent requirements.
Compared with linear discriminant analysis, logistic regression has several advantages:
However, it requires much more data to achieve stable, meaningful results.
Logistic regression also has strong connections with neural network and maximum entropy modeling. For example, binary logistic regression is equivalent to a one-layer, single-output neural network with a logistic activation function trained under log loss. Similarly, multinomial logistic regression is equivalent to a one-layer, softmax-output neural network.
Logistic regression estimation also obeys the maximum entropy principle, and thus logistic regression is sometimes called "maximum entropy modeling", and the resulting classifier the "maximum entropy classifier".
Maximum entropy is a technique for learning probability distributions from data. In maximum entropy models, the observed data itself is assumed to be the testable information. Maximum entropy models don't assume anything about the probability distribution other than what have been observed and always choose the most uniform distribution subject to the observed constraints.
def maxent(x: Array[Array[Int]], y: Array[Int], p: Int, lambda: Double = 0.1, tol: Double = 1E-5, maxIter: Int = 500): Maxent
where x
is the sparse training samples. Each sample is represented by a set of sparse
binary features. The features are stored in an integer array, of which
are the indices of nonzero features.
The parameter p
is the dimension of feature space, and lambda
is the regularization factor.
Basically, maximum entropy classifier is another name of multinomial logistic regression applied to categorical independent variables, which are converted to binary dummy variables. Maximum entropy models are widely used in natural language processing. Therefore, our implementation assumes that binary features are stored in a sparse array, of which entries are the indices of nonzero features.
A multilayer perceptron neural network consists of several layers of nodes, interconnected through weighted acyclic arcs from each preceding layer to the following, without lateral or feedback connections. Each node calculates a transformed weighted linear combination of its inputs (output activations from the preceding layer), with one of the weights acting as a trainable bias connected to a constant input. The transformation, called activation function, is a bounded non-decreasing (non-linear) function, such as the sigmoid functions (ranges from 0 to 1). Another popular activation function is hyperbolic tangent which is actually equivalent to the sigmoid function in shape but ranges from -1 to 1. More specialized activation functions include radial basis functions which are used in RBF networks.
For neural networks, the input patterns usually should be scaled/standardized. Commonly, each input variable is scaled into interval [0, 1] or to have mean 0 and standard deviation 1.
def mlp(x: Array[Array[Double]], y: Array[Int], numUnits: Array[Int], error: NeuralNetwork.ErrorFunction, activation: NeuralNetwork.ActivationFunction, epochs: Int = 25, eta: Double = 0.1, alpha: Double = 0.0, lambda: Double = 0.0): NeuralNetwork
where numUnits
is the number of units in each layer, error
is the error function, activation
is the activation function of output layer,
epochs
is the number of epochs of stochastic learning,
eta
is the learning rate,
alpha
is the momentum factor,
and lambda
is the weight decay for regularization.
The meaning of these parameters will be clear in the following discussion.
For error/penalty functions and output units, the following natural pairings are recommended:
By assigning a softmax activation function on the output layer of the neural network for categorical target variables, the outputs can be interpreted as posterior probabilities, which are very useful.
The representational capabilities of a neural network are determined by the range of mappings it may implement through weight variation. Single layer perceptrons are capable of solving only linearly separable problems. With the sigmoid function as activation function, the single-layer network is identical to the logistic regression model.
The universal approximation theorem for neural networks states that every continuous function that maps intervals of real numbers to some output interval of real numbers can be approximated arbitrarily closely by a multi-layer perceptron with just one hidden layer. This result holds only for restricted classes of activation functions, which are extremely complex and NOT smooth for subtle mathematical reasons. On the other hand, smoothness is important for gradient descent learning. Besides, the proof is not constructive regarding the number of neurons required or the settings of the weights. Therefore, complex systems will have more layers of neurons with some having increased layers of input neurons and output neurons in practice. The extra layers enable composition of features from lower layers, giving the potential of modeling complex data with fewer units than a similarly performing shallow network. But many issues can arise with deep neural networks if they are naively trained. Two common issues are overfitting and computation time.
The most popular algorithm to train neural networks is back-propagation, which is a gradient descent method. Based on chain rule, the algorithm propagates the error back through the network and adjusts the weights of each connection in order to reduce the value of the error function by some small amount. For this reason, back-propagation can only be applied on networks with differentiable activation functions.
During error back propagation, we usually times the gradient with a small number η, called learning rate, which is carefully selected to ensure that the network converges to a local minimum of the error function fast enough, without producing oscillations. One way to avoid oscillation at large η, is to make the change in weight dependent on the past weight change by adding a momentum term.
Although the back-propagation algorithm may performs gradient descent on the total error of all instances in a batch way, the learning rule is often applied to each instance separately in an online way or stochastic way. There exists empirical indication that the stochastic way results in faster convergence.
In practice, the problem of over-fitting has emerged. This arises in convoluted or over-specified systems when the capacity of the network significantly exceeds the needed free parameters. There are two general approaches for avoiding this problem: The first is to use cross-validation and similar techniques to check for the presence of over-fitting and optimally select hyper-parameters such as to minimize the generalization error. The second is to use some form of regularization, which emerges naturally in a Bayesian framework, where the regularization can be performed by selecting a larger prior probability over simpler models; but also in statistical learning theory, where the goal is to minimize over the "empirical risk" and the "structural risk".
A radial basis function network is an artificial neural network that uses radial basis functions as activation functions. It is a linear combination of radial basis functions. They are used in function approximation, time series prediction, and control.
In its basic form, radial basis function network is in the form
y(x) = Σ wi φ(||x-ci||)
where the approximating function y(x) is represented as a sum of N radial basis functions φ, each associated with a different center ci, and weighted by an appropriate coefficient wi. For distance, one usually chooses Euclidean distance. The weights wi can be estimated using the matrix methods of linear least squares, because the approximating function is linear in the weights.
RBF networks are typically trained by a two-step algorithm.
In the first step, the center vectors ci
of the RBF functions in the hidden layer are chosen.
This step can be performed in several ways; centers can be randomly
sampled from some set of examples, or they can be determined
using k-means clustering. Note that this step is unsupervised.
The second step simply fits a linear model with coefficients wi
to
the hidden layer's outputs with respect to some objective function.
A common objective function, at least for regression/function estimation,
is the least squares function.
A optional third backpropagation step can be performed to fine-tune all of the RBF network's parameters.
def rbfnet[T](x: Array[T], y: Array[Int], distance: Metric[T], rbf: RadialBasisFunction, centers: Array[T]): RBFNetwork[T]
def rbfnet[T](x: Array[T], y: Array[Int], distance: Metric[T], rbf: Array[RadialBasisFunction], centers: Array[T]): RBFNetwork[T]
def nrbfnet[T](x: Array[T], y: Array[Int], distance: Metric[T], rbf: RadialBasisFunction, centers: Array[T]): RBFNetwork[T]
def nrbfnet[T](x: Array[T], y: Array[Int], distance: Metric[T], rbf: Array[RadialBasisFunction], centers: Array[T]): RBFNetwork[T]
The popular choices for φ comprise the Gaussian function and the so called thin plate splines. The advantage of the thin plate splines is that their conditioning is invariant under scalings. Gaussian, multi-quadric and inverse multi-quadric are infinitely smooth and and involve a scale or shape parameter, r0 > 0. Decreasing r0 tends to flatten the basis function. For a given function, the quality of approximation may strongly depend on this parameter. In particular, increasing r0 has the effect of better conditioning (the separation distance of the scaled points increases).
val centers = new Array[Array[Double]](50)
val basis = gaussrbf(x, centers)
val classifier = rbfnet(x, y, new EuclideanDistance, basis, centers)
plot(x, y, classifier)
After running the above example, you may notice that the decision boundary
may be very different from the below picture. It is because the centers
learn from gaussrbf
, employing K-Means in this example,
are different every time, a nature of stochastic algorithm like K-Means.
This also shows that the decision boundary of RBF network is very sensistive
to the chosen centers.
A variant on RBF networks is normalized radial basis function (NRBF) networks, in which we require the sum of the basis functions to be unity. NRBF arises more naturally from a Bayesian statistical perspective. However, there is no evidence that either the NRBF method is consistently superior to the RBF method, or vice versa.
The basic support vector machine (SVM) is a binary linear classifier which chooses the hyperplane that represents the largest separation, or margin, between the two classes. If such a hyperplane exists, it is known as the maximum-margin hyperplane and the linear classifier it defines is known as a maximum margin classifier.
If there exists no hyperplane that can perfectly split the positive and negative instances, the soft margin method will choose a hyperplane that splits the instances as cleanly as possible, while still maximizing the distance to the nearest cleanly split instances.
The nonlinear SVMs are created by applying the kernel trick to maximum-margin hyperplanes. The resulting algorithm is formally similar, except that every dot product is replaced by a nonlinear kernel function. This allows the algorithm to fit the maximum-margin hyperplane in a transformed feature space. The transformation may be nonlinear and the transformed space be high dimensional. For example, the feature space corresponding Gaussian kernel is a Hilbert space of infinite dimension. Thus though the classifier is a hyperplane in the high-dimensional feature space, it may be nonlinear in the original input space. Maximum margin classifiers are well regularized, so the infinite dimension does not spoil the results.
def svm[T](x: Array[T], y: Array[Int], kernel: MercerKernel[T], C: Double, strategy: SVM.Multiclass = SVM.Multiclass.ONE_VS_ONE, epoch: Int = 1): SVM[T]
The effectiveness of SVM depends on the selection of kernel, the kernel's
parameters, and soft margin parameter C
. Given a kernel, best combination
of C
and kernel's parameters is often selected by a grid-search with
cross validation.
The dominant approach for creating multi-class SVMs is to reduce the single multi-class problem into multiple binary classification problems. Common methods for such reduction is to build binary classifiers which distinguish between (i) one of the labels to the rest (one-versus-all) or (ii) between every pair of classes (one-versus-one). Classification of new instances for one-versus-all case is done by a winner-takes-all strategy, in which the classifier with the highest output function assigns the class. For the one-versus-one approach, classification is done by a max-wins voting strategy, in which every classifier assigns the instance to one of the two classes, then the vote for the assigned class is increased by one vote, and finally the class with most votes determines the instance classification.
With a Gaussian kernel of γ = 1.0
and C = 10
,
the decission boundary of SVM on the toy data is shown as following.
Most machine learning libraries implement SVM with a batch learning algorithm. Smile employs an advanced online learning algorithm, which is very efficient and can update the model with new data easily.
// model is an existing SVM model.
// update the model with new data.
model.learn(x, y)
// polish the model
model.finish
SVMs with Gaussian kernel have similar structure as RBF networks with Gaussian radial basis functions. However, the SVM approach "automatically" solves the network complexity problem since the size of the hidden layer is obtained as the result of the quadratic programming procedure. Hidden neurons and support vectors correspond to each other, so the center problems of the RBF network is also solved, as the support vectors serve as the basis function centers. It was reported that with similar number of support vectors/centers, SVM shows better generalization performance than RBF network when the training data size is relatively small. On the other hand, RBF network gives better generalization performance than SVM on large training data.
A decision tree can be learned by splitting the training set into subsets based on an attribute value test. This process is repeated on each derived subset in a recursive manner called recursive partitioning. The recursion is completed when the subset at a node all has the same value of the target variable, or when splitting no longer adds value to the predictions.
The algorithms that are used for constructing decision trees usually work top-down by choosing a variable at each step that is the next best variable to use in splitting the set of items. "Best" is defined by how well the variable splits the set into homogeneous subsets that have the same value of the target variable. Different algorithms use different formulae for measuring "best". Used by the CART (Classification and Regression Tree) algorithm, Gini impurity is a measure of how often a randomly chosen element from the set would be incorrectly labeled if it were randomly labeled according to the distribution of labels in the subset. Gini impurity can be computed by summing the probability of each item being chosen times the probability of a mistake in categorizing that item. It reaches its minimum (zero) when all cases in the node fall into a single target category. Information gain is another popular measure, used by the ID3, C4.5 and C5.0 algorithms. Information gain is based on the concept of entropy used in information theory. For categorical variables with different number of levels, however, information gain are biased in favor of those attributes with more levels. Instead, one may employ the information gain ratio, which solves the drawback of information gain.
def cart(x: Array[Array[Double]], y: Array[Int], maxNodes: Int, attributes: Array[Attribute] = null, splitRule: DecisionTree.SplitRule = DecisionTree.SplitRule.GINI): DecisionTree
where maxNodes
is the maximum number of leaf nodes in the tree as
a regularization control.
The parameter attributes
provides
the attribute properties so that the algorithm can chooses the proper split
value for attributes. If not provided, all attributes are assumed real-valued.
The following picture is the decision boundary of decision tree with 200 nodes on the toy data.
Decision tree techniques have a number of advantages over many of those alternative techniques.
In most cases, the interpretation of results summarized in a tree is very simple. This simplicity is useful not only for purposes of rapid classification of new observations, but can also often yield a much simpler "model" for explaining why observations are classified or predicted in a particular manner.
Other techniques are usually specialized in analyzing datasets that have only one type of variable.
The final results of using tree methods for classification or regression can be summarized in a series of (usually few) logical if-then conditions (tree nodes). Therefore, there is no implicit assumption that the underlying relationships between the predictor variables and the dependent variable are linear, follow some specific non-linear link function, or that they are even monotonic in nature. Thus, tree methods are particularly well suited for data mining tasks, where there is often little a priori knowledge nor any coherent set of theories or predictions regarding which variables are related and how. In those types of data analytics, tree methods can often reveal simple relationships between just a few variables that could have easily gone unnoticed using other analytic techniques.
One major problem with classification and regression trees is their high variance. Often a small change in the data can result in a very different series of splits, making interpretation somewhat precarious. Besides, decision-tree learners can create over-complex trees that cause over-fitting. Mechanisms such as pruning are necessary to avoid this problem. Another limitation of trees is the lack of smoothness of the prediction surface.
Some techniques such as bagging, boosting, and random forest use more than one decision tree for their analysis.
Random forest is an ensemble classifier that consists of many decision trees and outputs the majority vote of individual trees. The method combines bagging idea and the random selection of features.
Each tree is constructed using the following algorithm:
def randomForest(x: Array[Array[Double]], y: Array[Int], attributes: Array[Attribute] = null, ntrees: Int = 500, maxNodes: Int = -1, nodeSize: Int = 1, mtry: Int = -1, subsample: Double = 1.0, splitRule: DecisionTree.SplitRule = DecisionTree.SplitRule.GINI, classWeight: Array[Int] = null): RandomForest
where ntrees
is the number of trees, and mtry
is the number
of attributed randomly selected at each node to choose the best split.
Although the original random forest algorithm trains each tree fully without pruning,
it is useful to control the tree size some times, which can be achieved by the
parameter maxNodes
. The tree can also be regularized by
limiting the minimum number of observations in trees' terminal nodes with the
parameter nodeSize
.
When subsample = 1.0
, we use the sampling with replacement to train
each tree as described above. If subsample < 1.0
, we instead select
a subset of samples (without replacement) to train each tree. If the classes are
not balanced, the user should provide the classWeight
(proportional
to the class priori) so that the sampling is done in stratified way. Otherwise,
small classes may be not sampled sufficiently.
The following picture is the decision boundary of random forest of 100 decision trees with 200 nodes on the toy data.
The advantages of random forest are:
The disadvantages are
The idea of gradient boosting originated in the observation that boosting can be interpreted as an optimization algorithm on a suitable cost function. In particular, the boosting algorithms can be abstracted as iterative functional gradient descent algorithms. That is, algorithms that optimize a cost function over function space by iteratively choosing a function (weak hypothesis) that points in the negative gradient direction Gradient boosting is typically used with CART regression trees of a fixed size as base learners.
Generic gradient boosting at the t-th step would fit a regression tree to pseudo-residuals. Let J be the number of its leaves. The tree partitions the input space into J disjoint regions and predicts a constant value in each region. The parameter J controls the maximum allowed level of interaction between variables in the model. With J = 2 (decision stumps), no interaction between variables is allowed. With J = 3 the model may include effects of the interaction between up to two variables, and so on. Hastie et al. comment that typically 4 ≤ J ≤ 8 work well for boosting and results are fairly insensitive to the choice of in this range, J = 2 is insufficient for many applications, and J > 10 is unlikely to be required.
Fitting the training set too closely can lead to degradation of the model's generalization ability. Several so-called regularization techniques reduce this over-fitting effect by constraining the fitting procedure. One natural regularization parameter is the number of gradient boosting iterations T (i.e. the number of trees in the model when the base learner is a decision tree). Increasing T reduces the error on training set, but setting it too high may lead to over-fitting. An optimal value of T is often selected by monitoring prediction error on a separate validation data set.
Another regularization approach is the shrinkage which times a parameter η (called the "learning rate") to update term. Empirically it has been found that using small learning rates (such as η < 0.1) yields dramatic improvements in model's generalization ability over gradient boosting without shrinking (η = 1). However, it comes at the price of increasing computational time both during training and prediction: lower learning rate requires more iterations.
def gbm(x: Array[Array[Double]], y: Array[Int], attributes: Array[Attribute] = null, ntrees: Int = 500, maxNodes: Int = 6, shrinkage: Double = 0.05, subsample: Double = 0.7): GradientTreeBoost
where ntrees
is the number of trees or iterations and shrinkage
is the learning rate.
The following picture is the decision boundary of gradient boosted trees of 100 iterations on the toy data.
Soon after the introduction of gradient boosting Friedman proposed a minor modification to the algorithm, motivated by Breiman's bagging method. Specifically, he proposed that at each iteration of the algorithm, a base learner should be fit on a subsample of the training set drawn at random without replacement. Friedman observed a substantial improvement in gradient boosting's accuracy with this modification.
Subsample size is some constant fraction f of the size of the training set. When f = 1, the algorithm is deterministic and identical to the one described above. Smaller values of f introduce randomness into the algorithm and help prevent over-fitting, acting as a kind of regularization. The algorithm also becomes faster, because regression trees have to be fit to smaller datasets at each iteration. Typically, f is set to 0.5, meaning that one half of the training set is used to build each base learner.
Also, like in bagging, sub-sampling allows one to define an out-of-bag estimate of the prediction performance improvement by evaluating predictions on those observations which were not used in the building of the next base learner. Out-of-bag estimates help avoid the need for an independent validation dataset, but often underestimate actual performance improvement and the optimal number of iterations.
In principle, AdaBoost (Adaptive Boosting) is a meta-algorithm, and can be used in conjunction with many other learning algorithms to improve their performance. In practice, AdaBoost with decision trees is probably the most popular combination. AdaBoost is adaptive in the sense that subsequent classifiers built are tweaked in favor of those instances misclassified by previous classifiers. AdaBoost is sensitive to noisy data and outliers. However in some problems it can be less susceptible to the over-fitting problem than most learning algorithms.
AdaBoost calls a weak classifier repeatedly in a series of rounds from total T classifiers. For each call a distribution of weights is updated that indicates the importance of examples in the data set for the classification. On each round, the weights of each incorrectly classified example are increased (or alternatively, the weights of each correctly classified example are decreased), so that the new classifier focuses more on those examples.
def adaboost(x: Array[Array[Double]], y: Array[Int], attributes: Array[Attribute] = null, ntrees: Int = 500, maxNodes: Int = 2): AdaBoost
The following picture is the decision boundary of AdaBoost of 100 trees on the toy data.
The basic AdaBoost algorithm is only for binary classification problem. For multi-class classification, a common approach is reducing the multi-class classification problem to multiple two-class problems. Smile implements a multi-class AdaBoost without such reductions.


Treebank Statistics: UD_Irish-Cadhan: Features: Form
This feature is language-specific.
It occurs with 7 different values: Direct
, Ecl
, Emp
, HPref
, Indirect
, Len
, VF
.
Some words have combined values of the feature; 3 combinations have been observed: Direct|Len
, Ecl|Emp
, Emp|Len
.
845 tokens (18%) have a non-empty value of Form
.
566 types (30%) occur at least once with a non-empty value of Form
.
364 lemmas (33%) occur at least once with a non-empty value of Form
.
The feature is used with 11 part-of-speech tags: NOUN (375; 8% instances), VERB (241; 5% instances), PART (90; 2% instances), PROPN (40; 1% instances), ADJ (31; 1% instances), ADP (22; 0% instances), NUM (21; 0% instances), PRON (12; 0% instances), AUX (11; 0% instances), DET (1; 0% instances), SCONJ (1; 0% instances).
NOUN
375 NOUN tokens (33% of all NOUN
tokens) have a non-empty value of Form
.
The most frequent other feature values with which NOUN
and Form
co-occurred: VerbForm=EMPTY (326; 87%), Number=Sing (275; 73%), Case=Nom (241; 64%), Gender=Masc (211; 56%), Definite=EMPTY (208; 55%).
NOUN
tokens may have the following values of Form
:
Ecl
(117; 31% of non-emptyForm
): bhfearann, bhflaitheas, ndeireadh, bhfear, ccoir, dtaobh, gcomhnaidhe, gcédna, gcéill, n-áitEcl,Emp
(3; 1% of non-emptyForm
): Ndíasa, mbreithirsean, natharsaEmp
(4; 1% of non-emptyForm
): ainmsean, monairc-si, sonsan, tsáoghailsiEmp,Len
(1; 0% of non-emptyForm
): chroidhe-seHPref
(21; 6% of non-emptyForm
): hoidhche, Hiudaighe, Híudaidhe, h-ala, h-anfa, h-éin, haicmeadha, haimsir, haimsire, haithrigheLen
(229; 61% of non-emptyForm
): bheith, fhios, chur, dhéanamh, shaoghail, thoil, thabhairt, thighearna, thús, bhocsa
| Paradigm duine | Ecl | Len |
|---|---|---|
| Case=Dat|Definite=Def|Number=Plur | dhaoinibh | |
| Case=Dat|Number=Plur | dhaoinibh | |
| Case=Gen|Definite=Def|NounType=Strong|Number=Plur | ndáoine | |
| Case=Nom|Definite=Def|Number=Sing | nduine | |
| Case=Nom|Number=Sing | dhuine |
VERB
241 VERB tokens (56% of all VERB
tokens) have a non-empty value of Form
.
The most frequent other feature values with which VERB
and Form
co-occurred: Mood=Ind (197; 82%), Number=EMPTY (184; 76%), Person=EMPTY (172; 71%), Tense=Past (159; 66%).
VERB
tokens may have the following values of Form
:
Direct
(8; 3% of non-emptyForm
): atá, tá, a-ta, átaEcl
(56; 23% of non-emptyForm
): bhfuil, mbeadh, ngairthear, ngoirthear, ttugadh, ttugais, bhfacadar, bhfhuilim, bhfuair, bhfuairsiodEmp,Len
(1; 0% of non-emptyForm
): ghlacadarsanHPref
(1; 0% of non-emptyForm
): háitigheadhLen
(175; 73% of non-emptyForm
): bhí, thug, bheadh, bhíodh, chuir, Dhearc, bhi, chualas, fhuil, fhág
| Paradigm bí | Direct | Ecl | Len |
|---|---|---|---|
| Aspect=Hab|Mood=Ind|Number=Plur|Person=3|Tense=Pres | bhíd | ||
| Aspect=Hab|Mood=Ind|Polarity=Neg|Tense=Pres | bhíonn | ||
| Aspect=Imp|Number=Plur|Person=3|Tense=Past | bhídís | ||
| Aspect=Imp|Tense=Past | mbiodh | bhíodh | |
| Mood=Cnd | mbeadh, mbeath | bheadh | |
| Mood=Cnd|Number=Sing|Person=1 | mbéinn | ||
| Mood=Cnd|Number=Plur|Person=3 | mbéidís | ||
| Mood=Cnd|Polarity=Neg | bheadh, bhiadh | ||
| Mood=Ind|Number=Sing|Person=1|Tense=Past | Bhíos | ||
| Mood=Ind|Number=Plur|Person=1|Tense=Past | Bhamar | ||
| Mood=Ind|PronType=Rel|Tense=Fut | bhias | ||
| Mood=Ind|PronType=Rel|Tense=Pres | atá, tá, a-ta, áta | ||
| Mood=Ind|Tense=Fut | bheidh, bhéas | ||
| Mood=Ind|Tense=Past | bhí, bhi | ||
| Mood=Ind|Tense=Pres | bhfuil, bhfhuilim | fhuil | |
| Mood=Sub|Tense=Pres | mbeith |
PART
90 PART tokens (29% of all PART
tokens) have a non-empty value of Form
.
The most frequent other feature values with which PART
and Form
co-occurred: Polarity=EMPTY (89; 99%), PronType=Rel (87; 97%), PartType=Vb (81; 90%).
PART
tokens may have the following values of Form
:
Direct
(62; 69% of non-emptyForm
): a, do, noch, roDirect,Len
(3; 3% of non-emptyForm
): dhoIndirect
(22; 24% of non-emptyForm
): a, d’á, da, dá, ‘nar, ar, dar, far, lér’, nachLen
(2; 2% of non-emptyForm
): dhoVF
(1; 1% of non-emptyForm
): dob
| Paradigm a | Direct | Direct,Len | Indirect | Len |
|---|---|---|---|---|
| PartType=Inf | dho | |||
| PartType=Vb|PronType=Rel | a, do, noch, ro | dho | a |
PROPN
40 PROPN tokens (18% of all PROPN
tokens) have a non-empty value of Form
.
The most frequent other feature values with which PROPN
and Form
co-occurred: Definite=Def (36; 90%), Number=Sing (31; 78%), Gender=Masc (28; 70%).
PROPN
tokens may have the following values of Form
:
Ecl
(10; 25% of non-emptyForm
): n-Éirinn, bhFailghe, bhFréamhainn, gConnachtaibh, mBaile, n-Áird, nAodh, nAssardha, neabhraHPref
(2; 5% of non-emptyForm
): hAodh, hÉireannLen
(28; 70% of non-emptyForm
): Bheannchair, Dhia, Dhía, Bhuck, Chairbre, Chesar, Chill, Chomhghaill, Chomhghall, Chriosd
| Paradigm Éire | Ecl | HPref |
|---|---|---|
| Case=Dat | n-Éirinn | |
| Case=Gen | hÉireann |
Form
seems to be lexical feature of PROPN
. 93% lemmas (28) occur only with one value of Form
.
ADJ
31 ADJ tokens (15% of all ADJ
tokens) have a non-empty value of Form
.
The most frequent other feature values with which ADJ
and Form
co-occurred: Degree=EMPTY (21; 68%), Number=Sing (21; 68%), Case=Nom (16; 52%).
ADJ
tokens may have the following values of Form
:
HPref
(3; 10% of non-emptyForm
): haereach, haireach, holcLen
(28; 90% of non-emptyForm
): mhór, ghloin, mhaith, shuthoin, bheg, bhig, bhuidhe, bhán, chatharmaigh, cheart
Form
seems to be lexical feature of ADJ
. 100% lemmas (23) occur only with one value of Form
.
ADP
22 ADP tokens (3% of all ADP
tokens) have a non-empty value of Form
.
The most frequent other feature values with which ADP
and Form
co-occurred: Number=Sing (15; 68%), Gender=EMPTY (14; 64%).
ADP
tokens may have the following values of Form
:
Len
(22; 100% of non-emptyForm
): dhe, dho, dhochum, dhom, dhá, dhíom, dhó, dhamh, dhamhsa, dhi
NUM
21 NUM tokens (35% of all NUM
tokens) have a non-empty value of Form
.
The most frequent other feature values with which NUM
and Form
co-occurred: NumType=Card (18; 86%).
NUM
tokens may have the following values of Form
:
Ecl
(3; 14% of non-emptyForm
): naon, náon, ttríLen
(18; 86% of non-emptyForm
): dhá, chéad, mhíle, thrí, cheithre, chúig, dhó, fhichid, sheachtmhoghad, tri
| Paradigm trí | Ecl | Len |
|---|---|---|
| ttrí | thrí, tri |
PRON
12 PRON tokens (6% of all PRON
tokens) have a non-empty value of Form
.
The most frequent other feature values with which PRON
and Form
co-occurred: PronType=EMPTY (11; 92%), Number=Sing (9; 75%), Person=3 (8; 67%), Gender=Masc (7; 58%).
PRON
tokens may have the following values of Form
:
HPref
(8; 67% of non-emptyForm
): hé, hiádLen
(4; 33% of non-emptyForm
): fhéin, mhé, shoin, thú
AUX
11 AUX tokens (14% of all AUX
tokens) have a non-empty value of Form
.
The most frequent other feature values with which AUX
and Form
co-occurred: Polarity=EMPTY (11; 100%), PronType=EMPTY (11; 100%), VerbForm=Cop (11; 100%), Tense=Pres (6; 55%).
AUX
tokens may have the following values of Form
:
Ecl
(1; 9% of non-emptyForm
): mbaVF
(10; 91% of non-emptyForm
): gurab, darab, dárab
| Paradigm is | Ecl | VF |
|---|---|---|
| Tense=Past | mba | gurab, dárab |
| Tense=Pres | gurab, darab |
DET
1 DET tokens (0% of all DET
tokens) have a non-empty value of Form
.
The most frequent other feature values with which DET
and Form
co-occurred: Case=EMPTY (1; 100%), Definite=Def (1; 100%), Gender=EMPTY (1; 100%), Number=EMPTY (1; 100%), Person=EMPTY (1; 100%), Poss=EMPTY (1; 100%), PronType=EMPTY (1; 100%).
DET
tokens may have the following values of Form
:
Ecl
(1; 100% of non-emptyForm
): gach
SCONJ
1 SCONJ tokens (1% of all SCONJ
tokens) have a non-empty value of Form
.
SCONJ
tokens may have the following values of Form
:
Len
(1; 100% of non-emptyForm
): dhá
Relations with Agreement in Form
The 10 most frequent relations where parent and child node agree in Form
:
VERB –[conj]–> VERB (21; 55%),
NOUN –[obl]–> PROPN (2; 67%),
NOUN –[vocative]–> NOUN (2; 67%),
NUM –[nmod]–> NUM (2; 100%),
NUM –[nummod]–> NUM (2; 67%),
PART –[fixed]–> PART (2; 67%),
VERB –[xcomp:pred]–> NUM (2; 100%),
NOUN –[advcl]–> ADJ (1; 100%),
NOUN –[ccomp]–> NOUN (1; 100%),
NOUN –[xcomp:pred]–> PROPN (1; 100%).


public interface ValidatorFactory
Validation rules are handled by validators, which must be registered with the ValidatorFactory (using the registerValidator method). The simplest way to do so is to add a file name validators.xml in the root of the classpath (/WEB-INF/classes) that declares all the validators you intend to use.
INFORMATION validators.xml if being defined should be available in the classpath. However this is not necessary, if no custom validator is needed. Predefined sets of validators will automatically be picked up when defined in com/opensymphony/xwork2/validator/validators/default.xml packaged in in the xwork jar file. See ValidatorFactory static block for details.
WARNING If custom validator is being defined and a validators.xml is created and place in the classpath, do remember to copy all the other pre-defined validators that is needed into the validators.xml as if not they will not be registered. Once a validators.xml is detected in the classpath, the default one (com/opensymphony/xwork2/validator/validators/default.xml) will not be loaded. It is only loaded when a custom validators.xml cannot be found in the classpath. Be careful.
Note:
The default validationWorkflowStack already includes this.
All that is required to enable validation for an Action is to put the
ValidationInterceptor in the interceptor refs of the action (see xwork.xml) like so:
<interceptor name="validator" class="com.opensymphony.xwork2.validator.ValidationInterceptor"/>
Field Validators Field validators, as the name indicate, act on single fields accessible through an action. A validator, in contrast, is more generic and can do validations in the full action context, involving more than one field (or even no field at all) in validation rule. Most validations can be defined on per field basis. This should be preferred over non-field validation wherever possible, as field validator messages are bound to the related field and will be presented next to the corresponding input element in the respecting view.
Non Field Validators Non-field validators only add action level messages. Non-field validators are mostly domain specific and therefore offer custom implementations. The most important standard non-field validator provided by XWork is ExpressionValidator.
NOTE: Non-field validators takes precedence over field validators regardless of the order they are defined in *-validation.xml. If a non-field validator is short-circuited, it will causes its non-field validator to not being executed. See validation framework documentation for more info.
VALIDATION RULES:
Validation rules can be specified:
Here is an example for SimpleAction-validation.xml:
<!DOCTYPE validators PUBLIC "-//Apache Struts//XWork Validator 1.0.3//EN" "http://struts.apache.org/dtds/xwork-validator-1.0.3.dtd"> <validators> <field name="bar"> <field-validator type="required"> <message>You must enter a value for bar.</message> </field-validator> <field-validator type="int"> <param name="min">6</param> <param name="max">10</param> <message>bar must be between ${min} and ${max}, current value is ${bar}.</message> </field-validator> </field> <field name="bar2"> <field-validator type="regex"> <param name="expression">[0-9],[0-9]</param> <message>The value of bar2 must be in the format "x, y", where x and y are between 0 and 9</message> </field-validator> </field> <field name="date"> <field-validator type="date"> <param name="min">12/22/2002</param> <param name="max">12/25/2002</param> <message>The date must be between 12-22-2002 and 12-25-2002.</message> </field-validator> </field> <field name="foo"> <field-validator type="int"> <param name="min">0</param> <param name="max">100</param> <message key="foo.range">Could not find foo.range!</message> </field-validator> </field> <validator type="expression"> <param name="expression">foo lt bar </param> <message>Foo must be greater than Bar. Foo = ${foo}, Bar = ${bar}.</message> </validator> </validators>
Here we can see the configuration of validators for the SimpleAction class. Validators (and field-validators) must have a type attribute, which refers to a name of an Validator registered with the ValidatorFactory as above. Validator elements may also have <param> elements with name and value attributes to set arbitrary parameters into the Validator instance. See below for discussion of the message element.
Each Validator or Field-Validator element must define one message element inside the validator element body. The message element has 1 attributes, key which is not required. The body of the message tag is taken as the default message which should be added to the Action if the validator fails. Key gives a message key to look up in the Action's ResourceBundles using getText() from LocaleAware if the Action implements that interface (as ActionSupport does). This provides for Localized messages based on the Locale of the user making the request (or whatever Locale you've set into the LocaleAware Action). After either retrieving the message from the ResourceBundle using the Key value, or using the Default message, the current Validator is pushed onto the ValueStack, then the message is parsed for \$\{...\} sections which are replaced with the evaluated value of the string between the \$\{ and \}. This allows you to parameterize your messages with values from the Validator, the Action, or both.
If the validator fails, the validator is pushed onto the ValueStack and the message - either the default or the locale-specific one if the key attribute is defined (and such a message exists) - is parsed for ${...} sections which are replaced with the evaluated value of the string between the ${ and }. This allows you to parameterize your messages with values from the validator, the Action, or both.
NOTE: Since validation rules are in an XML file, you must make sure you escape special characters. For example, notice that in the expression validator rule above we use ">" instead of ">". Consult a resource on XML for the full list of characters that must be escaped. The most commonly used characters that must be escaped are: & (use &), > (user >), and < (use <).
Here is an example of a parameterized message:
This will pull the min and max parameters from the IntRangeFieldValidator and the value of bar from the Action.
bar must be between ${min} and ${max}, current value is ${bar}.
Another notable fact is that the provided message value is capable of containing OGNL expressions. Keeping this in mind, it is possible to construct quite sophisticated messages.
See the following example to get an impression:
<message>${getText("validation.failednotice")}! ${getText("reason")}: ${getText("validation.inputrequired")}</message>
| Modifier and Type | Method and Description |
|---|---|
Validator |
getValidator(ValidatorConfig cfg)
Get a Validator that matches the given configuration.
|
String |
lookupRegisteredValidatorType(String name)
Lookup to get the FQ classname of the given validator name.
|
void |
registerValidator(String name,
String className)
Registers the given validator to the existing map of validators.
|
Validator getValidator(ValidatorConfig cfg)
cfg
- the configurator.void registerValidator(String name, String className)
name
- name of validator to add.className
- the FQ classname of the validator.String lookupRegisteredValidatorType(String name)
name
- name of validator to lookup.IllegalArgumentException
- is thrown if the name is not found.Copyright © 2000–2020 Apache Software Foundation. All rights reserved.


HairStep: Transfer Synthetic to Real Using Strand and Depth Maps for Single-View 3D Hair Modeling
CVPR 2023 Highlight
Yujian Zheng1, 2
Zirong Jin2
Moran Li3
Haibin Huang3
Chongyang Ma3
Shuguang Cui2, 1
Xiaoguang Han2, 1*
1FNii, CUHKSZ 2SSE, CUHKSZ 3Kuaishou Technology
*Corresponding author
[Paper]
[Video]
[Code and DataSet]
[Slides]
Given a single portrait image,
we first convert it to an intermediate representation HairStep consisting of a strand map and a depth map (shown in the bottom left and right for each example),
and then recover the corresponding 3D hair model at the strand level.
Our HairStep is capable to bridge the domain gap between synthetic and real data and achieves high-fidelity hair modeling results.
Abstract
In this work, we tackle the challenging problem of learning-based single-view 3D hair modeling.
Due to the great difficulty of collecting paired real image and 3D hair data, using synthetic data to provide prior knowledge for real domain becomes a leading solution.
This unfortunately introduces the challenge of domain gap.
Due to the inherent difficulty of realistic hair rendering, existing methods typically use orientation maps instead of hair images as input to bridge the gap.
We firmly think an intermediate representation is essential, but we argue that orientation map using the dominant filtering-based methods is sensitive to uncertain noise and far from a competent representation.
Thus, we first raise this issue up and propose a novel intermediate representation, termed as HairStep, which consists of a strand map and a depth map.
It is found that HairStep not only provides sufficient information for accurate 3D hair modeling, but also is feasible to be inferred from real images.
Specifically, we collect a dataset of 1,250 portrait images with two types of annotations.
A learning framework is further designed to transfer real images to the strand map and depth map.
It is noted that, an extra bonus of our new dataset is the first quantitative metric for 3D hair modeling.
Our experiments show that HairStep narrows the domain gap between synthetic and real and achieves state-of-the-art performance on single-view 3D hair reconstruction.
Overview
Overview of our approach. (a) The pipeline of single-view 3D hair modeling with our novel representation HairStep.
We collect two datasets HiSa and HiDa,
and propose effective approaches for HairStep extraction from real images and finally realize high-fidelity 3D hair strand reconstruction.
(b) Domain-adaptive depth estimation.
We first pre-train the Hourglass on synthetic dataset,
then generate depth priors as pseudo labels and finally obtain reasonable hair depth weakly-supervised by depth prior and annotated relative depth.
(c) Annotation of HiSa and HiDa.
Quantitative Comparisons
Our HairStep benefits different frameworks on both synthetic data (left table) and real data (right table).
Acknowledgements:
Website template is borrowed from gan_steerability.


Hospital dresses newborns as Mister Rogers in honor of World Kindness Day
by Kyle HicksPITTSBURGH, Penn. – Fred Rogers was known for his unwavering kindness, so a hospital in Pennsylvania decided to honor him in a special way on World Kindness Day.
Wednesday, the staff at Allegheny Health Network dressed their newborns up as the star of “Mister Rogers’ Neighborhood.”
In adorable photos shared by the hospital system, babies are seen sporting Rogers’ signature red cardigans and ties.
WQED, the PBS station that produced “Mister Rogers,” also dubbed Wednesday Cardigan Day as a nod to Rogers’ favorite sweater.
The TV station is encouraging the public to express their kindness by wearing a cardigan sweater in the spirit of Rogers.


In this weeks independent project we are supposed to get the users birthday and make the application calculate the day of the week they were born in reference to their Akan name.
The akan day names are written as follows:
Akan names. The Akan people of Ghana frequently name their children after the day of the week they were born and the order in which they were born. ... During the 18th–19th centuries, slaves in the Caribbean from the region that is modern-day Ghana were referred to as Coromantees.
FILL ME!


Interface CurveDefinition
-
- All Known Subinterfaces:
NodalCurveDefinition
- All Known Implementing Classes:
InterpolatedNodalCurveDefinition
,ParameterizedFunctionalCurveDefinition
public interface CurveDefinition
Provides the definition of how to calibrate a curve.A curve is built from a number of parameters and described by metadata. Calibration is based on a list of
CurveNode
instances that specify the underlying instruments.
-
-
Method Summary
All Methods Instance Methods Abstract Methods Default Methods Modifier and Type Method Description Curve
curve(LocalDate valuationDate, CurveMetadata metadata, DoubleArray parameters)
Creates the curve from an array of parameter values.CurveDefinition
filtered(LocalDate valuationDate, ReferenceData refData)
Returns a filtered version of this definition with no invalid nodes.CurveName
getName()
Gets the curve name.ImmutableList<CurveNode>
getNodes()
Gets the nodes that define the curve.int
getParameterCount()
Gets the number of parameters in the curve.ValueType
getYValueType()
Gets the y-value type, providing meaning to the y-values of the curve.ImmutableList<Double>
initialGuess(MarketData marketData)
Gets the list of all initial guesses.CurveMetadata
metadata(LocalDate valuationDate, ReferenceData refData)
Creates the curve metadata.default CurveParameterSize
toCurveParameterSize()
Converts this definition to the summary form.
-
-
-
Method Detail
-
getName
CurveName getName()
Gets the curve name.- Returns:
- the curve name
-
getParameterCount
int getParameterCount()
Gets the number of parameters in the curve.This returns the number of parameters in the curve, which is not necessarily equal the size of
getNodes()
.- Returns:
- the number of parameters
-
getYValueType
ValueType getYValueType()
Gets the y-value type, providing meaning to the y-values of the curve.This type provides meaning to the y-values. For example, the y-value might represent a zero rate, as represented using
ValueType.ZERO_RATE
.- Returns:
- the y-value type
-
getNodes
ImmutableList<CurveNode> getNodes()
Gets the nodes that define the curve.The nodes are used to calibrate the curve. If the objective curve is a nodal curve, each node is used to produce a parameter in the final curve.
- Returns:
- the nodes
-
filtered
CurveDefinition filtered(LocalDate valuationDate, ReferenceData refData)
Returns a filtered version of this definition with no invalid nodes.A curve is formed of a number of nodes, each of which has an associated date. To be valid, the curve node dates must be in order from earliest to latest. Each node has certain rules,
CurveNodeDateOrder
, that are used to determine what happens if the date of one curve node is equal or earlier than the date of the previous node.Filtering occurs in two stages. The first stage looks at each node in turn. The previous and next nodes are checked for clash. If clash occurs, then one of the two nodes is dropped according to the clash action "drop" values. The second stage then looks again at the nodes, and if there are still any invalid nodes, an exception is thrown.
This approach means that in most cases, only those nodes that have fixed dates, such as futures, need to be annotated with
CurveNodeDateOrder
.- Parameters:
valuationDate
- the valuation daterefData
- the reference data- Returns:
- the resolved definition, that should be used in preference to this one
- Throws:
IllegalArgumentException
- if the curve nodes are invalid
-
metadata
CurveMetadata metadata(LocalDate valuationDate, ReferenceData refData)
Creates the curve metadata.This method returns metadata about the curve and the curve parameters.
For example, a curve may be defined based on financial instruments. The parameters might represent 1 day, 1 week, 1 month, 3 months, 6 months and 12 months. The metadata could be used to describe each parameter in terms of a
Period
.The optional parameter-level metadata will be populated on the resulting metadata. The size of the parameter-level metadata will match the number of parameters of this curve.
- Parameters:
valuationDate
- the valuation daterefData
- the reference data- Returns:
- the metadata
-
curve
Curve curve(LocalDate valuationDate, CurveMetadata metadata, DoubleArray parameters)
Creates the curve from an array of parameter values.The meaning of the parameters is determined by the implementation. The size of the array must match the count of parameters.
- Parameters:
valuationDate
- the valuation datemetadata
- the curve metadataparameters
- the array of parameters- Returns:
- the curve
-
toCurveParameterSize
default CurveParameterSize toCurveParameterSize()
Converts this definition to the summary form.The
CurveParameterSize
class provides a summary of this definition consisting of the name and parameter size.- Returns:
- the summary form
-
initialGuess
ImmutableList<Double> initialGuess(MarketData marketData)
Gets the list of all initial guesses.This returns initial guess for the curve parameters. The valuation date is defined by the market data.
- Parameters:
marketData
- the market data required to build a trade for the instrument, including the valuation date- Returns:
- the initial guess
-
-


About Me
Hello! Thank you for taking the time to grace my page with your eyes. My name is Thomas Musler (Though
you probably noticed that from the header). I'm a full stack develper in training!
Personally I'm somone that enjoys detail oriented work and anything that resembles a good puzzle. I have
a background in management and team leadership. To keep those abilities sharp over the years I have
often used my personal time to advance my leadership skills though weekly team building exercises
involving problem solving,
conflict resolution training, strategic goal planning and social bonding, all while encouraging skills
in data figuring, team work, time management, and listening skills while in a diverse team setting.
In terms of programming, I've grown confortable with HTML5, CSS, JavaScript, NodeJS, NPM(and its various
applications), as well as MySql, Mongo, Heroku. I have an active understanding of current responsive web
design and an open and inquiring mind for when new skills can be learned!
Don't just take my work for it, go ahead and peruse my works featured here, as well as my GitHub and
LinkedIn.
Hello! Thank you for taking the time to grace my page with your eyes. My name is Thomas Musler (Though you probably noticed that from the header). I'm a full stack develper in training!
Personally I'm somone that enjoys detail oriented work and anything that resembles a good puzzle. I have a background in management and team leadership. To keep those abilities sharp over the years I have often used my personal time to advance my leadership skills though weekly team building exercises involving problem solving, conflict resolution training, strategic goal planning and social bonding, all while encouraging skills in data figuring, team work, time management, and listening skills while in a diverse team setting.
In terms of programming, I've grown confortable with HTML5, CSS, JavaScript, NodeJS, NPM(and its various applications), as well as MySql, Mongo, Heroku. I have an active understanding of current responsive web design and an open and inquiring mind for when new skills can be learned!
Don't just take my work for it, go ahead and peruse my works featured here, as well as my GitHub and LinkedIn.


{% extends 'layout.html' %} {% block pageTitle %} Example - passing data from page to page {% endblock %} {% block content %}
CSV, XML, JSON, EXCEL
" }) }}We recommend SNOMED codes are used when uploading data to the screening platform. Any codes that we don't recognise will need to be manually mapped.


Earthquakes are one of the deadliest natural disasters in the world that can cause thousands of deaths and millions homeless in an instance. Unfortunately, some countries do not have a proper disaster mitigation plan in place when an earthquake strikes. Often, victims get trapped inside buildings and do not get help in time.
We here at Concat101 has come-up with a rescue robot that can detect movement and send a response to the website. Wolverine, the rescue robot is remote controlled and will be placed in areas that a person will not be able to enter and detect movement within a 3m to 5m radius.
Wolverine is equipped with a PIR motion sensor that has an operating voltage range of 4.5 to 20V and a lens size sensor of 0.23mm in diameter. The board dimensions is 32mm by 24mm and can be placed on top of a small remote controlled car.
The technologies used for this project are: HTML, CSS, JavaScript, jQuery, C#, C++, ASP.NET, Bootstrap, JSON, and Particle Electron Kit.
About Our teamHC-SR501 uses an infrared technology that is based on a Germany imported LHI778 design. It has a sensing range of less than 120 degrees and covers a radius of 7 meters
The material of the prototype model will offer protection against dust, shock, vibration, and blowing rain. It is not meant for underwater however, it can be immersed for up to 20 minutes.
We use a Lithium-ion (Li-ion) battery based on 2 factors: weight and energy capacity. The Li-ion batteries are lighter than your standard Alkaline batteries and has a high energy density.
The camera will be implemented in future model pending the success of our current model. It will be equipped with night vision for monitoring in the dark and will transmit data via cellular data.
0 = static 1 = incoming signal
The PIR sensor is initially static and monitors for any signal within a 3-meter radius. Once a signal is picked up, it will be delayed for 5 seconds and will send the signal to the cloud using cellular data. The signal will then be uploaded to the laptop and be converted into readable data. If the curve goes up, it indicates there is movement in the sensor. If it rests, there is no movement. The graph is real-time as it collects data every second


Updates
Change in Roster Arrangement for Alliance Quests
Posted on July 26, 2016
Due to the fact that we have been unsuccessful to 100% alliance quests recently, the officers and I have decided to change the way the roster is organized.
Veteran players, or players who are shown to be more active/skillful, will now be designated to the first 2 groups of AQ. Newer/not so active players will be designated to group 3.
By having our top players in the 2 groups, we greatly increase our chances of receiving Alliance Quest crystals for all members and getting a chance to get T4 catalysts, which we all love and pray for.
If you find that you are in group 3, it is not meant as a form of punishment. It is meant to prove your worth to this alliance. We still expect you to perform your best just as we would with any member. It does not give you an excuse to be lazy and not participate. If you don’t like it, then feel free to leave any time.
Alternatively, members in groups 1 & 2 that are found to be lacking in participation/skill, will be bumped down to group 3, giving a group 3 member a chance to be in the top 2 groups.
Our hope is to someday 100% all 3 Alliance Quests in all 3 groups. This will show us that this alliance is moving on up to the east side. Which, in turn, will allow us to open 3 groups for war, helping guarantee all members a chance to win more shards.
Until that day comes though, this is the way it will be for now.
Thank you.
Discord Game Chat App for Communication
Posted on July 18, 2016
In an attempt to help communication with our alliance, we will now start requiring members to download Discord.
This is an online game chat app available on android, iOS, as well as desktop devices.
Once you have downloaded Discord., search for and add the username: GeeShepherd#2311 (Capitalization matters)
Once you have done so, GeeShepherd will shortly accept your friend request and send you a link to the alliance chat.
Thank you.
Participation in AQ for 3 War Groups
Posted on July 7, 2016
We realize that now that we have started to move up in rank for war, people may want to join war, but can't, because slots have been taken up.
We want to open up all 3 slots in war, but we will only do so until we get almost full participation in all Alliance Quests.
If we can't get full participation in Alliance Quests, then it's a sign we won't get full participation in Alliance Wars, thus making us lose a war and missing out on winning shards.
So until we start getting full participation in all Aliiance Quests consitently, we will continue to do 2 groups for war until further notice.
Thank you.
Possible 3 groups for war
Posted on June 20, 2016
Over the next 3 AQs, we expect members to follow the roster for AQs.
If everyone follows the roster well and each group explores most/all the map, then we plan on opening up war to 3 groups, so everyone can have the chance to win war rewards.
However, this depends on the cooperation of everyone in the alliance. If we find that there is not good participation and that members aren’t following the roster, then we will stay with 2 groups for wars until otherwise.
Thank you.
Website Up and Running
Posted on June 15, 2016
Today we are releasing our new website. http://tiny.cc/coulsonmcoc
From here, players can find rules, updates, spreadsheets, and helpful resources to become stronger.
We will also now be implementing a roster, where certain players will have designated spots for certain groups in alliance wars/quests.
To find out which group you are assigned to, simply go to the Roster page and find your name on the list. We expect all members to follow this roster. Those who continually don’t will be kicked.
Notice that in Alliance Wars, we have given some of our top veteran members reserved spots in each group. All other spots are open to other players. This is because currently, we only do 2 groups for war.
However, if players follow the roster and do well in AQs, then we hope to open 3 groups for war in the future.
Some new changes
Posted on June 8, 2016
Due to the fact that we have not been reaching the minimum requirements for Summoner Advancement, and have therefor suffered by not receiving our weekly dose of 4* shards, the officers and I have decided that we will now be doing 3 wars (2 groups) a week to make up for it.
We also plan on doing AQ 3 once a week, AQ 2 while not in war (or in placement phase), and AQ 1 during war.
We also plan on implementing a roster sometime in the future to better improve our efficiency in completing AQ and AW. So certain players will be assigned to certain groups in AQ and AW. But that is still in the works.
/pWe know these are quite a few changes, so if you have any questions, just ask.


Reports conditional expressions which can be replaced by simpler but equivalent expressions.
Example
→
Replacement
condition ? true : false
condition
condition ? false : true
!condition
value == null ? null : value
value
result != 0 ? result : 0
result
a == b ? a : b
b


Nitro is a heavily opinionated template for building Serverless applications with
NestJS, the fastest-growing
NodeJS framework in the world by github stars.
Database support
with Azure Table Storage.
File Upload
with Azure Blob Storage.
Hosting
with Azure Functions.
Built natively in TypeScript a Nitro-Stack NestJS application gives you the confidence of Type safety with the productivity of JavaScript.
Nitro applications are Serverless, they scale automatically and you only pay for what you use. From the application hosting, to file-upload to the database support, all parts of a Nitro stack auto-scale.
A NestJS application sits on either express or fastify (you choose) with access to both platforms diverse ecosystem of modules and packages
No need to choose upfront, a NestJS application allows you to expose entities as either a series of Rest endpoints or a single GraphQL endpoint.
Your gateway to over 100 Azure cloud services.
A Nitro application is just a NestJS application with Serverless support on Azure. So to begin create a NestJS application
npm i -g @nestjs/cli
nest new nitro-cats
cd nitro-cats
nest start
To deploy as a serverless application on Azure you first need to add the
nest add @nestjs/azure-func-http
module, this converts your NestJS application to a
Serverless Azure Function.
npm i -g @nestjs/cli
nest new nitro-cats
cd nitro-cats
nest start
We need to build a home for our application in the Cloud, to do that we use Hexa. Hexa is a CLI tool to help you easily create and manage resources on the Azure platform.
npm install --global @manekinekko/hexa
You will need an account on the Azure platform, you can get one for free
hexa login
Then create our home ❤ in the cloud.
hexa init --just=functions
Finally deploy our application to the cloud!
hexa deploy


--- layout: post title: It's been a while... date: 2009-12-13 13:55:31.000000000 +01:00 categories: [] tags: [] status: publish type: post published: true meta: _edit_last: '5236424' author: login: stricksucht email<EMAIL_ADDRESS>display_name: stricksucht first_name: '' last_name: '' ---
I realize that I haven't been blogging in, like, months, but little has been going on here - so what should I have been blogging about?
Ever since I knitted the Grace Lace Beret I kind of developed a thing for hats. Slouchy berets look surprisingly good on me, so now I'm knitting hats whenever I feel the urge to. Plus they're like 24h projects, so I can challenge them whenever I don't want to knit anything else. Also, many of my friends came and asked if I could knit a hat for them, so things have been pretty hat-ty around here ;)
And pretty stressful, I can tell you. There've been a lot of parties, and girls night outs, and game nights, and between studying the whole day and kind of partying all night, there's not much room left for knitting. (Of course I'm a little exaggerating on the studying-all-day-partying-all-night-thing, but it comes pretty close to the truth and it *really* wears me out ;))
So what's the status? I'm putting the finishing touches to Conny's Seascape Stole where I'm halfway through Chart C. (It's about time, since it still has to be a Christmas gift!) The Owls Sweater is at something like 80% or so, I'll just have to finish the second sleeve and then I'll be able to knit the owls - finally!! :D then there's the Snowboard Hat I'm knitting for a friend of mine, plus all the other WIPs I don't even dare to think about ;) I think I'll just return to the Seascape Stole now - I intend to have it finished by the end of the day.


| Methods' Details |
insertAtPosition
| void |
| insertAtPosition( |
[in] ::com::sun::star::table::CellAddress |
aDestPos, |
| [in] string |
aFileName, |
| [in] string |
aSourceArea, |
| [in] string |
aFilter, |
| [in] string |
aFilterOptions ); |
- Description
- creates an area link and adds it to the collection.
- Parameter aDestPos
- the address of the first cell of the range inside the current
document.
- Parameter aFileName
- the URL of the source document.
- Parameter aSourceArea
- the name of the range in the source document.
This can be the address of a cell or range in the form
"Sheet1.A1:C5", or the name of a named range or database range.
- Parameter aFilter
- the name of the filter used to load the source document.
- Parameter aFilterOptions
- optional filter options for the specified filter.
|
|
removeByIndex
| void |
| removeByIndex( |
[in] long |
nIndex ); |
- Description
- removes an area link from the collection.
|
|
Top of Page
Copyright © 2013, The Apache Software Foundation, Licensed under the Apache License, Version 2.0. Apache, the Apache feather logo, Apache OpenOffice and OpenOffice.org are trademarks of The Apache Software Foundation. Other names may be trademarks of their respective owners.


Heap data structure is created on the concept of complete binary tree. The height of the tree can be calculated as the height of the root. The height of the node in a tree is referred to the number of edges on the simple longest downward path from the node to the leaf.
The heap of size n with the height h has the following characteristics:
• The height of heap is log n.
• The total number of leaves in the heap are n/2.
• The number of nodes of the height h is less than equal to.
Use induction method to show that there are the at most nodes with height h in any kind of heap of n - element.
For the heap size of n which is greater than zero, the number of leaves is , assume that tree is nearly a complete binary tree.
Base case:
• The number of leaves at the height of h is equal to zero or the number of leaves in the heap of n element at height is zero.
• The parent of last node will be node at , it is the last parent for the heap. It means, after this node all nodes will be counted as leaves.
• Hence the count of number of leaves is equal to the , which shows that the base case is completely true.
Now suppose that at the height of number of nodes is given by formula.
If all the leaves are removed from the existing heap, nodes which are placed at the height 1 will be counted as the leaves in new heap and have the zero height.
In the same way all nodes for the new one tree will have one less than height from the old height. Use it in the induction step.
Induction:
• Consider that tree Told of n-node, which have the Nh number of node at the height of h and Tnew is the formed tree after removing the leaves from Told tree. The tree Tnew has the nodes as: .
• The number of nodes at the height of h in Told would be same at the height of the in the Tnew tree. Assume that represents the number of node at the height of in Tnew tree.
Hence it is prove that there are the at most nodes with height h in any kind of heap of n -element.


Cliff surprised Melissa after returning home from a Wedding with a decorated gazebo that he lead her to under false pretenses. It was there that he asked her to marry her.


Stromberg's top models, with their colossal, nineteen-inch-wide bodies, provided the tremendous volume and projection needed for a rhythm guitarist to be heard in the large jazz orchestras of the 1940s. Charles and Elmer, father and son, worked together in Boston, beginning in the early 1930s. Though they competed for business with John D'Angelico in New York, their total output was only 636 instruments, a little more than half the number made by D'Angelico.
NMM 10863 reflects Stromberg's post-1940 style, with a single diagonal top brace and carved, rather than pressed, arching. The Master 400 was their top grade instrument, with a decoratively engraved and painted headstock veneer and maximum body size. Both Charles and Elmer died in 1955, as musical tastes were changing; rock and roll and country had begun to overshadow big band jazz as America's most popular musical styles.
Soundboard: arched, two-piece, quarter-cut spruce: wide grain. Back: two-piece maple cut off-the-quarter: medium, horizontal curl; wide grain. Ribs: two-piece, slab-cut maple with faint blister figure. Head: maple with black celluloid over white celluloid veneer; model and signature engraved in headstock and painted ivory and gold. Neck: two-piece maple; integral with head; maple of inlay along center line with brown-stained maple strips on each side.
Binding: white celluloid; trim comprised of five-ply (black-white-black-white-black) celluloid purfling strips on top and two-ply (black-white) celluloid purfling strips on back and ribs. Soundholes: f-holes bound in two-ply black and white celluloid purfling strip trim.
Heel cap: five-ply thin black-white-black-white-black celluloid layers. Fingerboard: ebony bound in ivoroid with five-ply (black-white-black-white-black) celluloid purfling strip trim on top sides, two-ply black-white celluloid purfling strip on lower end and sides; end terminating in a point, with rounded edges on the sides; 20 nickel-silver frets; mother-of-pearl block behind 15th fret; split mother-of-pearl blocks behind 1st, 5th, 7th, 9th, and 12th frets; black celluloid side dots behind 3rd, 5th, 7th, 9th, 12th, and 15th frets.
Bridge: two-piece rosewood (compensated) with brass screws for height adjustment; triangular mother-of-pearl inlays on each bridge foot. Tuners: six Kluson gold-plated brass worm-gear machine tuners with tulip-shaped, yellow, marbleized celluloid heads.
Endpin: white celluloid. Pick guard: translucent imitation tortoise-shell celluloid bound in white celluloid with five-ply (black-white-black-white-black) celluloid purfling strip trim with stair-step outline on treble side, mounted on guitar with gold-plated brass bracket and three gold-plated steel screws; bracket possibly later. Lacquer: dark red-brown sunburst.


5 Books of Moses
19 Books of the Prophets
11 Other Writings
The 4 Gospels
1 Book
21 Letters
Final Book


Locale Codes
Language Locale Codes
You can specify a Language Locale Code to customize web pages on supported websites. With SeleniumBase you can change the web browser's Locale on the command line by adding:
--locale=CODE # Example: --locale=ru
List of Language Locale Codes:
| Language | Code |
|---|---|
| Afrikaans | af |
| Amharic | am |
| Arabic | ar |
| Arabic (Egypt) | ar_ |
| Arabic (Saudi Arabia) | ar_ |
| Basque | eu |
| Belarusian | be |
| Bengali | bn |
| Bulgarian | bg |
| Catalan | ca |
| Chinese | zh |
| Chinese (China Mainland) | zh_ |
| Chinese (Hong Kong) | zh_ |
| Chinese (Taiwan) | zh_ |
| Croatian | hr |
| Czech | cs |
| Danish | da |
| Dutch | nl |
| English | en |
| English (United States) | en_ |
| English (Australia) | en_ |
| English (Canada) | en_ |
| English (United Kingdom) | en_ |
| English (Ireland) | en_ |
| English (India) | en_ |
| English (Singapore) | en_ |
| English (South Africa) | en_ |
| Estonian | et |
| Farsi | fa |
| Filipino | fil |
| Finnish | fi |
| French | fr |
| French (Canada) | fr_ |
| French (Switzerland) | fr_ |
| Galician | gl |
| German | de |
| German (Austria) | de_ |
| Greek | el |
| Gujarati | gu |
| Hebrew | he |
| Hindi | hi |
| Hungarian | hu |
| Icelandic | is |
| Indonesian | id |
| Italian | it |
| Japanese | ja |
| Kannada | kn |
| Korean | ko |
| Lao | lo |
| Latvian | lv |
| Lingala | ln |
| Lithuanian | lt |
| Malay | ms |
| Malayalam | ml |
| Marathi | mr |
| Norwegian | no |
| Polish | pl |
| Portuguese | pt |
| Portuguese (Brazil) | pt_ |
| Portuguese (Portugal) | pt_ |
| Romanian | ro |
| Russian | ru |
| Serbian | sr |
| Slovak | sk |
| Slovenian | sl |
| Spanish | es |
| Spanish (Latin America) | es_ |
| Spanish (Argentina) | es_ |
| Spanish (Chile) | es_ |
| Spanish (Colombia) | es_ |
| Spanish (Costa Rica) | es_ |
| Spanish (Dominican Rep.) | es_ |
| Spanish (Ecuador) | es_ |
| Spanish (El Salvador) | es_ |
| Spanish (Guatemala) | es_ |
| Spanish (Honduras) | es_ |
| Spanish (Mexico) | es_ |
| Spanish (Nicaragua) | es_ |
| Spanish (Panama) | es_ |
| Spanish (Peru) | es_ |
| Spanish (Puerto Rico) | es_ |
| Spanish (Paraguay) | es_ |
| Spanish (United States) | es_ |
| Spanish (Uruguay) | es_ |
| Spanish (Venezuela) | es_ |
| Swahili | sw |
| Swedish | sv |
| Swiss German | gsw |
| Tagalog | tl |
| Tamil | ta |
| Telugu | te |
| Thai | th |
| Turkish | tr |
| Ukrainian | uk |
| Urdu | ur |
| Vietnamese | vi |
| Zulu | zu |