Write a function that takes two character vectors as its left and right arguments and returns 1
if they are anagrams of each other. An anagram of a string uses all of the letters of the string ignoring word spacing, capitalisation, and punctuation.
'anagram' your_function 'Nag A Ram'
1
'Dyalog APL' your_function 'Dog Pay All'
1
'' your_function ' !#!'
1
'abcde' your_function 'zyxwvu'
0
Write a function that takes a numeric vector and returns the length of the longest streak of positive growth.
your_function 1 2 3 4 5 6 7 8 9
8
your_function 1
0
your_function 9 8 7 6 5 4
0
your_function 1 5 3 4 2 6 7 8
3
In mathematics, the Farey_sequence of order n is the sequence of completely reduced fractions between 0 and 1 which, when in lowest terms, have denominators less than or equal to n, arranged in order of increasing size. Each Farey sequence starts with the value 0, denoted by the fraction 0⁄1, and ends with the value 1, denoted by the fraction 1⁄1.
Write a function that takes an integer right argument and returns a vector of the terms in the Farey sequence of that order. Each element in the returned vector is itself a 2-element vector of numerator and denominator for the corresponding term.
your_function 0
┌───┐
│0 1│
└───┘
your_function 1
┌───┬───┐
│0 1│1 1│
└───┴───┘
your_function 5
┌───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───┐
│0 1│1 5│1 4│1 3│2 5│1 2│3 5│2 3│3 4│4 5│1 1│
└───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┘
The index-of function X⍳Y
returns a simple integer vector that identifies where the major cells of Y
are first found in X
. If a major cell of Y
cannot be found in X
, then the corresponding item of the returned array will be 1+≢X
.
Example of X⍳Y
:
'DYALOG APL' ⍳ 'AAALLLB'
3 3 3 4 4 4 11
Progressive dyadic iota is similar to ⍳
except that it returns the index of subsequent matches in the left argument until they are exhausted. Write a function that implements progressive dyadic iota.
'DYALOG APL' your_function 'AAALLLB'
3 8 11 4 10 11 11
'' your_function 'test' ⍝ should work with empty left argument
1 1 1 1
⍬≡'test' {your_function} '' ⍝ should work with empty right argument
1
The standard deviation of a population is calculated by taking the square root of the average of the squared differences of the values from their average value. The mathematical formula is:
where x̄ is the mean average value and n is the number of values.
Write a function that returns the population standard deviation of its numeric array right argument.
your_function 1 2 3 4 5
1.414213562
your_function 10 10⍴⍳100 ⍝ and higher dimensions as well
28.86607005
Write a function that takes, as each of its right and left arguments, a pair of co-ordinates representing diagonally opposite corners of two rectangles and returns a 1
if the rectangles overlap. The co-ordinates could be either (upper-left, lower-right) or (upper-right, lower left).
(1 1)(5 5) your_function (3 3)(7 7)
1
(1 1)(5 5) your_function (5 5)(1 1)
1
(1 1)(3 3) your_function (4 4)(7 7)
0
(1.1 1.1)(5.5 5.5) your_function (3.3 3.3)(4.4 4.4)
1
Membership X∊Y
returns a boolean array of shape ⍴X
with 1
s indicating where elements of Y
occur in X
. For a vector X
this results in a very convenient boolean mask.
In many instances, it is desirable to perform case-insensitive comparisons and operations. Write a function to perform case-insensitive membership between two arrays.
For full marks, consider membership of cells in high-rank arrays, your function should still return a vector when comparing cells of the same rank, for example (3 2⍴⎕A
) and (1 2⍴'cd'
).
'dyalog' your_function 'APL'
0 0 1 1 0 0
'bramley' your_function 'HAMPSHIRE'
0 1 1 1 0 1 0
You’ve just been hired by a new company and you’ve learned that the owner has triskaidekaphobia (fear of the number 13). As such he’s issued a corporate mandate that no output from any program shall include the number 13 – 12.99 will be used instead. Your boss wants you to implement a function to process output and change the 13s to 12.99. For now, you need only concern yourself with numeric data and not worry about 13s that are formatted in text.
your_function 13
12.99
your_function ⍳15
1 2 3 4 5 6 7 8 9 10 11 12 12.99 14 15
your_function 6+5 6 7 8 9
11 12 12.99 14 15
your_function 13 130 13.13 1300 ⍝ we only care about the number 13
12.99 130 13.13 1300
There is an urban myth about some research that was supposedly done at Cambridge University that purports changing the order of the interior letters in the words of a sentence does not significantly hamper the readability of the sentence.
Put another way…
Teher is an ubran mtyh aobut smoe rseaecrh taht was spuopesldy dnoe at Cmarbdige Uinevsrtiy taht pruoptrs cahgnnig the odrer of the itnreoir lteetrs in the wrods of a sneetcne deos not sgiinifactnly hmaepr the raeadibilty of the sneetcne.
In the above example, for words that contain more than 3 letters, the first and last letters remain the same while all of the interior letters are transposed in groups of two, ignoring punctuation. If there are an odd number of letters in the interior of the word, then the last letter is left as it is.
Write a function that takes a character vector word as its left argument and returns the word’s letters juxtaposed as described above.
your_function 'argument'
agrmunet
]box on
your_function¨'this' 'is' 'awesome'
┌────┬──┬───────┐
│tihs│is│aewosme│
└────┴──┴───────┘
]box off
your_function¨'this' 'is' 'awesome'
tihs is aewosme
The APL expression to compute Pascal’s triangle of order n
is fairly simple.
{{⍉⍵∘.!⍵}0,⍳⍵} 10
1 0 0 0 0 0 0 0 0 0 0
1 1 0 0 0 0 0 0 0 0 0
1 2 1 0 0 0 0 0 0 0 0
1 3 3 1 0 0 0 0 0 0 0
1 4 6 4 1 0 0 0 0 0 0
1 5 10 10 5 1 0 0 0 0 0
1 6 15 20 15 6 1 0 0 0 0
1 7 21 35 35 21 7 1 0 0 0
1 8 28 56 70 56 28 8 1 0 0
1 9 36 84 126 126 84 36 9 1 0
1 10 45 120 210 252 210 120 45 10 1
However, the output leaves something to be desired. Write a function that takes an integer right argument representing the order of the Pascal’s triangle to be created and returns a “nicely” formatted character matrix. Because we’re using a mono-spaced font and can only format whole character positions, lines do not have to be perfectly centered and may vary by a character position.
your_function 10
1
1 1
1 2 1
1 3 3 1
1 4 6 4 1
1 5 10 10 5 1
1 6 15 20 15 6 1
1 7 21 35 35 21 7 1
1 8 28 56 70 56 28 8 1
1 9 36 84 126 126 84 36 9 1
1 10 45 120 210 252 210 120 45 10 1
your_function 1
1
1 1
your_function 0
1


"""Incremental Principal Components Analysis."""
# Author: Kyle Kastner<EMAIL_ADDRESS># Giorgio Patrini
# License: BSD 3 clause
import numpy as np
from scipy import linalg
from .base import _BasePCA
from ..utils import check_array, gen_batches
from ..utils.extmath import svd_flip, _incremental_mean_and_var
class IncrementalPCA(_BasePCA):
"""Incremental principal components analysis (IPCA).
Linear dimensionality reduction using Singular Value Decomposition of
centered data, keeping only the most significant singular vectors to
project the data to a lower dimensional space.
Depending on the size of the input data, this algorithm can be much more
memory efficient than a PCA.
This algorithm has constant memory complexity, on the order
of ``batch_size``, enabling use of np.memmap files without loading the
entire file into memory.
The computational overhead of each SVD is
``O(batch_size * n_features ** 2)``, but only 2 * batch_size samples
remain in memory at a time. There will be ``n_samples / batch_size`` SVD
computations to get the principal components, versus 1 large SVD of
complexity ``O(n_samples * n_features ** 2)`` for PCA.
Read more in the :ref:`User Guide <IncrementalPCA>`.
Parameters
----------
n_components : int or None, (default=None)
Number of components to keep. If ``n_components `` is ``None``,
then ``n_components`` is set to ``min(n_samples, n_features)``.
whiten : bool, optional
When True (False by default) the ``components_`` vectors are divided
by ``n_samples`` times ``components_`` to ensure uncorrelated outputs
with unit component-wise variances.
Whitening will remove some information from the transformed signal
(the relative variance scales of the components) but can sometimes
improve the predictive accuracy of the downstream estimators by
making data respect some hard-wired assumptions.
copy : bool, (default=True)
If False, X will be overwritten. ``copy=False`` can be used to
save memory but is unsafe for general use.
batch_size : int or None, (default=None)
The number of samples to use for each batch. Only used when calling
``fit``. If ``batch_size`` is ``None``, then ``batch_size``
is inferred from the data and set to ``5 * n_features``, to provide a
balance between approximation accuracy and memory consumption.
Attributes
----------
components_ : array, shape (n_components, n_features)
Components with maximum variance.
explained_variance_ : array, shape (n_components,)
Variance explained by each of the selected components.
explained_variance_ratio_ : array, shape (n_components,)
Percentage of variance explained by each of the selected components.
If all components are stored, the sum of explained variances is equal
to 1.0.
singular_values_ : array, shape (n_components,)
The singular values corresponding to each of the selected components.
The singular values are equal to the 2-norms of the ``n_components``
variables in the lower-dimensional space.
mean_ : array, shape (n_features,)
Per-feature empirical mean, aggregate over calls to ``partial_fit``.
var_ : array, shape (n_features,)
Per-feature empirical variance, aggregate over calls to
``partial_fit``.
noise_variance_ : float
The estimated noise covariance following the Probabilistic PCA model
from Tipping and Bishop 1999. See "Pattern Recognition and
Machine Learning" by C. Bishop, 12.2.1 p. 574 or
http://www.miketipping.com/papers/met-mppca.pdf.
n_components_ : int
The estimated number of components. Relevant when
``n_components=None``.
n_samples_seen_ : int
The number of samples processed by the estimator. Will be reset on
new calls to fit, but increments across ``partial_fit`` calls.
Notes
-----
Implements the incremental PCA model from:
`D. Ross, J. Lim, R. Lin, M. Yang, Incremental Learning for Robust Visual
Tracking, International Journal of Computer Vision, Volume 77, Issue 1-3,
pp. 125-141, May 2008.`
See http://www.cs.toronto.edu/~dross/ivt/RossLimLinYang_ijcv.pdf
This model is an extension of the Sequential Karhunen-Loeve Transform from:
`A. Levy and M. Lindenbaum, Sequential Karhunen-Loeve Basis Extraction and
its Application to Images, IEEE Transactions on Image Processing, Volume 9,
Number 8, pp. 1371-1374, August 2000.`
See http://www.cs.technion.ac.il/~mic/doc/skl-ip.pdf
We have specifically abstained from an optimization used by authors of both
papers, a QR decomposition used in specific situations to reduce the
algorithmic complexity of the SVD. The source for this technique is
`Matrix Computations, Third Edition, G. Holub and C. Van Loan, Chapter 5,
section 5.4.4, pp 252-253.`. This technique has been omitted because it is
advantageous only when decomposing a matrix with ``n_samples`` (rows)
>= 5/3 * ``n_features`` (columns), and hurts the readability of the
implemented algorithm. This would be a good opportunity for future
optimization, if it is deemed necessary.
References
----------
D. Ross, J. Lim, R. Lin, M. Yang. Incremental Learning for Robust Visual
Tracking, International Journal of Computer Vision, Volume 77,
Issue 1-3, pp. 125-141, May 2008.
G. Golub and C. Van Loan. Matrix Computations, Third Edition, Chapter 5,
Section 5.4.4, pp. 252-253.
See also
--------
PCA
RandomizedPCA
KernelPCA
SparsePCA
TruncatedSVD
"""
def __init__(self, n_components=None, whiten=False, copy=True,
batch_size=None):
self.n_components = n_components
self.whiten = whiten
self.copy = copy
self.batch_size = batch_size
[docs] def fit(self, X, y=None):
"""Fit the model with X, using minibatches of size batch_size.
Parameters
----------
X : array-like, shape (n_samples, n_features)
Training data, where n_samples is the number of samples and
n_features is the number of features.
y : Passthrough for ``Pipeline`` compatibility.
Returns
-------
self : object
Returns the instance itself.
"""
self.components_ = None
self.n_samples_seen_ = 0
self.mean_ = .0
self.var_ = .0
self.singular_values_ = None
self.explained_variance_ = None
self.explained_variance_ratio_ = None
self.singular_values_ = None
self.noise_variance_ = None
X = check_array(X, copy=self.copy, dtype=[np.float64, np.float32])
n_samples, n_features = X.shape
if self.batch_size is None:
self.batch_size_ = 5 * n_features
else:
self.batch_size_ = self.batch_size
for batch in gen_batches(n_samples, self.batch_size_):
self.partial_fit(X[batch], check_input=False)
return self
[docs] def partial_fit(self, X, y=None, check_input=True):
"""Incremental fit with X. All of X is processed as a single batch.
Parameters
----------
X : array-like, shape (n_samples, n_features)
Training data, where n_samples is the number of samples and
n_features is the number of features.
check_input : bool
Run check_array on X.
Returns
-------
self : object
Returns the instance itself.
"""
if check_input:
X = check_array(X, copy=self.copy, dtype=[np.float64, np.float32])
n_samples, n_features = X.shape
if not hasattr(self, 'components_'):
self.components_ = None
if self.n_components is None:
self.n_components_ = n_features
elif not 1 <= self.n_components <= n_features:
raise ValueError("n_components=%r invalid for n_features=%d, need "
"more rows than columns for IncrementalPCA "
"processing" % (self.n_components, n_features))
else:
self.n_components_ = self.n_components
if (self.components_ is not None) and (self.components_.shape[0] !=
self.n_components_):
raise ValueError("Number of input features has changed from %i "
"to %i between calls to partial_fit! Try "
"setting n_components to a fixed value." %
(self.components_.shape[0], self.n_components_))
# This is the first partial_fit
if not hasattr(self, 'n_samples_seen_'):
self.n_samples_seen_ = 0
self.mean_ = .0
self.var_ = .0
# Update stats - they are 0 if this is the fisrt step
col_mean, col_var, n_total_samples = \
_incremental_mean_and_var(X, last_mean=self.mean_,
last_variance=self.var_,
last_sample_count=self.n_samples_seen_)
# Whitening
if self.n_samples_seen_ == 0:
# If it is the first step, simply whiten X
X -= col_mean
else:
col_batch_mean = np.mean(X, axis=0)
X -= col_batch_mean
# Build matrix of combined previous basis and new data
mean_correction = \
np.sqrt((self.n_samples_seen_ * n_samples) /
n_total_samples) * (self.mean_ - col_batch_mean)
X = np.vstack((self.singular_values_.reshape((-1, 1)) *
self.components_, X, mean_correction))
U, S, V = linalg.svd(X, full_matrices=False)
U, V = svd_flip(U, V, u_based_decision=False)
explained_variance = S ** 2 / (n_total_samples - 1)
explained_variance_ratio = S ** 2 / np.sum(col_var * n_total_samples)
self.n_samples_seen_ = n_total_samples
self.components_ = V[:self.n_components_]
self.singular_values_ = S[:self.n_components_]
self.mean_ = col_mean
self.var_ = col_var
self.explained_variance_ = explained_variance[:self.n_components_]
self.explained_variance_ratio_ = \
explained_variance_ratio[:self.n_components_]
if self.n_components_ < n_features:
self.noise_variance_ = \
explained_variance[self.n_components_:].mean()
else:
self.noise_variance_ = 0.
return self


This website will be ready soon! We are planning this event for September. 2018! Click the button to sign up for more information.
Or learn more about the event here


A section requires at least a space inside curly brackets { } in order to be managed differently from an horizontal rule.
A first text.
A second text.
"If debugging is the process of removing software bugs,
then programming must be the process of putting them in."
Any content
Next.
Or paragraphs.
End of sections.


contact info
<EMAIL_ADDRESS>778-456-7891
business hours
Monday-Friday:9am to 5 pm
Saturday:10am to 2pm
Sunday:Closed


?'
š
9
10
Copyright
reserved
The
New
York
Botanical
Garden
ÇMUied
j,
AHEŸICAN
THE
NEW
YORK
BOTANICAL
GARDEN
EXPLORATION
OF
SANTO
DOMINGO
CONSUELO,
PROVINCE
OF
MACORIS
.....tal.
/Ji’’U
....
IktylM-
Jf—&L_____
lÈ¡È€AL
Qarden
N.
Taylor,
Collector
Nov,
1909
DET.
BY
MILDRED
E.
MATHIAS
Sih
Legend -
Level of confidence that token is an accurately-transcribed word
extremely low
very low
low
undetermined
medium
high
very high


Contents
2.5.1)
StatsAgg application configuration
2.5.2)
StatsAgg database configuration
<IP_ADDRESS>)
Apache Derby Embedded
<IP_ADDRESS>)
Apache Derby vs MySQL
2.5.3)
StatsAgg WebUI security configuration (optional)
<IP_ADDRESS>)
Sampling-based counters
3.3.3)
StatsD legacy namespacing
3.3.4) Other
differences between StatsD & StatsAgg
3.3.5)
Sample StatsAgg configuration file compared to a StatsD configuration file
4.2) What
happens to triggered alerts when StatsAgg is restarted?
5)
Administrative Website User Interface
5.1) Normal
‘path’ through the website
5.4.1)
Notification Groups table
5.4.2) Create
Notification Group
5.6.1)
Alerts Suspensions table
5.6.2)
Create Alert Suspension
6) Tips
& recommended usage patterns.
StatsAgg is a metric aggregation and alerting platform. It currently accepts Graphite-formatted metrics and StatsD formatted metrics.
StatsAgg works by receiving Graphite & StatsD metrics, aggregating them, alerting on them, and outputting the aggregated metrics to a Graphite-compatible metric storage platform. The diagram (linked to below) shows a typical deployment & use-case pattern for StatsAgg.
· A complete re-implementation of StatsD
a. TCP & UDP support (both can run concurrently).
b. Support for all metric-types.
· Receives (and optionally, aggregates) Graphite metrics
a. TCP & UDP support (both can run concurrently).
b. Aggregates into minimum, average, maximum for the values of an individual metric-path (during an 'aggregation window').
· Outputs aggregated metrics to Graphite (and Graphite compatible services)
· A robust alerting mechanism
a. Can alert off of any received/aggregated metric.
b. Regular-expression based mechanism for tying together metrics & alerts.
c. 'threshold' based alerting, or 'availability' based alerting.
d.
e. Alerts notifications can be sent via email, or viewed in the StatsAgg website.
· A web-based UI for managing alerts & metrics
StatsAgg was originally written to fill some gaps that in some other popular open-source monitoring tools. Specifically...
· Graphite and StatsD do not have a native alerting mechanism
a. Most alerting solutions for StatsD and/or Graphite metrics are provided by (expensive) SaaS venders.
b. The alerting mechanism in StatsAgg compares favourably to many pay-based solutions.
· Provides an alternative way of managing servers/services/etc compared to tools like Nagios, Zabbix, etc
a. StatsAgg allows you to break away from viewing everything from the perspective of servers/hosts. You can structure your metrics in such a way so as to group everything by host, but you aren't required to.
b. Generally speaking, tools like Nagios, Zabbix, etc lack the ability to alert off of free-form metrics. Since StatsAgg uses regular-expressions to tie metrics to alerts, you can just as easily alert off of a 'free-form metric hierarchy' as you can a 'highly structured metric hierarchy'.
c. Tools like Nagios, Zabbix, etc aren't built around having fine datapoint granularity (more frequent than once per minute). StatsAgg was written to be able to receive, aggregate, alert on, and output metrics that are sent at any interval.
· StatsD limitations
a. StatsD doesn't allow the TCP server & the UDP server to be active at the same time. StatsAgg does.
b. StatsD doesn’t persist Gauge metrics, so a restart will result of StatsD ‘forgetting’ what the previous Gauge metric values were. StatsAgg can persists Gauge values, so a restart won't result in Gauges resetting themselves.
· StatsAgg provides a web-based UI for managing alerts & metrics.
· Performance
a. StatsAgg is Java-based, and has been thoroughly tuned for performance.
b. A server with 2 cpu cores & 4 gigabytes of RAM should have no trouble processing thousands of metrics per second.
Note: The installation/deployment process for StatsAgg is somewhat complex (compared to StatsD). A simpler/streamlined installer for StatsAgg is planned for a future release.
· Oracle Java 1.7+ JRE (64bit recommended)
· Tomcat 7.0.x
§ Other versions of Tomcat may work, but are untested. Likewise, other application servers may work (Weblogic, JBoss, etc), but are untested.
· 1 CPU core for smaller StatsAgg servers. 4 CPU cores (or more) for heavy-use StatsAgg servers.
· 3GB RAM for smaller StatsAgg servers. 16GB of RAM (or more) for larger StatsAgg servers.
· 1.2GB hard drive space. The main StatsAgg deployment (along with Tomcat) will take less than 100MB, but StatsAgg is configured to retain up to 1.1GB of log file data.
· Any OS capable of running Oracle Java 1.7 & Tomcat 7.
§ Windows XP & higher (though using a recent release of Windows is highly recommended)
§ Almost all recent versions of Linux
1. Install an Oracle 1.7+ Java JDK
· Section 5.3 discusses this in more detail.
2. Install the latest version of Apache Tomcat 7 using the MSI installer. The following non-default configuration settings are recommended:
· Check: Install the APR based native .dll (recommended, not required)
· Check: Service Startup (recommended, not required)
· Set: HTTP/1.1 connector port = 80 (recommended, not required)
3. (Optional, Recommended) Add these ‘Java options’ to Tomcat through the Tomcat configuration application.
· More on this in section 6.4.
· Xmx (max heap size) is discussed in more detail in section 6.4. -Xmx2560m is merely a suggestion.
· Remote JMX options can be added to allow tools like VisualVM to connect. If you don’t intend on using tools like VisualVM, then don’t set the jmxremote options.
-Xmx2560m
-XX:+UnlockCommercialFeatures
-XX:+UseConcMarkSweepGC
-Dcom.sun.management.jmxremote=true
-Dcom.sun.management.jmxremote.port=20000
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
4. Configure Tomcat
· See section 2.4.
5. Deploy StatsAgg to Tomcat
· Stop the “Apache Tomcat 7.0 Tomcat7” service
· Create a folder in (Tomcat_Home)/webapps called “StatsAgg”
· Unzip the StatsAgg file into (Tomcat_Home)/webapps/StatsAgg
6. Configure the application.properties file & the database.properties file (see section 2.5)
· Be especially mindful to properly configure the database.properties file. If this this isn’t configured properly, StatsAgg won’t function properly.
7. Start
the “Apache Tomcat 7.0 Tomcat7” service.
Because installation instructions vary widely based on distro, only high-level instructions will be provided for Linux deployments.
1. Install an Oracle 1.7+ Java JDK
2. Install Tomcat 7
a. Configure Tomcat to explicitly use the version of Java that was installed in step 1, or change the OS path to use JDK installed in step 1 as the default JDK.
3. Configure Tomcat
a. (Optional, Recommended) Set Java options.
i. Create a file -- (Tomcat_Home)/bin/setenv.sh
· Make sure the file can be read by the user that Tomcat will run as
ii.
Add this line into the file:
export JAVA_OPTS="-Xmx2560m -XX:+UnlockCommercialFeatures -XX:+UseConcMarkSweepGC
-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=20000 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"
· More on this in section 6.4.
· Xmx (max heap size) is discussed in more detail in section 6.4. -Xmx2560m is merely a suggestion.
· Remote JMX options can be added to allow tools like VisualVM to connect. If you don’t intend on using tools like VisualVM, then don’t set the jmxremote options.
· If you have Tomcat configured to read JAVA_OPTS from some other location, then set the JAVA_OPTS there.
b. Make the changes to server.xml discussed in section 2.4.
4. Deploy StatsAgg to Tomcat 7.
a. Stop Tomcat. On many systems, this can be done with “sudo service tomcat7 stop”.
b. Create a folder in (Tomcat_Home)/webapps called “StatsAgg”
c. Unzip the StatsAgg war file into (Tomcat_Home)/webapps/StatsAgg
5. Configure the application.properties file & the database.properties file (see section 2.5)
a. Be especially mindful to properly configure the database.properties file. If this file isn’t configured properly, StatsAgg may not function properly.
6. Start Tomcat. On many systems, this can be done with “sudo service tomcat7 start”.
In the (Tomcat_Home)/conf/server.xml
file, change the HTTP connection to look like this:
…
<!-- port should be set to whatever port want to
interact with the StatsAgg WebUI
on -->
<Connector port="80"
protocol="HTTP/1.1"
connectionTimeout="20000"
compression="on"
useSendfile="false"
compressableMimeType="text/html,text/xml,application/javascript,text/css,application/x-font-woff"
compressionMinSize="2048"
redirectPort="8443" />
…
For StatsAgg to function properly, a minimum number of configuration settings must be set. This documentation will point out the most critical fields that need to be set. All settings are documented in the configuration files.
StatAgg’s application configuration file usually requires minimal configuration. That said, StatsAgg still expects to see a user-maintained application configuration file (application.properties).
1. Edit
“application.properties” @ (tomcat-location)\webapps\StatsAgg\WEB-INF\config
Example location (Windows): C:\Program Files\Apache Software Foundation\Tomcat
7.0\webapps\StatsAgg\WEB-INF\config\application.properties
·
You can override the default configuration file
location by adding a Java argument to Tomcat: -DsaAppConfLocation="YourLocation"
Example (Windows): -DsaAppConfLocation="C:\StatsAgg_Conf\application.properties"
2. Put any values in this file that you want to deviate from the default.
The full list settings, default values, and examples usages can be found in StatsAgg\WEB-INF\config\example_application.properties
Some common settings to put in the application.properties file include:
· Output to Graphite
o graphite_output_module_1 = true,some-graphite-server.some-domain.com,2003,3
·
Enabling email alerts
o
alert_send_email_enabled
= true
o alert_statsagg_location = (url of your StatsAgg deployment. Ex: http://some-domain.com/StatsAgg)
o
Fill in the alert_smtp_*
settings with the details of your SMTP server.
·
Prefix StatsAgg
output. This groups all StatsAgg output together
in Graphite via a top-level directory.
o global_metric_name_prefix_enabled = true
o
global_metric_name_prefix_value
= statsagg
StatsAgg uses a relational database for storing StatsD Gauge values, metric group info, alert info, etc. The database is NOT used for storage of metric data (Graphite/InfluxDB/etc is used for this). StatsAgg supports two different database technologies: Apache Derby Embedded & MySQL.
StatsAgg expects to see a user-created database configuration file (database.properties). If one is not specified, then StatsAgg will use an ephemeral (in-memory) database. While functional, the ephemeral database will lose all its data as soon as StatsAgg is shutdown or restarted. For users that don’t care about StatsAgg’s alerting capabilities & just want to use StatsAgg as a StatsD metric aggregator, this mode should work just fine. Please note that when StatsAgg fails to connect to a database that you have assigned to it, it may instead attempt to use ephemeral database.
This is the default database type, and the simplest to configure. The advantages of using Apache Derby are…
· No separate sever needed.
· The engine runs inside the StatsAgg JVM, so there is 0 query latency.
· Minimal configuration required.
· Upgrades to the database engine are embedded into the StatsAgg deployment.
· No worries about losing database connectivity.
The disadvantages of using this database type are…
· Due to engine limitations, this database will perform worse than MySQL when processing large numbers of Gauge metrics.
· You can’t connect to the database from an external client while StatsAgg is running. This is a limitation inherit in Apache Derby Embedded.
· Less community support than MySQL.
· Backup mechanisms are less robust.
· The database engine uses up JVM heap space, leaving less room for the rest StatsAgg’s data.
If you choose to use Apache Derby, then the steps to configure StatsAgg to use it are…
1. Edit
“database.properties” @ (tomcat-location)\webapps\StatsAgg\WEB-INF\config
Example location (Windows): C:\Program Files\Apache Software Foundation\Tomcat
7.0\webapps\StatsAgg\WEB-INF\config\database.properties
·
You can override the default configuration file
location by adding a Java argument to Tomcat: -DsaDbConfLocation="YourLocation"
Example (Windows): -DsaDbConfLocation="C:\StatsAgg_Conf\database.properties"
2. Create a folder somewhere on your StatsAgg server for the database data. StatsAgg must have read/write permissions to this folder. In the database.properties file, assign the location of this folder to the variable ‘db_localpath’.
· Failure to perform this step may lead to Apache Derby attempting to create the database at a location of its choosing (ex – at the root of your filesystem).
3. The
settings listed below are the recommended set of database.properties
settings for Apache Derby Embedded.
#database.properties settings
cp_max_connections = 50
cp_acquire_retry_attempts = 3
cp_acquire_retry_delay = 250
cp_connection_timeout = 5000
cp_enable_statistics = false
cp_default_auto_commit = false
connection_validity_check_timeout = 5
# Set db_localpath to the location created in step
#2. Example: C:\\StatsAgg\\Data
# Set the username/password to whatever values you deem
appropriate.
db_type = derby_embedded
db_localpath =
db_name = statsagg_db
db_username =
db_password =
db_attributes = create=true;upgrade=true;
derby.system.home = ${db_localpath}
derby.infolog.append = true
derby.stream.error.file =
${db_localpath}${file.separator}${db_name}${file.separator}log${file.separator}db_log.txt
derby.storage.pageSize = 16384
derby.storage.pageCacheSize = 15000
A full list settings, default values, and examples usages can be found in StatsAgg\WEB-INF\config\example_database.properties
MySQL can be used instead of Apache Derby. MySQL is recommended for more customers that require a more enterprise-friendly database. The advantages of MySQL are…
· Faster than Apache Derby at processing amounts of StatsD Gauge metrics.
· Database processing can be performed on a separate server from the StatsAgg server. This has the overall effect freeing up resources on the StatsAgg server.
· You can connect to the database from an external client while StatsAgg is running.
· Backup mechanisms are robust.
The disadvantages of using this database type are…
· Runs as a separate process from StatsAgg. If the database goes down (or becomes unavailable to StatsAgg), StatsAgg will not function properly.
o Without a restart, StatsAgg doesn’t guarantee a complete recovery from a database connectivity issue. It is recommended that StatsAgg be restarted if there is a MySQL connectivity loss.
· Complicated configuration (compared to Apache Derby).
If you choose to use MySQL, then the steps to configure StatsAgg to use it are…
1. Install/configure MySQL 5.6+
· Percona Server 5.6+ & MariaDB 10+ are supported.
·
MySQL’s configuration can get really complex.
The MySQL configuration settings listed below are recommended to ensure that StatsAgg functions properly. Note that there are MANY other
settings that can be set on MySQL to achieve faster performance, greater
durability, etc, but those are out-of-scope for this
documentation.
#In my.ini on Windows. On Linux, set in my.cnf.
autocommit = 0
innodb_file_format = Barracuda
innodb_file_format_max = Barracuda
innodb_file_per_table = ON
innodb_strict_mode = ON
max_allowed_packet = 64M
sql_mode = STRICT_ALL_TABLES
2. Connect to MySQL & run “create database statsagg;”.
3. Create appropriate usernames, passwords, and privileges on the MySQL server. Make sure that the user that the StatsAgg application connects as has full grants/privileges on the ‘statsagg’ database that was created in step 1.
4. Edit
“database.properties” @ (tomcat-location)\webapps\StatsAgg\WEB-INF\config
Example location (Windows): C:\Program Files\Apache Software Foundation\Tomcat
7.0\webapps\StatsAgg\WEB-INF\config\database.properties
·
You can override the default configuration file
location by adding a Java argument to Tomcat: -DsaDbConfLocation="YourLocation"
Example (Windows): -DsaDbConfLocation="C:\StatsAgg_Conf\database.properties"
5. The
settings listed below are the recommended set of database.properties
settings for MySQL.
#database.properties settings
cp_max_connections = 50
cp_acquire_retry_attempts = 3
cp_acquire_retry_delay = 250
cp_connection_timeout = 1000
cp_enable_statistics = false
cp_default_auto_commit = false
connection_validity_check_timeout = 1
# Set the hostname/port to point to the server that MySQL runs on. If running
MySQL on the same server as StatsAgg, use db_hostname=<IP_ADDRESS>
# Set the username/password to whatever values you deem appropriate.
db_type = mysql
db_hostname =
db_port = 3306
db_name = statsagg
db_username =
db_password =
db_attributes = autoReconnect=true&failOverReadOnly=false&maxReconnects=1
Notes
· A full list settings, default values, and examples usages can be found in StatsAgg\WEB-INF\config\example_database.properties
· StatsAgg was only tested against MySQL 5.6. However, it should work on MySQL 5.5+ (the author has no intentions of testing on any platform lower than MySQL 5.6).
· StatsAgg expects/requires a low-latency/high-bandwidth connection to the MySQL database. If such a connection isn’t available, then it is recommended that Apache Derby Embedded be used instead.
The author of StatsAgg generally recommends Derby over MySQL. Derby is simpler to setup, has adequate performance for almost all use-cases, and is incapable of being unavailable (at least independently of the application), and is more portable. MySQL is generally only recommended for users that want to standardize around a MySQL tech-stack (which may make backups easier, etc), or are trying to achieve higher performance (depending on the use-case, MySQL can be faster).
The initial release plans for StatsAgg did not include any form of security for the WebUI. However, since Tomcat provides a mechanism for relatively easy/simple authentication, StatsAgg can be configured to use authentication.
This can be accomplished via the following steps:
1. Open “(tomcat-location)\webapps\StatsAgg\WEB-INF\web.xml” in a text editor
a. Uncomment the <security-constraint> & <login-config> sections
2. Open “(tomcat-location)\conf\tomcat-users.xml” in a text editor
a.
Add the following roles & users into the
<tomcat-users> section. Substitute your own passwords.
<role
rolename="statsagg-admin"/>
<user
username="admin" password="somePW_1" roles="statsagg-admin"/>
<role rolename="statsagg-read-only"/>
<user username="guest" password="somePW_2" roles="statsagg-read-only"/>
3. Restart Tomcat
Notes
· Without securing the wire traffic, Tomcat’s authentication mechanism is not particularly secure. Ideally, HTTP traffic should be SSL encrypted. It is up to the user to configure this.
· StatsAgg currently does not provide a ‘logout’ mechanism. Usually closing & reopening your browser will accomplish this.
· Authentication is NOT enabled by default
StatsAgg supports multiple metric formats, and this documentation tries to use the naming convention that is appropriate for each metric-type. This makes the documentation easier for people that know their specific metric format, but it also means that the same thing can go by several different names. Listed below are some of the naming conventions used.
‘The identifier of a metric’
StatsD calls this a ‘bucket’
Graphite calls this a ‘metric-path’
Many metric storage engines call this a ‘metric-series’
StatsAgg refers to this as a ‘metric-key’
‘Metric-key + metric-value’
This combination is often referred to as a ‘datapoint’.
This combination is sometimes referred to as a ‘metric’
Graphite is a popular tool for visualizing & storing time-series data. Among open-source tools that provide this functionality, Graphite was a pioneer. Many other tools are compatible with the Graphite format, so providing native support for Graphite metrics was a core design goal in StatsAgg.
By default, StatsAgg will listen for Graphite metrics on ports 2003 & 2004 (TCP & UDP, on both ports). Port 2003 is used for Graphite ‘passthrough’ metrics, and port 2004 is used for Graphite ‘aggregated’ metrics. For more information on the distinction between ‘passthrough’ and ‘aggregated’, please read sections 3.2.3 & 3.2.4.
The Graphite format is: metricPath metricValue unixEpochTime\n
Example: some.descriptive.name.for.a.metric<PHONE_NUMBER>334418\n
Notes:
· Every individual metric MUST end with a newline character.
· StatsAgg can handle metric values of almost any size & scale, with a few limitations.
o Graphite may not be able to handle extremely large and/or small numbers.
o ‘Aggregated’ Graphite metrics will round to 7 decimal places.
· Sending multiple metrics in a single TCP or UDP push is allowed, but every individual metric must end in a newline character.
o Example: some.description.name.for.a.metric1<PHONE_NUMBER>418\nsome.description.name.for.a.metric2<PHONE_NUMBER>418\n
o Sending more than a few metrics per UDP push is not recommended (due to limitations in the UDP protocol).
· In the actual Graphite program, several characters are not allowed in metric-paths. Some examples include (but are not limited to): ‘%’, ‘\’, ‘/’, ‘[‘, ‘]’, ‘{‘, ‘}’, ‘ ‘.
o It is up to the program sending the Graphite metrics to use an ‘allowed’ character set. StatsAgg will not reformat/correct metrics that use characters that don’t play well with Graphite.
· In the actual Graphite program, the ‘.’ character is used to create folders between the metric-path descriptors.
o This is only relevant if you intend on outputting your metrics to Graphite for visualization.
o Example: folder1.folder2.value1<PHONE_NUMBER>418\n folder1.folder2.value2<PHONE_NUMBER>418\n
o folder1
- folder2
* value1=123
* value2=456
· StatsAgg doesn’t know/care that the ‘.’ character is used for creating folder hierarchies, nor does it know/care about other ‘special’ characters. These are distinctions that matter to the actual Graphite program, but not to StatsAgg.
By default, port 2003 is used for Graphite ‘passthrough’ metrics. The inputs to ‘passthrough’ Graphite metrics are normal/plain Graphite metrics. The ‘passthrough’ aspect refers to how StatsAgg handles the metrics. ‘Passthrough’ means that StatsAgg performs no aggregation on the metrics. All metrics that are received as inputs are also capable of being outputted. In other words, every metric that is sent to StatsAgg is simply ‘passing through’.
Why use ‘passthrough’ Graphite metrics?
· You don’t benefit from aggregating metrics if you send Graphite metrics less frequently than the StatsAgg ‘flush interval’.
· Aggregating Graphite metric uses far more CPU resources on StatsAgg servers than ‘passthrough’ Graphite metrics does.
· Graphite ‘passthrough’ metrics are the most lightweight metric type that StatsAgg is capable of processing.
· StatsAgg’s built-in alerting mechanism can alert on Graphite ‘passthrough’ metrics.
Notes:
· In the actual Graphite program, metrics that have the same metric-path & timestamp overwrite each other. StatsAgg doesn’t know/care about this, and if metrics are sent with the same metric-path & timestamp, StatsAgg will pass all the metrics through.
· After a restart, StatsAgg forgets metric-paths for Graphite metrics. As a result, StatsAgg won’t be capable of outputting the previous value for a metric-path until it receives at least one new datapoint for that metric-path (assuming that StatsAgg is configured to output the previous value for a metric-path that hasn’t recently had a new value).
By default, port 2004 is used for Graphite ‘aggregated’ metrics. The inputs to ‘aggregated’ Graphite metrics are normal/plain Graphite metrics. The ‘aggregated’ aspect refers to how StatsAgg handles the metrics. ‘Aggregated’ means that StatsAgg will aggregate the Graphite metrics that are sent to it.
It works as follows:
1) StatsAgg receives Graphite metrics.
2) Every ‘flush interval’ (10 seconds, by default), StatsAgg will aggregate the Graphite metrics.
a. The values from identical metric-paths are aggregated together.
b. StatsAgg will output the following aggregated metrics: Average Value, Minimum Value, Maximum Value
c. The average value will round to 7 decimal places.
d. The timestamps are averaged together.
3) Only the aggregated values are capable of being alerted on & outputted. The original metric values are discarded after they are aggregated.
Example:
folder1.folder2.value1 4<PHONE_NUMBER>\n
folder1.folder2.value1 2<PHONE_NUMBER>\n
folder1.folder2.value1 8<PHONE_NUMBER>\n
folder1.folder2.value1 10<PHONE_NUMBER>\n
folder1.folder2.value1 6<PHONE_NUMBER>\n
is aggregated into…
folder1.folder2.value1.Min 2<PHONE_NUMBER>\n
folder1.folder2.value1.Max 10<PHONE_NUMBER>\n
folder1.folder2.value1.Avg 6<PHONE_NUMBER>\n
Why use ‘aggregated’ Graphite metrics?
· In the actual Graphite program, metrics that have the same metric-path & timestamp overwrite each other. Aggregating metrics in StatsAgg preserves the range of values for the metric-path that were sent during the ‘flush interval’.
o Warning: if the current ‘averaged metric timestamp’ for a metric-path is the same as a previously outputted ‘averaged metric timestamp’, then the new aggregated metrics will overwrite the previous aggregated metrics in the actual Graphite application.
· StatsAgg’s built-in alerting mechanism can alert on the aggregated metrics.
· Example use-case: you send a metric-path, such as CPU %, very frequently (ex, once per second). You want to know the range of values that were sent, but you don’t want to store every individual datapoint.
Notes:
· After a restart, StatsAgg forgets metric-paths for Graphite metrics. As a result, StatsAgg won’t be capable of outputting the previous value for a metric-path until it receives at least one new datapoint for that metric-path (assuming that StatsAgg is configured to output the previous value for a metric-path that hasn’t recently had a new value).
StatsD is a popular protocol/program for sending/aggregating data. While StatsD has a time-series component to it, the protocol does not send timestamps. StatsD is oriented toward sending business metrics & leaving it up to the metric receiver (the official StatsD app, StatsAgg, etc) to make sense of them. By default, StatsAgg will listen for StatsD metrics on ports 8125 (TCP & UDP, on both ports).
StatsD works as follows:
1) StatsAgg receives StatsD metrics.
2) Every ‘flush interval’ (10 seconds, by default), StatsAgg will aggregate the StatsD metrics.
a. The values from identical metric-paths are aggregated together (the aggregation logic will depend on their metric type).
b. All aggregations that involve division will round to 7 decimal places.
c. The timestamps are averaged together.
3) The aggregated metrics are made available to StatsAgg’s alerting & outputting routines.
A good-faith effort was made to accurately re-implement the StatsD protocol in StatsAgg. However, there are implementation differences that users should be aware of. Please read the metric format section for more details.
The metric format closely mirrors the format of the official StatsD application.
The official StatsD documentation can be found here: https://github.com/etsy/statsd/blob/master/docs/metric_types.md
General format: bucket:metricValue|statsdMetricType\n
Notes:
· StatsD calls the name of a metric a “bucket” (instead of ‘metric-path’, ‘metric key’, etc)
· Best practice is to end metrics in a ‘\n’ character.
o This is required for sending metrics via TCP.
o This is not required for sending individual UDP metrics, but newlines are required for sending multiple metrics in a single UDP post.
· There are minor implementation differences between how the official StatsD application works compared to StatsAgg. Please read notes for each metric type for details.
Below is a list of StatsD formats that StatsAgg supports.
Format: bucket:metricValue|c
Example:
myBucket:15.5|c
myBucket:3|c
myBucket:-9|c
… outputs (assuming a ‘flush interval’ of 10 seconds):
myBucket.count = 9.5
myBucket.rate = .95
This is a simple counter. The input values are added together at each ‘flush interval’, and then output. Two outputs are generated: a count (which is the sum of the aggregated values), and a per-second rate (count/’flush interval’).
The counts & sums from different ‘flush interval’ windows are NOT aggregated together. Only metrics that arrive during the current ‘flush interval’ are aggregated together. If you want StatsAgg to aggregate metrics over multiple flush intervals, then you must use a ‘gauge’ metric.
count & rate reset to 0 after each aggregation.
Note:
· StatsAgg does support the StatsD ‘legacy namespacing’ convention, but it is not enabled by default. Please consult the application configuration file for more details.
· If no new metrics are received by StatsAgg for a bucket during a flush interval, StatsAgg can be configured to output either ‘0’ or nothing.
o The default behavior of StatsAgg is to send ‘0’.
o After a restart, StatsAgg forgets bucket names for counter metrics. As a result, if StatsAgg is configured to output ‘0’ for a counter-metric that has had a new value, StatsAgg won’t begin outputting ‘0’ again until this bucket receives at least one new datapoint.
o After a restart, StatsAgg forgets bucket names for StatsD counter metrics. As a result, StatsAgg won’t be capable of outputting ‘0’s for a bucket until it receives at least one new datapoint for that bucket (assuming that StatsAgg is configured to output ‘0’ for a bucket that hasn’t recently had a new value).
Format: bucket:metricValue|c|@samplePercent
Example:
<EMAIL_ADDRESS><EMAIL_ADDRESS>… outputs (assuming a ‘flush interval’ of 10 seconds):
myBucket.sum = 161
myBucket.rate_ps = 16.1
Sampling based counters work the same way as regular counters, but the metric value is divided by the sampling percentage. In the example, the sampling percentage for the value of ‘15.5’ is 10%. The sampling percentage for the value of ‘3’ is 50%. The math works as follows: (15.5 / 0.1) + (3 / 0.5) = 161.
Sum & rate reset to 0 after each aggregation.
Note:
· Note: regular counter & sampling-based counters can safely be used together on the same bucket.
· StatsAgg does support the StatsD ‘legacy namespacing’ convention, but it is not enabled by default. Please consult the application configuration file for more details.
· If no new metrics are received by StatsAgg for a bucket during a flush interval, StatsAgg can be configured to output either ‘0’ or nothing.
o The default behavior of StatsAgg is to send ‘0’.
o After a restart, StatsAgg forgets bucket names for StatsD counter metrics. As a result, StatsAgg won’t be capable of outputting ‘0’s for a bucket until it receives at least one new datapoint for that bucket (assuming that StatsAgg is configured to output ‘0’ for a bucket that hasn’t recently had a new value).
Format: bucket:metricValue|g
Example:
myBucket:+4|g
myBucket:15.5|g
myBucket:+0.5|g
myBucket:-3|g
… outputs “myBucket = 13”. This is because …
0 (starting value) – 4 = -4
Set to 15.5 (‘set’ values always overwrite the previous value)
15.5 + 0.5 = 16
16 – 3 = 13
Gauges are arbitrary values. They can be static values or additive values. Additive values are prefixed by +/- signs. ‘Set’ values do not have a prefix. Anytime a ‘set’ value is encountered, it immediately sets the gauge to equal the value that it is being set to.
Unlike all other StatsD metric types, the value of a gauge metric persists after every aggregation. That means that you can add or subtract values to a gauge across an infinite period of time & never have the metric reset to 0 (unless you purposefully set it to 0).
Notes:
· Unlike the official StatsD application, you can set a gauge to a negative number without first setting it to zero.
· If no new metrics are received by StatsAgg for a bucket during a flush interval, StatsAgg can be configured to output either “the gauge’s value from the previous flush-interval’s aggregation” or nothing.
o If StatsAgg is configured to ‘send previous values when no new value is received’, then gauges values are persisted in StatsAgg’s database. That means that when StatsAgg is restarted, the last known gauge value is automatically loaded into StatsAgg. Also, additive values are executed against whatever the previous value of the gauge was. In this configuration, gauge values will never reset to 0.
o If StatsAgg configured to ‘send nothing when no new value is received’, then gauge values are reset to 0 after each StatsD aggregation.
o The default behavior of StatsAgg is to send the gauge’s previous value.
Format: bucket:metricValue|ms
Example:
myBucket: 4|ms
myBucket:12|ms
myBucket:2|ms
… outputs (assuming a ‘flush interval’ of 10 seconds. For percentile values, assume the 70th percentile):
myBucket.count = 3
myBucket.count_70 = 2
myBucket.count_ps = 0.3
myBucket.lower = 2
myBucket.mean = 6
myBucket.mean_70 = 3
myBucket.median = 6
myBucket.std = 4.32
myBucket.sum = 18
myBucket.sum_70 = 6
myBucket.sum_squares = 164
myBucket.sum_squares_70 = 20
myBucket.upper = 12
myBucket.upper_70 = 4
Timer metrics are the most complex/heavy metric type that StatsD
supports. The input is expected to be in the form of a time (in milliseconds),
but it can be any arbitrary list of values. On an aggregation, all the metric
values associated with a bucket are aggregated together & the following
metrics are outputted:
· count (the count of ‘how many metrics were in a bucket’s list of metrics’)
· count_ps (the per-second rate of ‘how many metrics were in a bucket’s list of metrics’)
o count_ps = count/flush_interval
· count_nth (the count of ‘how many metrics were in a within the bucket’s nth percentile of metrics’)
o The ‘nth’ part of ‘count_nth’ is the numeric value of the nth percentile that is calculated. Example: ‘count_40’
o In the series “120, 334, 450, 496, 553, 675, 844, 994”, the 40th percentile count is ‘3’.
· count_topnth (the count of ‘how many metrics were in a within the bucket’s top nth percentile of metrics’)
o The ‘nth’ part of ‘top_topnth’ is the numeric value of the top nth percentile that is calculated. Example: ‘count_top40’
o When a negative nth percentile is specified by the user, this makes StatsD present the ‘top’ nth percentile.
o In the series “120, 334, 450, 496, 553, 675, 844, 994”, the top 35th percentile (user specified ‘-35’) count is ‘3’.
· lower (the smallest value of a bucket’s list of metrics)
· lower_topnth (the smallest value, within the top nth percentile, of a bucket’s metrics)
o The ‘nth’ part of ‘lower_nth’ is the numeric value of the nth percentile that is calculated. Example: ‘lower_top40’
o When a negative nth percentile is specified by the user, this makes StatsD present the ‘top’ nth percentile.
o In the series “120, 334, 450, 496, 553, 675, 844, 994”, the top 35th percentile (user specified ‘-35’) lower is ‘675’.
· mean (the average of a bucket’s list of metrics)
· mean_nth (the average of a bucket’s nth percentile of metrics)
o The ‘nth’ part of ‘mean_nth’ is the numeric value of the nth percentile that is calculated. Example: ‘mean_40’
o In the series “120, 334, 450, 496, 553, 675, 844, 994”, the 40th percentile mean is ‘301.3333333’.
· mean_topnth (the average of a bucket’s top nth percentile of metrics)
o The ‘nth’ part of ‘mean_topnth’ is the numeric value of the top nth percentile that is calculated. Example: ‘mean_top40’
o When a negative nth percentile is specified by the user, this makes StatsD present the ‘top’ nth percentile.
o In the series “120, 334, 450, 496, 553, 675, 844, 994”, the top 35th percentile (user specified ‘-35’) mean is ‘837.6666667’.
· median (the median value of a bucket’s list of metrics)
· std (the ‘population standard deviation’ of a bucket’s list of metrics).
· sum (the sum of a bucket’s list of metrics)
· sum_nth (the sum of a bucket’s nth percentile of metrics)
o The ‘nth’ part of ‘sum_nth’ is the numeric value of the nth percentile that is calculated. Example: ‘sum_40’
o In the series “120, 334, 450, 496, 553, 675, 844, 994”, the 40th percentile sum is ‘904’.
· sum_topnth (the sum of a bucket’s top nth percentile of metrics)
o The ‘nth’ part of ‘sum_topnth’ is the numeric value of the top nth percentile that is calculated. Example: ‘sum_top40’
o When a negative percentile is specified by the user, this makes StatsD present the ‘top’ nth percentile.
o In the series “120, 334, 450, 496, 553, 675, 844, 994”, the top 35th percentile (user specified ‘-35’) sum is ‘2513’.
· sum_squares (the ‘sum of squares’ of a bucket’s list of metrics)
o sum_squares = metricValueA^2 + metricValueB^2 + …
· sum_squares_nth (the ‘sum of squares’ of a bucket’s nth percentile of metrics)
o The ‘nth’ part of ‘sum_squares_nth’ is the numeric value of the nth percentile that is calculated. Example: ‘sum_squares_40’
o In the series “120, 334, 450, 496, 553, 675, 844, 994”, the 40th percentile ‘sum of squares’ is ‘328456’.
· sum_squares_topnth (the ‘sum of squares’ of a bucket’s top nth percentile of metrics)
o The ‘nth’ part of ‘sum_squares_topnth’ is the numeric value of the top nth percentile that is calculated. Example: ‘sum_squares_top40’
o When a negative percentile is specified by the user, this makes StatsD present the ‘top’ nth percentile.
o In the series “120, 334, 450, 496, 553, 675, 844, 994”, the top 35th percentile (user specified ‘-35’) ‘sum of squares’ is ‘2155997’.
· upper (the largest value of a bucket’s list of metrics)
· upper_nth (the large value, within the nth percentile, of a bucket’s metrics)
o The ‘nth’ part of ‘upper_nth’ is the numeric value of the nth percentile that is calculated. Example: ‘upper_40’
o In the series “120, 334, 450, 496, 553, 675, 844, 994”, the 40th percentile upper is ‘450’.
The values reset to 0 after each aggregation.
Notes:
· At this time, StatsAgg does not support timer histograms. This will likely be added in a future release of StatsAgg.
· The outputted metric-keys for the nth percentile keys will substitute the ‘_’ for the ‘.’ character in the event that the user specifies an nth percentile that has a fractional component.
o Example: if the user-specified nth percentile is “70.5”, then an example output metric-key would be “myBucket.upper_70_5”
· When a negative percentile is specified by the user, this makes StatsD present the ‘top’ nth percentile. This also has the side effect of changing the users negative percentage into a positive percentage, and will result in metric-keys between outputted with ‘top’ in place of ‘-‘.
o Example: if the user-specified top nth percentile is ‘-45, then an example output metric-key would be “myBucket.sum_top45”.
· StatsAgg allows for multiple ‘nth percentile’ values to be computed & outputted. This can be configured in the StatsAgg application configuration file.
· There is undocumented support for sampling based timers in the official StatsD application. At this time, StatsAgg does not support this functionality, and will not support this functionality until StatsD’s documentation formally declares support for this syntax.
· If no new metrics are received by StatsAgg for a bucket during a flush interval, StatsAgg can be configured to output either ‘0’ or nothing.
o The default behavior of StatsAgg is to send ‘0’.
o After a restart, StatsAgg forgets bucket names for StatsD timer metrics. As a result, StatsAgg won’t be capable of outputting ‘0’s for a bucket until it receives at least one new datapoint for that bucket (assuming that StatsAgg is configured to output ‘0’ for a bucket that hasn’t recently had a new value).
Format: bucket:metricValue|s
Example:
myBucket:15.5|s
myBucket:15.5|s
myBucket:88|s
myBucket:2.6|s
myBucket:88|s
… outputs: “myBucket = 3”
Sets count the number of unique metric values that a bucket has associated with it at aggregation time. If the value of ‘1000’ is sent 30 times during the flush interval, it will only count as +1 for the set.
The value resets to 0 after each aggregation.
Note:
· If no new metrics are received by StatsAgg for a bucket during a flush interval, StatsAgg can be configured to output either ‘0’ or nothing.
o The default behavior of StatsAgg is to send ‘0’.
o After a restart, StatsAgg forgets bucket names for StatsD set metrics. As a result, StatsAgg won’t be capable of outputting ‘0’s for a bucket until it receives at least one new datapoint for that bucket (assuming that StatsAgg is configured to output ‘0’ for a bucket that hasn’t recently had a new value).
StatsD has a metric output naming convention called ‘legacy namespacing’. ‘Legacy namespacing’ has the following effects:
· All metric output Sprefixes that are specific to StatsD counters are ignored
· Counter ‘rates’ are put in stats.{myBucket}
· Counter ‘counts’ are put in stats_counts.{myBucket}
· All other metric types are put in stats.{metricTypePrefix}.{myBucket}
StatsAgg supports this convention, but it is disabled by default.
While the various metric formats have some differences compared to the official StatsD application (see the individual metric format sections for more details), there are a few other differences in StatsAgg (compared to StatsD) that are worth mentioning. They are:
· StatsAgg is not capable of being a StatsD ‘repeater’
o If you wish to accomplish this functionality, you could use the actual StatsD application as a repeater & have the final destination be StatsAgg. This will only work for StatsD UDP metrics.
· StatsAgg cannot disable counters (“flush_counts”) like StatsD can.
· StatsAgg does not have a mechanism for user-provided ‘backends’.
o Because StatsAgg is a compiled/bundled program, ‘backend’ functionality must be built into the core program.
o Graphite is the only currently supported backend (for now). However, since several services support the receiving Graphite metrics (ex- influxdb), you are not necessarily limited to only Graphite.
StatsD
{
server: "./servers/tcp",
graphitePort: 2003,
graphiteHost: "graphite.some-domain.com",
port: 8125,
graphite: {globalPrefix : "stats", legacyNamespace : false}
}
StatsAgg
graphite_output_module_1 = true,graphite.some-domain.com,2003,3
statsd_tcp_listener_enabled = true
statsd_tcp_listener_port = 8125
statsd_metric_name_prefix_enabled = true
statsd_metric_name_prefix_value = stats
statsd_counter_metric_name_prefix_enabled = true
statsd_counter_metric_name_prefix_value = counters
statsd_timer_metric_name_prefix_enabled = true
statsd_timer_metric_name_prefix_value = timers
statsd_gauge_metric_name_prefix_enabled = true
statsd_gauge_metric_name_prefix_value = gauges
statsd_set_metric_name_prefix_enabled = true
statsd_set_metric_name_prefix_value = sets
statsd_use_legacy_name_spacing = false
StatsAgg can optionally output metrics to Graphite. Graphite, and Graphite compatible programs (ex- InfluxDB), are the only programs that StatsAgg currently supports outputting to.
Notes:
· The only metrics that are outputted are ‘aggregated’ metrics.
o In the case of Graphite-Passthrough metrics, this simply means that the metrics are sent to Graphite (though their metric-key may have been slightly altered by StatsAgg).
· After X retries (by default, X=3), StatsAgg will give up on trying to send a ‘batch’ metrics to Graphite. Any metrics that haven’t been outputted after the configured number of retries are discarded permanently.
o A ‘batch’ means the aggregation window’s worth of StatsD metrics, Graphite-Passthrough metrics, or Graphite-Aggregation metrics. StatsAgg handles these different metric types on separate threads, so for a given aggregation window, the ‘batchs’ for StatsD metrics, Graphite-Passthrough metrics, and Graphite-Aggregation metrics are independent of each other.
· If the connection between StatsAgg & Graphite is slow, this could cause stability problems in StatsAgg. The potential risk occurs when StatsAgg is receiving/processing/outputting metrics at a faster rate than they can be sent to Graphite. Slowness on metric transmission to Graphite could result in metrics building up in StatsAgg’s memory, which could ultimately cause StatsAgg to experience degraded performance (and possibly crash).
o Oftentimes the network isn’t the problem, and Graphite’s ability to receive/consume/process metrics is. If you have ample bandwidth between StatsAgg & Graphite, but Graphite doesn’t seem to be keeping up, look into optimizing your Graphite deployment.
One of the key reasons StatsAgg was created was to provide a robust alerting mechanism for various metric formats.
The alert routine is independent from all of the metric aggregation routines (StatsD, Graphite-Passthrough, Graphite-Aggregated). It runs periodically (the period can be changed in the application configuration file). When it executes, it will typically go through the following steps:
1. Metric association. This is the process of mapping metric-keys (Graphite metric-paths, StatsD buckets, etc) to StatsAgg ‘Metric Groups’.
a. This process can be CPU intensive if StatsAgg receives a large burst of metric-keys that it hasn’t seen before.
2. Determine what alerts are “suspended”. This maps StatsAgg ‘Alert Suspensions’ to StatsAgg ‘Alerts’. If an alert is suspended, StatsAgg will not perform some (or all) of the remaining routines on a particular alert.
3. Determine which StatsAgg ‘alerts’ are in a triggered state. Caution & Danger alerts are both evaluated at this stage. Alerts that are in a triggered state are marked as such.
4. Send notifications via email. When appropriate (based on an alert’s configuration), emails are sent out to the recipients that are specified in the alert’s ‘Notification Group’.
a. Caution & Danger notifications are treated as being completely independent from each other.
b. If an alert is ‘acknowledged’, it will not be ‘resent’ (even if ‘resend after…’ is set).
5. Output
alert status metrics. StatsAgg can be configured to
output the status of all alerts to Graphite.
Note: when an alert’s name has characters that don’t translate well into
Graphite’s naming convention, StatsAgg may try to
reformat the alert-name. Also, the ‘metric-path’ in Graphite will have a unique
identifier at the end of it. This was done to guarantee that no two alerts are
capable of outputting to the same metric-path in Graphite (which could
otherwise happen as a result of StatsAgg’s
metric-path renaming).
Here is the alert-status key:
a. 0 = alert not triggered
b. 1 = caution triggered
c. 2 = danger triggered
d. 3 = caution & danger triggered
6. Metric cleanup. StatsAgg evicts metric key/value pairs out of memory after they are no longer of any use.
a. Warning – if you set the StatsAgg ‘alert routine execution frequency’ to a higher-than-recommended value, then the metric cleanup routine will execute less frequently. This means that you increase the potential of filling up StatsAgg’s heap memory space.
7. Update global variables. After everything else has finished, the results of the alert routine are made available to the rest of the application. ‘Metric group associations’, ‘alert triggered statuses’, etc are now capable of being viewed in the WebUI.
a. A larger ‘alert routine execution frequency’ will result in increased WebUI lag for ‘Alerts’, ‘Metric Groups’, etc.
When StatsAgg is restarted, StatsAgg will handle alerts as follows:
· Threshold alerts that were in a triggered state before the restart will go into a status of ‘pending’. This gives them time to evaluate their status before erroneously firing ‘positive alert’ emails. A threshold alert will get out of ‘pending’ status when one of the following criteria is met:
o StatsAgg has received enough new datapoints to determine that the alert is still in a ‘triggered’ status. No email notifications will be sent in this case.
§ This is only evaluated for ‘Threshold’ alerts.
o StatsAgg has been running for at least ‘window duration’ seconds, and the alert has not re-entered a ‘triggered’ status. The alert will no longer be triggered & a ‘positive alert’ notification will be send (if configured).
o StatsAgg’s configuration file has a ‘max pending’ time. If this timeout is reached & the alert is not in a ‘triggered’ condition, then the alert will no longer be triggered & a ‘positive alert’ notification will be send (if configured).
· Availability alerts pick-up exactly where they left off. There is no ‘pending’ status for an availability alert.
·
All metric-keys must be re-associated with all
‘metric groups’. This is done as new metric-keys arrive into StatsAgg. This can be a slow/heavy process if a lot of
unique metrics-keys are sent to StatsAgg, so it may
take a little while (seconds to minutes) for all metric-keys to become
associated with all ‘metric groups’.
· The alert routine runs in a separate thread from the other ‘aggregation’ routine threads. In other words, alert evaluation is NOT performed as part of the StatsD & Graphite metric aggregation routines.
·
For StatsD & Graphite-Aggregated metrics, alert evaluation
is only performed on aggregated
metrics.
o
Example: If you send 4000 StatsD
gauge metrics during a single ‘flush interval’ (which is 10secs, by default),
then StatsAgg will only use the final aggregated
value of those 4000 gauges for alert evaluation.
o
Since Graphite-Passthrough
metrics are not aggregated, every single individual datapoint
is used for alert evaluation.
Once data is being
fed into StatsAgg, you may want to alert on it. This
section describes the general path through the website to create an alert.
1. Create a ‘notification group’. ‘Notification groups’ are simply a list of email addresses that alerts get sent to. If you do not want/need StatsAgg to send email alerts, then you can ignore ‘notification groups’ entirely.
2. Create a ‘metric group’. This ‘metric group’ should have regular expressions that match against whatever metric-keys you’re interested in alerting on. If you which, you can also ‘tag’ your ‘metric group’.
3. Create an alert. When you create an alert, it ties into a single ‘metric group’. This allows you to alert on whatever values are associated with the ‘metric group’.
4. Create an ‘alert suspension’. ‘Alert suspensions’ allow you to temporarily disable alerting. If you don’t need to suspend alerts, then you can ignore ‘alert suspensions’ entirely.
The remains subsections go into greater detail about how to create &
configure notification groups, metric groups, alerts, and alert suspensions.
Tables
·
All tables in the StatsAgg
UI are generated server-side. On each page load, entire table is sent to the
client, and Javascript is used for pagination,
filtering, etc.
·
If a table appears to be mangled, try clearing
all StatsAgg related cookies in your browser.
Sorting tables by column
On a table, any column, or set of columns, can be used for sorting.  Using
regular expressions, you can group together any set of metrics. You can then
create an alert that alerts off of the metrics that are associated with the
metric group.
This table lists all current created notification groups. Much of the functionality on this page speaks for itself, but some functionality requires some explanation.
# Metric Associations
· Altering a ‘metric group’ will erase all ‘metric group associations’ with the ‘metric group’. As StatsAgg receives new metrics, the ‘metric group associations’ for a ‘metric group’ will repopulate.
· Individual metrics are associated with ‘metric groups’ when the ‘alert routine’ is executed. That means that one should expect a delay between when the metric is initially received by StatsAgg & when it shows up as being associated with a metric group.
· Any metric that has had a new datapoint within the last 24hrs will show up as being ‘associated’ with a ‘metric group’. Metrics that haven’t received any new datapoints for 24hrs will be ‘forgotten’ by StatsAgg.
Name (required): A unique name for this metric group. Other than uniqueness, the only limitation is that it must be under 500 characters long.
Description (optional): A
description of the metric group. Avoid descriptions longer than 1000000
characters.
Regular Expressions (optional):
These are regular expressions that are used to tie individual metrics to the metric
group. If a metric matches any one of the provided regular expressions, then it
is considered to be ‘associated’ with the metric group. If using multiple
regular expressions, put each regular expression on a separate line in the
textbox. The regular expression format is Java Regex. For more information,
visit: https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html
Tags (optional): Metric groups can be ‘tagged’. Tags allows for convenient filtering in the various tables on the StatsAgg web user-interface. Alerts can also be suspended by metric group tags.
This table lists all current created notification groups. Much of the functionality on this page speaks for itself, but some functionality requires some explanation.
Test
Clicking on ‘test’ will send an email to all the members of the notification group. This functionality works regardless of whether the alert-routine is configured to send emails. The only requirement for this functionality to work is that the SMTP settings in the StatsAgg configuration file need to be valid. Emails will only be sent to email addresses that are properly formatted.
Name (required): A unique name for this notification group. Other than uniqueness, the only limitation is that it must be under 500 characters long.
Email addresses (optional): A common
separated list of email addresses.
This table lists all current created alerts. Much of the functionality on this page speaks for itself, but some functionality requires some explanation.
Colored rows
Red: Danger alert is currently active & not suspended
Caution: Caution alert is currently active & not suspended
Blue: Either a caution or danger alert is currently suspended. Also, blue can indicate that StatsAgg was recently restarted & the alerting mechanism hasn’t determined whether a caution or danger alert that was active prior to the restart is still triggered after the restart.
Clone
Clicking this button clones the alert. A new alert will be spawned with the same configuration. The only differences are:
· The new alert will be named “previousAlertName_#” (where # is some number that doesn’t collide with another alert name).
· The state of the alert will not be cloned. That means that triggered/not-triggered, acknowledged/unacknowledged, last alert send time, etc are not cloned.
Acknowledge/Ucknowledge
Alerts can be ‘acknowledged’ when they are in a ‘triggered’ state. This is done so that StatsAgg users can know that someone (hopefully the receipient of the notification group’s email notification) has seen that the alert is in a triggered state & is looking into the cause. When alerts are ‘acknowledged’, this disables resending of email notifications (even if resending is enabled).
Core Alert
Criteria
Name (required): A unique name for this alert. Other than uniqueness, the only limitation is that it must be under 500 characters long.
Description (optional): A description of the alert. Avoid descriptions longer than 1000000 characters.
Metric group name (required): The ‘metric group’ that this alert is associated with. Alerts *must* be paired with ‘metric groups’, but multiple alerts can be associated with the same ‘metric group’. Alerts are only capable of alerting on metric-keys that are associated with the alert’s ‘metric group’.
Alert type (required): Availability, Threshold
Availability: ‘Availability alerts’ are triggered when a ‘metric-key’ (a unique metric that is associated with the metric group) stops reporting datapoints. Put another way, if StatsAgg stops receiving metric values for a particular metric-key (no new data points for the last ‘window duration’ seconds), then an alert will be triggered. Receipt of a new datapoint for a ‘metric-key’ is the ‘positive-alert’ criteria for the availability alerts.
Threshold: Threshold-alerts are triggered when the value of an individual metric-key is above/below/equal-to a specified ‘threshold’ value. The values of a metric-key that fall within the ‘window duration’ are used for alerting consideration.
Is alert enabled? (required): If you want the alert to be enabled right away after creation or alteration, then check this checkbox. When an alert is disabled, caution & danger alerts will not fire (they are not even evaluated).
Is caution alerting enabled? (required): When caution alerting is disabled, caution alerts will not trigger (they are not even evaluated).
Is danger alerting enabled? (required): When danger alerting is disabled, danger alerts will not trigger (they are not even evaluated).
Alert on positive? (required): If this is checked, alerts that change states from ‘triggered’ to ‘not triggered’ will send an email notification to the notification group’s recipients.
Resend alert? (required): If this is checked, alerts that are in a ‘triggered’ state will send a new email notification to the notification group’s recipients after the specified amount of time. The new email notification will reflect whatever metrics are currently resulting in the alert to be triggered (not just whatever was in the original email notification). Note— if an alert has been acknowledged, then notifications will not be resent (regardless of whether ‘resend alert’ is enabled).
Resent alert every... (in seconds): If ‘resend alert?’ is enabled, then alerts in a ‘triggered’ state will send a new email notification to the notification group’s recipients after the amount of time specified by this option.
Caution/Danger
Criteria
Notification group name (optional): The notification group that is associated with the alert. Alerts that are triggered will be sent to the members of this notification group.
Window duration (availability and threshold alerts): The ‘window duration’ is an amount of time (between ‘now’ and ‘x seconds ago’) that metrics values are allowed to be considered for alerts. That is to say, if you specify that the ‘window duration’ is 300 seconds, then the rest of the alert criteria variables will only operate on metric values that have timestamps between ‘now and ‘300 seconds ago’.
This field must be set to a value greater than 0. To preserve StatsAgg resources, it is highly recommended that you try to keep the window duration as small as possible.
Stop tracking after (availability alerts only): For availability alerts, StatsAgg requires that you eventually ‘give up’ on tracking a metric that hasn’t had any new datapoints. This prevents StatsAgg from having to track metrics forever, and prevents availability alerts from potentially being in a ‘triggered’ state forever. ‘Stop tracking after’ is the amount of time (in seconds) that you will wait for a new datapoint to show up for a metric before the availability alert ‘gives up’ on looking for a new datapoint to show up. When an availability alert ‘gives up’, it is treated as a positive alert, and the reason given will be that the alert reached its ‘stop tracking time limit’.
Minimum sample count (threshold alerts only): For a threshold-alert to be triggered, one of its associated metric-key must have a minimum number of recent data points within the ‘window duration’. For example, if the ‘window duration’ is 300 seconds, and the ‘minimum sample count’ is 2, then an alert can only be triggered if 2 or more metric values (of an individual metric-key) have been received by StatsAgg during the last 300 seconds.
This field must be set to a value greater than 0. The most commonly set value is 1.
Operator (threshold alerts only): The values of a metric-key are considered for threshold-based alerting when they are above/below/equal-to a certain threshold. This value controls the above/below/equal-to aspect of the alert.
Combination (threshold alerts only): For a threshold-alert, we must decide how the values will be used for alert consideration.
For example, say a metric-key has several values within the ‘window duration’, and those values are ‘1, 3, 5, 7’. Let’s also say that we have a threshold of ‘6’ & an operator of ‘less than’. What happens – did we trigger the alert? Do we consider the average of those numbers? Do all of them have to be less than 6?
The ‘combination’ field makes the user choose how the metric values are evaluated. One can choose from ‘any value’, ‘all values’, the ‘average of the values’, ‘at most X values’, or ‘at least X values’.
In the above example, if we choose ‘average’, then we’d be saying “The average value of the metric-key must be less than of 6”. Since the average value is 4, the alert would be triggered.
Combination count (threshold alerts only): When one chooses a ‘combination’ of ‘at most X values’ or ‘at least X values’, a count is required. The ‘combination count’ specifies how many values are required.
In the previous example of ‘1, 3, 5, 7’ < 6, if we have a ‘combination’ of “at least” and a ‘combination count’ of 2, then we’re effectively saying: “At least 2 metric-key values must be less than 6”. Since both 1, 3, and 5 are all less than 6, the alert would be triggered.
This value must be greater than or equal to 0.
Threshold (threshold alerts only): The ‘threshold’ is the value that is compared against when deciding whether an alert is active or not. For example, if an alert states that “At least 2 metric-key values must be less than the value of 6”, then the ‘threshold’ in that alert is 6.
Notes:
· For an alert, one can set the ‘caution criteria’, the ‘danger criteria’, both, or neither. StatsAgg does not require that the caution or danger criteria be valid for an alert to be created. However, if you want the alert to *do* something, then you should set some valid alert criteria. You ca
· A good way to test alert criteria for validity is to hit the ‘preview’ button. This will give you a preview of what the email alert would look like. The wording of the email alerts was very carefully chosen to give readers a ‘plain english’ version of the alert criteria.
· When an alert is triggered by one or more metrics, a single notification is sent to the receipts listed in the alert’s ‘notification group’. StatsAgg does not currently have the capability of alerting on each metric-key that tripped the alert separately.
· Each metric-key that is associated with the alert’s ‘metric-group’ is individually considered for triggering the alert. The datapoints from different metric-keys are never aggregated together. For example, say that we have ‘combination’ set to “average”:
MetricSeries1 values: 1 2 3 4 5 -- avg = 3
MetricSeries2 values: 1 1 1 1 1 -- avg = 1
If the alert’s criterion says to alert when the average value is greater than 2, then ‘MetricSeries1’ will trigger the alert, but ‘MetricSeries2’ won’t. In the notification email, only ‘MetricSeries1’ will be mentioned.
This table lists all current created ‘alert suspensions’. Much of the functionality on this page speaks for itself, but some functionality requires some explanation.
Colored rows
Blue: The ‘alert suspension’ is currently active.
Options
Name (required): A unique name for this ‘alert suspension’. Other than uniqueness, the only limitation is that it must be under 500 characters long.
Is enabled? (required): If you want the alert suspension to be enabled after creation or alteration, then check this checkbox. When an ‘alert suspension’ is disabled, the alert suspension won’t be able to suspend any caution or danger alerts.
Suspend notification only? (required): When checked, suspended alerts still evaluate their alert criteria (and display the caution/danger triggered status on the StatsAgg WebUI); they just don’t send out emails alerts. When unchecked, suspended alerts will not be evaluated at all. The most common approach is to have this field checked.
Suspend by…
Alert name (optional): If you want to suspend a single alert, then suspend it by alert name.
Tags (optional): If you want to suspend a group of alerts that have one or more tags in common, then suspend by tags. For an alert to be suspended, it must be tagged with every tag specified in the ‘alert suspension’. Only put one tag per line in the ‘tags’ textbox.
Example: if the ‘alert suspension’ suspends by tags, with tags: “tag1”, “tag2”, “tag3”…
· Alert_1 (tags: “tag1”, “tag2”, “tag3”) – meets suspension criteria
· Alert_2 (tags: “tag1”, “tag2”) – does not meet suspension criteria
· Alert_3 (tags: “tag1”, “tag2”, “tag3”, “tag4”) – meets suspension criteria
Everything (optional): If you want to suspend ALL alerts, except for a select few, then select this option. For an alert to be spared from suspension, it must be tagged with at least one of the tags specified in the ‘alert suspension’. Only put one tag per line in the ‘everything’ textbox.
Example: if the alert suspension suspends by everything, with tags: “tag1”, “tag2”, “tag3”…
· Alert_1 (tags: “tag1”, “tag2”, “tag3”) – meets suspension criteria
· Alert_2 (tags: “tag3”) – meets suspension criteria
· Alert_3 (tags: “tag4”) – does not meets suspension criteria
Suspension
Type…
Recurring (daily):
This suspension type means that the ‘alert suspension’ recurs every day.
One Time: This suspension type means that the ‘alert suspension’ is only ever used once. Once the ‘alert suspension’ is complete, it will self-delete.
Start Date: Recurring ‘alert suspensions’ are only active on or after
the date specified in this field.
The format is: MM/dd/yyyy
Example: 03/23/2014
Recurs on: ‘Alert suspensions’ are active only on the days specified by ‘recurs on’. This can be used for cases such as “we have maintenance only on Wednesday nights, so we only want to suspend alerts on Wednesday night”. In a case like this, you would put a check next to ‘W’ (and no other boxes).
Start Time: What time does the ‘alert suspension’ become active?
The format is: h:mm a.
Example: 9:23 AM
Duration (mins): Specifies the amount of time that the ‘alert suspension’ remains active (in minutes). This must be a positive value, and must be less than 1441.
Notes
· If an ‘alert suspension’ becomes active on a particular day, and the specified duration takes it into the next day, then the alert suspension will continue to remain active (until ‘duration’ minutes have passed). This happens regardless of whether the ‘recurs on’ field allows alert suspensions on the ‘next day’ or not.
· The ‘Preview Alert Associations’ button next to ‘Suspend By…’ can be used to determine if your ‘alert suspension’ rule is suspending the alert(s) that you want to suspend.
If you want to see what metrics StatsAgg is currently aware of (seen in the last 24 hours), then you can use free-form regular expressions to view the list of known metrics.
Note: regular expressions are matched against aggregated metrics. For example, if they are StatsD metrics, then you should expect to metrics with keys like originalMetricName.sum. Also, if metric prefixes and/or suffixes were used, then they will also be matched against.
The regular expression format is Java Regex. For more information, visit:
https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html
‘Forget Metric(s)’ is a way of telling StatsAgg to completely forgot about a metric-key & all
its values. Everything that there is to know about the metric key is wiped out.
Examples of when you would want to use this:
· You have StatsD metrics that are constantly sending their last seen value (like gauges), or are sending 0s (like timers, counters, etc). If you don’t care about these metric-keys anymore, then ‘forgetting’ them will have StatsAgg stop sending values for them.
· You have an alert that is triggered based off of criteria that you know is no longer true. A quick way of clearing the alert-status could be to ‘forget’ the metric-key(s) that are triggering the alert.
On the ‘forget metric(s)’ page, you can either forget a
single metric-key, or forget several (via a regex). If you enter values for
both fields, then both fields will be evaluated.
Note: regular expressions are matched against aggregated metrics. For example, If they are StatsD metrics, then
you should expect to metrics with keys like originalMetricName.sum. Also, if metric
prefixes and/or suffixes were used, then they will also be matched against.
The regular expression format is Java Regex. For more
information, visit:
https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html
StatsAgg exposes a health-check page @ /HealthCheck
Under normal circumstances, the health-check page should return some
information about StatsAgg, and a status code of HTTP
200.
StatsAgg will return HTTP 500 on the health-check page when one of the following criteria is met:
· Failure on connecting to the database.
· Application initialization has failed. Many conditions can lead to this (failure to initialize the logging library, failure to start the StatsD and/or Graphite server listeners, etc.
StatsAgg achieves high-performance by keeping nearly all of its data in memory. As a result, the more metrics that are sent to it, the more memory is needed. ‘Metric groups’ & alerts also contribute to memory usage. Here are some tips for making sure you don’t run out of memory:
· Try to keep the number of metrics that get associated with a metric group to a minimum. StatsAgg has to keep track of how each metric maps to each ‘metric group’.
· For alerts, keep ‘window duration’ to as narrow of a value as possible. StatsAgg only keeps the absolute minimum number of data points in memory that it needs to, but if the ‘window duration’ for an alert is set to a wide time-frame (ex—1 day), then StatsAgg will have to keep all of the data points associated with that alert’s metric group in memory for the entire ‘window duration’.
· A 4GB heap should be enough for most moderate-use use-cases. When consuming metrics at higher rates, an 8GB heap (or more) may be needed.
o Proper memory sizing will ultimately on your specific usage pattern, so it may take some trial/error.
· StatsD timer metrics use a lot of memory compared to other metric types. This is because StatsD timer metrics, upon aggregation, are ballooned up into many separate metrics.
o Incidentally, StatsD timer metrics also use the most CPU of any metric type.
· StatsAgg has absolutely no limitations on how many metrics it will try to consume/keep in memory/aggregate/output/etc. If you create a metric group that captures every single metric, and then create an alert (using that ‘metric group’) with a window-duration of 1000 years, StatsAgg will let you do this. Given enough datapoints, this will likely result in StatsAgg running out of memory & crashing.
· Because StatsAgg is written in Java & it keeps much of its data in-memory, heap dumps can be very useful in figuring out what specifically is causing all your memory usage. Is it an alert with an overly-wide ‘window duration’? Is it that an application is feeding metrics into StatsAgg at a rate of 1,000,000,000 metrics per second? Heap dumps are great at helping to figure this stuff out.
o Eclipse MAT is the first-choice for heap dump analysis.
o VisualVM is also good at analyzing heap dumps.
Under most circumstances, TCP is recommended over UDP for use with StatsAgg. UDP is an inherently packet-loss prone protocol, and under normal conditions, it is often possible to lose a non-trivial percentage of your packets. TCP has more guarantees on connectivity & delivery of data, so it is the preferred protocol of the author of StatsAgg. TCP & UDP metric consumption are very lightweight in StatsAgg, so there is no real performance consideration for using one or the other.
For TCP, the recommended approach is to open a connection, send the metrics, and close the connection. Leaving a connection open/hanging will tie up StatsAgg & operating system resources with no meaningful benefit.
· Java 1.7 is the minimum Java spec that StatsAgg supports.
· Oracle Java 1.8, x64, is the recommended runtime engine for StatsAgg.
· StatsAgg was primarily developed & tested against Oracle Java on Windows.
· OpenJDK should work with StatsAgg, but it is untested, so functionality & performance are unknown.
· Using the full JDK package is recommended, as the JDK package comes with diagnostic tools that can be used to troubleshoot StatsAgg.
o Ex – VisualVM
· Recommended garbage collector: Concurrent Mark & Sweep.
o -XX:+UseConcMarkSweepGC
· Heap size: this heavily depends on your usage of StatsAgg.
o A good starting place is a 2.5GB heap.
o StatsAgg has been tested with max heap sizes as low as 1GB & as high as 8GB.
1) Why bother completely re-implementing the StatsD protocol?
a. When StatsAgg was first started, StatsD did not support TCP-based metric transmission, so StatsAgg sought to correct that. Also, limitations like ‘forgets Gauge values after a restart’ were a deal-breaker for many use-cases, and that would be difficult to fix in the main StatsD application.
b. If you don’t want to use StatsAgg’s implementation of StatsD, then you can send metrics to an instance of the official StatsD application & then have StatsD output its metrics to StatsAgg (which would receive them as Graphite metrics).
2) “Do I need to output to Graphite/Influx/etc”?
a. No. StatsAgg doesn’t mandate that metrics be output anywhere.
3) “Can I run a multi-server deployment of StatsAgg?”
a. No. StatsAgg only works in a single-server deployment model. There are no plans to support a multi-server deployment model in the future.
b. Why not!?
i. StatsD Gauge metrics. Gauge metrics can be directly set, incremented, or decremented – and the order matters. If the order is mixed up, even slightly, then the value of the Gauge metric could end up being completely different/wrong. There is no efficient way to guarantee correct ordering with multiple servers for metrics types like this (at least not that this author could think of…).
ii. Alerting. StatsAgg’s alerting mechanism expects to be able to quickly assess what metrics are in an alerted state. If multiple servers had a partial dataset of the metric-values for a metric-key, then alert criteria couldn’t be properly evaluated.
iii. You can always just run multiple StatsAgg servers that are completely independent from each other.
4) “Performance?”
a. StatsAgg’s author has a day-job of ‘Performance Engineer’, so a lot of effort has gone into making StatsAgg perform well under heavy-load conditions. To give a general sense of overall performance, StatsAgg will use…
· around 40% CPU & ~1GB JVM Heap on a 4-core Intel i5 processor VM (running Windows 2008 R2)…
· … while consuming 100,000 Graphite-Aggregated metrics per second & a diverse set of 100,000 StatsD metrics per second. That’s 200,000 metrics per second total, and does not include any Graphite-Passthrough metrics…
· … with 50 metric groups & 50 alerts set (window durations under 5mins).
If one were to send most Graphite-Passthrough metrics, then StatsAgg could scale to millions of metrics per second on moderate hardware.
5) StatsAgg uses a lot of CPU after a restart! What gives?!
a. As a result, after a StatsAgg restart, it is common to have a period of high CPU utilization while metrics-keys are re-associated with ‘metric groups’. The time it takes will be proportional to (#unique-metric-keys * #metric-groups).
6) I altered a metric-group and/or alert and my alert status changed from triggered->not triggered. Why?
a. Altering a ‘metric group’ is treated as a complete refresh of everything unique about that ‘metric group’. That means that all ‘metric group’ associations are reset, and all datapoints that were being kept in memory because of their association with the ‘metric group’ were allowed to be cleaned up.
b. Altering an alert mandates a reset of the status of the alert. If the alert is still in a triggered state after the alteration, then it will re-alert (complete with a new notification).
7) Will StatsAgg ever output to service XYZ?
a. Support for writing to other backends will likely be built into StatsAgg over time. Possible candidates include: OpenTSDB, MySql, Mongo, PostgreSQL, Apache Derby. Backends for commercial services are unlikely to be built into StatsAgg. This is because the author of StatsAgg doesn’t feel that it is appropriate to selectively implement solutions for particular closed-sourced (for-profit) companies & technologies.
1. Many errors/issues with StatsAgg will be captured in StatsAgg log file(s). The main log file is located at (tomcat-location)/logs/statsagg.log, but you may also find it useful to inspect other log files (like catalina.out).
2. StatsAgg doesn’t seem to shutdown, or litters the logs when shutting down.
a. StatsAgg attempts to shutdown gracefully, which gives each of the threads several seconds (up to minutes) to shut down. If these threads don’t shut down after a certain period of time, they are forcibly terminated. This may result in errors in the log files, but the errors should be harmless overall. StatsAgg was built to be resilient to hard crashes.
3. The StatsAgg WebUI homepage says “StatsAgg is using its default application configuration settings. This occurs when there was an error loading the 'application.properties' configuration file.”
a. As the error suggests, this usually means that the application.properties file was not created. Section 2.5.1 discusses how create/configure the application.properties file. If there was some other type of error (invalid settings, etc), then odds are good that you will find a more detail description of the error in the StatsAgg log file.
4. The StatsAgg WebUI homepage says “StatsAgg is using an ephemeral (in-memory) database. This occurs when there was an error loading the 'database.properties' configuration file.”
a. As the error suggests, this usually means that the database.properties file was not created and/or improperly configured. Section 2.5.2.x discusses how create/configure the database.properties file. If there was some other type of error (invalid settings, etc), then odds are good that you will find a more detail description of the error in the StatsAgg log file.
5. The StatsAgg WebUI homepage says “StatsAgg did not successfully connect to the database. Please view the StatsAgg log files for more details.”
a. This most frequently occurs StatsAgg can’t connect to a MySQL database. This could be because MySQL has crashed, connectivity was lost (network is down, firewall, etc), MySQL is overload/unresponsive, etc. You will have to investigate what is wrong with the StatsAgg ßà database interaction to solve this problem. Oftentimes the StatsAgg log file is helpful in these circumstances. The StatsAgg log file is located at: (tomcat-location)/logs/statsagg.log.
b. Once the issue is discovered/fixed, it is recommended that StatsAgg be restarted.
6. Alerts/emails/WebUI are out of sync with each other. What gives?
a. While this behavior is rare, such behavior can happen when the StatsAgg database becomes unavailable for a period of time. This is almost exclusively a MySQL concern. The underlying reason for this behavior is that the StatsAgg application caches a lot of data to keep performance high. Cases like this are being identified/fixed, but there may be edge-cases that could lead to synchronization issues.
b. When such behavior is noticed, restart StatsAgg.
· A few StatsD features that are in the main StatsD program are missing in StatsAgg. Missing features include…
o histograms on timers (will be included in a future build)
o configuring StatsAgg to be a ‘repeater’ (you can use the official StatsD program to do this, and forward to StatsAgg)
o configuring StatsAgg to be in a ‘proxy cluster’ configuration (you can use the official StatsD program to do this, and forward to StatsAgg)
o outputting frequently sent metric-keys to log files
· StatsAgg is not (currently) meant for use as an incident management tool. It doesn't support alert history, alert acknowledgement, etc.
· StatsAgg only supports running in a single-server configuration. See the FAQs section for more information.
· StatsAgg does not currently support the Graphite 'Pickle' format (will be included in a future build).
· Do not run a build of StatsAgg that was compiled with JDK8 on a JRE7 runtime. For whatever reason (likely a compiler bug), this will result in broken functionality in StatsAgg.
StatsAgg is developed in Java. Java was chosen because it is cross-platform, well supported, mature, and achieves relatively high performance.
StatsAgg’s design was based on the following principles:
· Keep framework usage to an absolute minimum. Only modular libraries were used (libraries that can easily be swapped out).
o No hibernate, struts, spring, play!, etc
· CPU & Memory usage must linearly scale by input metric count, metric group count, alerts count, etc. Exponential usage of CPU or Memory as a factor of any StatsAgg input was specifically avoided.
· Assume people want LOTS of unique metrics & want frequent datapoints for each metric.
· An admin-interface that is intuitive, but powerful.
StatsAgg
is released under that Apache 2.0 license.
http://www.apache.org/licenses/LICENSE-2.0
Lead developer/designer/architect/documenter: Jeffrey Schmidt
WebUI consultants: Pasi Jouhikainen, Adam Parsons, Deepak Patnam
Architecture consultations: Pasi Jouhikainen, Adam Parsons, Michael Anderson
StatsD: etsy @ https://github.com/etsy/statsd/
Graphite: Orbitz @ http://graphite.wikidot.com/
Pearson Assessments, a division of Pearson Education: http://www.pearsonassessments.com/


created by Cody Beckner
"...a software development process that relies on the repetition of a very short development cycle..."
"Unit testing is the process of validating that pieces of your code operate as you expect"
"Unit testing is a design strategy"ME
Fail First. Fail Early. Fail Often.
(for the most part)
Don't test layout code
XCTAssert(vc.numberLabel.width == 80.0)
DO test layout values
XCTAssert(vc.numberLabel.text == "80.0")
(especially after calling a function that changed them)
Basically, if your code alters a variable or data that will be shown to the user, test it.
func testGetRelativeTimeSinceTwoDaysAgo() {
// Arrange
var testDate = Date()
let calendar = NSCalendar.current
testDate = calendar.date(byAdding: .day, value: -2, to: testDate)!
let format = "EEEE hh:mm a"
let expectedResult = Date.format(date: testDate, format: format)
// Act
let answer = testDate.relativeFormatToNow()
// Assert
XCTAssertEqual(answer, expectedResult)
}
override func spec() {
describe("Date helper") {
context("time since") {
it ("two day ago should return two days") {
// Given
var testDate = Date()
let calendar = NSCalendar.current
testDate = calendar.date(byAdding: .day, value: -2, to: testDate)!
let format = "EEEE hh:mm a"
let expectedResult = Date.format(date: testDate, format: format)
// When
let answer = testDate.relativeFormatToNow()
// Then
expect {answer}.to(equal(expectedResult))
}
}
}
}
}
fastlane ios test
fastlane scan
@Test
public void testGetRelativeTimeSinceTwoDaysAgo() {
// Arrange
Calendar calendar = Calendar.getInstance();
calendar.add(Calendar.DAY_OF_YEAR, -2);
Date testDate = calendar.getTime();
DateFormat sdf = new SimpleDateFormat("EEEE hh:mm a");
String expectedResult = sdf.format(testDate);
// Act
String result = Util.getRelativeTimeSince(testDate);
// Assert
assertTrue(result.equalsIgnoreCase(expectedResult));
}
./gradlew test