Built-in analyzer reference
OpenSearch ships with a wide range of built-in analyzers, which can be used in any index without further configuration:
- Standard Analyzer
-
The
standard
analyzer divides text into terms on word boundaries, as defined by the Unicode Text Segmentation algorithm. It removes most punctuation, lowercases terms, and supports removing stop words. - Simple Analyzer
-
The
simple
analyzer divides text into terms whenever it encounters a character which is not a letter. It lowercases all terms. - Whitespace Analyzer
-
The
whitespace
analyzer divides text into terms whenever it encounters any whitespace character. It does not lowercase terms. - Stop Analyzer
-
The
stop
analyzer is like thesimple
analyzer, but also supports removal of stop words. - Keyword Analyzer
-
The
keyword
analyzer is a ``noop'' analyzer that accepts whatever text it is given and outputs the exact same text as a single term. - Pattern Analyzer
-
The
pattern
analyzer uses a regular expression to split the text into terms. It supports lower-casing and stop words. - Language Analyzers
-
OpenSearch provides many language-specific analyzers like
english
orfrench
. - Fingerprint Analyzer
-
The
fingerprint
analyzer is a specialist analyzer which creates a fingerprint which can be used for duplicate detection.
Custom analyzers
If you do not find an analyzer suitable for your needs, you can create a
custom
analyzer which combines the appropriate
character filters,
tokenizer, and token filters.
Fingerprint analyzer
The fingerprint
analyzer implements a
fingerprinting algorithm
which is used by the OpenRefine project to assist in clustering.
Input text is lowercased, normalized to remove extended characters, sorted, deduplicated and concatenated into a single token. If a stopword list is configured, stop words will also be removed.
Example output
POST _analyze
{
"analyzer": "fingerprint",
"text": "Yes yes, Gödel said this sentence is consistent and."
}
The above sentence would produce the following single term:
[ and consistent godel is said sentence this yes ]
Configuration
The fingerprint
analyzer accepts the following parameters:
separator
|
The character to use to concatenate the terms. Defaults to a space. |
max_output_size
|
The maximum token size to emit. Defaults to |
stopwords
|
A pre-defined stop words list like |
stopwords_path
|
The path to a file containing stop words. |
See the Stop Token Filter for more information about stop word configuration.
Example configuration
In this example, we configure the fingerprint
analyzer to use the
pre-defined list of English stop words:
PUT my-index-000001
{
"settings": {
"analysis": {
"analyzer": {
"my_fingerprint_analyzer": {
"type": "fingerprint",
"stopwords": "_english_"
}
}
}
}
}
POST my-index-000001/_analyze
{
"analyzer": "my_fingerprint_analyzer",
"text": "Yes yes, Gödel said this sentence is consistent and."
}
The above example produces the following term:
[ consistent godel said sentence yes ]
Definition
The fingerprint
tokenizer consists of:
- Tokenizer
- Token Filters (in order)
-
-
Stop Token Filter (disabled by default)
If you need to customize the fingerprint
analyzer beyond the configuration
parameters then you need to recreate it as a custom
analyzer and modify
it, usually by adding token filters. This would recreate the built-in
fingerprint
analyzer and you can use it as a starting point for further
customization:
PUT /fingerprint_example
{
"settings": {
"analysis": {
"analyzer": {
"rebuilt_fingerprint": {
"tokenizer": "standard",
"filter": [
"lowercase",
"asciifolding",
"fingerprint"
]
}
}
}
}
}
Keyword analyzer
The keyword
analyzer is a ``noop'' analyzer which returns the entire input
string as a single token.
Example output
POST _analyze
{
"analyzer": "keyword",
"text": "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone."
}
The above sentence would produce the following single term:
[ The 2 QUICK Brown-Foxes jumped over the lazy dog's bone. ]
Configuration
The keyword
analyzer is not configurable.
Definition
The keyword
analyzer consists of:
- Tokenizer
If you need to customize the keyword
analyzer then you need to
recreate it as a custom
analyzer and modify it, usually by adding
token filters. Usually, you should prefer the
Keyword type when you want strings that are not split
into tokens, but just in case you need it, this would recreate the
built-in keyword
analyzer and you can use it as a starting point
for further customization:
PUT /keyword_example
{
"settings": {
"analysis": {
"analyzer": {
"rebuilt_keyword": {
"tokenizer": "keyword",
"filter": [ (1)
]
}
}
}
}
}
-
You’d add any token filters here.
Language analyzers
A set of analyzers aimed at analyzing specific language text. The
following types are supported:
arabic
,
armenian
,
basque
,
bengali
,
brazilian
,
bulgarian
,
catalan
,
cjk
,
czech
,
danish
,
dutch
,
english
,
estonian
,
finnish
,
french
,
galician
,
german
,
greek
,
hindi
,
hungarian
,
indonesian
,
irish
,
italian
,
latvian
,
lithuanian
,
norwegian
,
persian
,
portuguese
,
romanian
,
russian
,
sorani
,
spanish
,
swedish
,
turkish
,
thai
.
Configuring language analyzers
Stopwords
All analyzers support setting custom stopwords
either internally in
the config, or by using an external stopwords file by setting
stopwords_path
. Check Stop Analyzer for
more details.
Excluding words from stemming
The stem_exclusion
parameter allows you to specify an array
of lowercase words that should not be stemmed. Internally, this
functionality is implemented by adding the
keyword_marker
token filter
with the keywords
set to the value of the stem_exclusion
parameter.
The following analyzers support setting custom stem_exclusion
list:
arabic
, armenian
, basque
, bengali
, bulgarian
, catalan
, czech
,
dutch
, english
, finnish
, french
, galician
,
german
, hindi
, hungarian
, indonesian
, irish
, italian
, latvian
,
lithuanian
, norwegian
, portuguese
, romanian
, russian
, sorani
,
spanish
, swedish
, turkish
.
Reimplementing language analyzers
The built-in language analyzers can be reimplemented as custom
analyzers
(as described below) in order to customize their behaviour.
|
Note
|
If you do not intend to exclude words from being stemmed (the
equivalent of the stem_exclusion parameter above), then you should remove
the keyword_marker token filter from the custom analyzer configuration.
|
arabic
analyzer
The arabic
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /arabic_example
{
"settings": {
"analysis": {
"filter": {
"arabic_stop": {
"type": "stop",
"stopwords": "_arabic_" (1)
},
"arabic_keywords": {
"type": "keyword_marker",
"keywords": ["مثال"] (2)
},
"arabic_stemmer": {
"type": "stemmer",
"language": "arabic"
}
},
"analyzer": {
"rebuilt_arabic": {
"tokenizer": "standard",
"filter": [
"lowercase",
"decimal_digit",
"arabic_stop",
"arabic_normalization",
"arabic_keywords",
"arabic_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
armenian
analyzer
The armenian
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /armenian_example
{
"settings": {
"analysis": {
"filter": {
"armenian_stop": {
"type": "stop",
"stopwords": "_armenian_" (1)
},
"armenian_keywords": {
"type": "keyword_marker",
"keywords": ["օրինակ"] (2)
},
"armenian_stemmer": {
"type": "stemmer",
"language": "armenian"
}
},
"analyzer": {
"rebuilt_armenian": {
"tokenizer": "standard",
"filter": [
"lowercase",
"armenian_stop",
"armenian_keywords",
"armenian_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
basque
analyzer
The basque
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /basque_example
{
"settings": {
"analysis": {
"filter": {
"basque_stop": {
"type": "stop",
"stopwords": "_basque_" (1)
},
"basque_keywords": {
"type": "keyword_marker",
"keywords": ["Adibidez"] (2)
},
"basque_stemmer": {
"type": "stemmer",
"language": "basque"
}
},
"analyzer": {
"rebuilt_basque": {
"tokenizer": "standard",
"filter": [
"lowercase",
"basque_stop",
"basque_keywords",
"basque_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
bengali
analyzer
The bengali
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /bengali_example
{
"settings": {
"analysis": {
"filter": {
"bengali_stop": {
"type": "stop",
"stopwords": "_bengali_" (1)
},
"bengali_keywords": {
"type": "keyword_marker",
"keywords": ["উদাহরণ"] (2)
},
"bengali_stemmer": {
"type": "stemmer",
"language": "bengali"
}
},
"analyzer": {
"rebuilt_bengali": {
"tokenizer": "standard",
"filter": [
"lowercase",
"decimal_digit",
"bengali_keywords",
"indic_normalization",
"bengali_normalization",
"bengali_stop",
"bengali_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
brazilian
analyzer
The brazilian
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /brazilian_example
{
"settings": {
"analysis": {
"filter": {
"brazilian_stop": {
"type": "stop",
"stopwords": "_brazilian_" (1)
},
"brazilian_keywords": {
"type": "keyword_marker",
"keywords": ["exemplo"] (2)
},
"brazilian_stemmer": {
"type": "stemmer",
"language": "brazilian"
}
},
"analyzer": {
"rebuilt_brazilian": {
"tokenizer": "standard",
"filter": [
"lowercase",
"brazilian_stop",
"brazilian_keywords",
"brazilian_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
bulgarian
analyzer
The bulgarian
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /bulgarian_example
{
"settings": {
"analysis": {
"filter": {
"bulgarian_stop": {
"type": "stop",
"stopwords": "_bulgarian_" (1)
},
"bulgarian_keywords": {
"type": "keyword_marker",
"keywords": ["пример"] (2)
},
"bulgarian_stemmer": {
"type": "stemmer",
"language": "bulgarian"
}
},
"analyzer": {
"rebuilt_bulgarian": {
"tokenizer": "standard",
"filter": [
"lowercase",
"bulgarian_stop",
"bulgarian_keywords",
"bulgarian_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
catalan
analyzer
The catalan
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /catalan_example
{
"settings": {
"analysis": {
"filter": {
"catalan_elision": {
"type": "elision",
"articles": [ "d", "l", "m", "n", "s", "t"],
"articles_case": true
},
"catalan_stop": {
"type": "stop",
"stopwords": "_catalan_" (1)
},
"catalan_keywords": {
"type": "keyword_marker",
"keywords": ["example"] (2)
},
"catalan_stemmer": {
"type": "stemmer",
"language": "catalan"
}
},
"analyzer": {
"rebuilt_catalan": {
"tokenizer": "standard",
"filter": [
"catalan_elision",
"lowercase",
"catalan_stop",
"catalan_keywords",
"catalan_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
cjk
analyzer
|
Note
|
You may find that icu_analyzer in the ICU analysis plugin works better
for CJK text than the cjk analyzer. Experiment with your text and queries.
|
The cjk
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /cjk_example
{
"settings": {
"analysis": {
"filter": {
"english_stop": {
"type": "stop",
"stopwords": [ (1)
"a", "and", "are", "as", "at", "be", "but", "by", "for",
"if", "in", "into", "is", "it", "no", "not", "of", "on",
"or", "s", "such", "t", "that", "the", "their", "then",
"there", "these", "they", "this", "to", "was", "will",
"with", "www"
]
}
},
"analyzer": {
"rebuilt_cjk": {
"tokenizer": "standard",
"filter": [
"cjk_width",
"lowercase",
"cjk_bigram",
"english_stop"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. The default stop words are almost the same as theenglish
set, but not exactly the same.
czech
analyzer
The czech
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /czech_example
{
"settings": {
"analysis": {
"filter": {
"czech_stop": {
"type": "stop",
"stopwords": "_czech_" (1)
},
"czech_keywords": {
"type": "keyword_marker",
"keywords": ["příklad"] (2)
},
"czech_stemmer": {
"type": "stemmer",
"language": "czech"
}
},
"analyzer": {
"rebuilt_czech": {
"tokenizer": "standard",
"filter": [
"lowercase",
"czech_stop",
"czech_keywords",
"czech_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
danish
analyzer
The danish
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /danish_example
{
"settings": {
"analysis": {
"filter": {
"danish_stop": {
"type": "stop",
"stopwords": "_danish_" (1)
},
"danish_keywords": {
"type": "keyword_marker",
"keywords": ["eksempel"] (2)
},
"danish_stemmer": {
"type": "stemmer",
"language": "danish"
}
},
"analyzer": {
"rebuilt_danish": {
"tokenizer": "standard",
"filter": [
"lowercase",
"danish_stop",
"danish_keywords",
"danish_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
dutch
analyzer
The dutch
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /dutch_example
{
"settings": {
"analysis": {
"filter": {
"dutch_stop": {
"type": "stop",
"stopwords": "_dutch_" (1)
},
"dutch_keywords": {
"type": "keyword_marker",
"keywords": ["voorbeeld"] (2)
},
"dutch_stemmer": {
"type": "stemmer",
"language": "dutch"
},
"dutch_override": {
"type": "stemmer_override",
"rules": [
"fiets=>fiets",
"bromfiets=>bromfiets",
"ei=>eier",
"kind=>kinder"
]
}
},
"analyzer": {
"rebuilt_dutch": {
"tokenizer": "standard",
"filter": [
"lowercase",
"dutch_stop",
"dutch_keywords",
"dutch_override",
"dutch_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
english
analyzer
The english
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /english_example
{
"settings": {
"analysis": {
"filter": {
"english_stop": {
"type": "stop",
"stopwords": "_english_" (1)
},
"english_keywords": {
"type": "keyword_marker",
"keywords": ["example"] (2)
},
"english_stemmer": {
"type": "stemmer",
"language": "english"
},
"english_possessive_stemmer": {
"type": "stemmer",
"language": "possessive_english"
}
},
"analyzer": {
"rebuilt_english": {
"tokenizer": "standard",
"filter": [
"english_possessive_stemmer",
"lowercase",
"english_stop",
"english_keywords",
"english_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
estonian
analyzer
The estonian
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /estonian_example
{
"settings": {
"analysis": {
"filter": {
"estonian_stop": {
"type": "stop",
"stopwords": "_estonian_" (1)
},
"estonian_keywords": {
"type": "keyword_marker",
"keywords": ["näide"] (2)
},
"estonian_stemmer": {
"type": "stemmer",
"language": "estonian"
}
},
"analyzer": {
"rebuilt_estonian": {
"tokenizer": "standard",
"filter": [
"lowercase",
"estonian_stop",
"estonian_keywords",
"estonian_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
finnish
analyzer
The finnish
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /finnish_example
{
"settings": {
"analysis": {
"filter": {
"finnish_stop": {
"type": "stop",
"stopwords": "_finnish_" (1)
},
"finnish_keywords": {
"type": "keyword_marker",
"keywords": ["esimerkki"] (2)
},
"finnish_stemmer": {
"type": "stemmer",
"language": "finnish"
}
},
"analyzer": {
"rebuilt_finnish": {
"tokenizer": "standard",
"filter": [
"lowercase",
"finnish_stop",
"finnish_keywords",
"finnish_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
french
analyzer
The french
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /french_example
{
"settings": {
"analysis": {
"filter": {
"french_elision": {
"type": "elision",
"articles_case": true,
"articles": [
"l", "m", "t", "qu", "n", "s",
"j", "d", "c", "jusqu", "quoiqu",
"lorsqu", "puisqu"
]
},
"french_stop": {
"type": "stop",
"stopwords": "_french_" (1)
},
"french_keywords": {
"type": "keyword_marker",
"keywords": ["Example"] (2)
},
"french_stemmer": {
"type": "stemmer",
"language": "light_french"
}
},
"analyzer": {
"rebuilt_french": {
"tokenizer": "standard",
"filter": [
"french_elision",
"lowercase",
"french_stop",
"french_keywords",
"french_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
galician
analyzer
The galician
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /galician_example
{
"settings": {
"analysis": {
"filter": {
"galician_stop": {
"type": "stop",
"stopwords": "_galician_" (1)
},
"galician_keywords": {
"type": "keyword_marker",
"keywords": ["exemplo"] (2)
},
"galician_stemmer": {
"type": "stemmer",
"language": "galician"
}
},
"analyzer": {
"rebuilt_galician": {
"tokenizer": "standard",
"filter": [
"lowercase",
"galician_stop",
"galician_keywords",
"galician_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
german
analyzer
The german
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /german_example
{
"settings": {
"analysis": {
"filter": {
"german_stop": {
"type": "stop",
"stopwords": "_german_" (1)
},
"german_keywords": {
"type": "keyword_marker",
"keywords": ["Beispiel"] (2)
},
"german_stemmer": {
"type": "stemmer",
"language": "light_german"
}
},
"analyzer": {
"rebuilt_german": {
"tokenizer": "standard",
"filter": [
"lowercase",
"german_stop",
"german_keywords",
"german_normalization",
"german_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
greek
analyzer
The greek
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /greek_example
{
"settings": {
"analysis": {
"filter": {
"greek_stop": {
"type": "stop",
"stopwords": "_greek_" (1)
},
"greek_lowercase": {
"type": "lowercase",
"language": "greek"
},
"greek_keywords": {
"type": "keyword_marker",
"keywords": ["παράδειγμα"] (2)
},
"greek_stemmer": {
"type": "stemmer",
"language": "greek"
}
},
"analyzer": {
"rebuilt_greek": {
"tokenizer": "standard",
"filter": [
"greek_lowercase",
"greek_stop",
"greek_keywords",
"greek_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
hindi
analyzer
The hindi
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /hindi_example
{
"settings": {
"analysis": {
"filter": {
"hindi_stop": {
"type": "stop",
"stopwords": "_hindi_" (1)
},
"hindi_keywords": {
"type": "keyword_marker",
"keywords": ["उदाहरण"] (2)
},
"hindi_stemmer": {
"type": "stemmer",
"language": "hindi"
}
},
"analyzer": {
"rebuilt_hindi": {
"tokenizer": "standard",
"filter": [
"lowercase",
"decimal_digit",
"hindi_keywords",
"indic_normalization",
"hindi_normalization",
"hindi_stop",
"hindi_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
hungarian
analyzer
The hungarian
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /hungarian_example
{
"settings": {
"analysis": {
"filter": {
"hungarian_stop": {
"type": "stop",
"stopwords": "_hungarian_" (1)
},
"hungarian_keywords": {
"type": "keyword_marker",
"keywords": ["példa"] (2)
},
"hungarian_stemmer": {
"type": "stemmer",
"language": "hungarian"
}
},
"analyzer": {
"rebuilt_hungarian": {
"tokenizer": "standard",
"filter": [
"lowercase",
"hungarian_stop",
"hungarian_keywords",
"hungarian_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
indonesian
analyzer
The indonesian
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /indonesian_example
{
"settings": {
"analysis": {
"filter": {
"indonesian_stop": {
"type": "stop",
"stopwords": "_indonesian_" (1)
},
"indonesian_keywords": {
"type": "keyword_marker",
"keywords": ["contoh"] (2)
},
"indonesian_stemmer": {
"type": "stemmer",
"language": "indonesian"
}
},
"analyzer": {
"rebuilt_indonesian": {
"tokenizer": "standard",
"filter": [
"lowercase",
"indonesian_stop",
"indonesian_keywords",
"indonesian_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
irish
analyzer
The irish
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /irish_example
{
"settings": {
"analysis": {
"filter": {
"irish_hyphenation": {
"type": "stop",
"stopwords": [ "h", "n", "t" ],
"ignore_case": true
},
"irish_elision": {
"type": "elision",
"articles": [ "d", "m", "b" ],
"articles_case": true
},
"irish_stop": {
"type": "stop",
"stopwords": "_irish_" (1)
},
"irish_lowercase": {
"type": "lowercase",
"language": "irish"
},
"irish_keywords": {
"type": "keyword_marker",
"keywords": ["sampla"] (2)
},
"irish_stemmer": {
"type": "stemmer",
"language": "irish"
}
},
"analyzer": {
"rebuilt_irish": {
"tokenizer": "standard",
"filter": [
"irish_hyphenation",
"irish_elision",
"irish_lowercase",
"irish_stop",
"irish_keywords",
"irish_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
italian
analyzer
The italian
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /italian_example
{
"settings": {
"analysis": {
"filter": {
"italian_elision": {
"type": "elision",
"articles": [
"c", "l", "all", "dall", "dell",
"nell", "sull", "coll", "pell",
"gl", "agl", "dagl", "degl", "negl",
"sugl", "un", "m", "t", "s", "v", "d"
],
"articles_case": true
},
"italian_stop": {
"type": "stop",
"stopwords": "_italian_" (1)
},
"italian_keywords": {
"type": "keyword_marker",
"keywords": ["esempio"] (2)
},
"italian_stemmer": {
"type": "stemmer",
"language": "light_italian"
}
},
"analyzer": {
"rebuilt_italian": {
"tokenizer": "standard",
"filter": [
"italian_elision",
"lowercase",
"italian_stop",
"italian_keywords",
"italian_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
latvian
analyzer
The latvian
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /latvian_example
{
"settings": {
"analysis": {
"filter": {
"latvian_stop": {
"type": "stop",
"stopwords": "_latvian_" (1)
},
"latvian_keywords": {
"type": "keyword_marker",
"keywords": ["piemērs"] (2)
},
"latvian_stemmer": {
"type": "stemmer",
"language": "latvian"
}
},
"analyzer": {
"rebuilt_latvian": {
"tokenizer": "standard",
"filter": [
"lowercase",
"latvian_stop",
"latvian_keywords",
"latvian_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
lithuanian
analyzer
The lithuanian
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /lithuanian_example
{
"settings": {
"analysis": {
"filter": {
"lithuanian_stop": {
"type": "stop",
"stopwords": "_lithuanian_" (1)
},
"lithuanian_keywords": {
"type": "keyword_marker",
"keywords": ["pavyzdys"] (2)
},
"lithuanian_stemmer": {
"type": "stemmer",
"language": "lithuanian"
}
},
"analyzer": {
"rebuilt_lithuanian": {
"tokenizer": "standard",
"filter": [
"lowercase",
"lithuanian_stop",
"lithuanian_keywords",
"lithuanian_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
norwegian
analyzer
The norwegian
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /norwegian_example
{
"settings": {
"analysis": {
"filter": {
"norwegian_stop": {
"type": "stop",
"stopwords": "_norwegian_" (1)
},
"norwegian_keywords": {
"type": "keyword_marker",
"keywords": ["eksempel"] (2)
},
"norwegian_stemmer": {
"type": "stemmer",
"language": "norwegian"
}
},
"analyzer": {
"rebuilt_norwegian": {
"tokenizer": "standard",
"filter": [
"lowercase",
"norwegian_stop",
"norwegian_keywords",
"norwegian_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
persian
analyzer
The persian
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /persian_example
{
"settings": {
"analysis": {
"char_filter": {
"zero_width_spaces": {
"type": "mapping",
"mappings": [ "\\u200C=>\\u0020"] (1)
}
},
"filter": {
"persian_stop": {
"type": "stop",
"stopwords": "_persian_" (2)
}
},
"analyzer": {
"rebuilt_persian": {
"tokenizer": "standard",
"char_filter": [ "zero_width_spaces" ],
"filter": [
"lowercase",
"decimal_digit",
"arabic_normalization",
"persian_normalization",
"persian_stop"
]
}
}
}
}
}
-
Replaces zero-width non-joiners with an ASCII space.
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters.
portuguese
analyzer
The portuguese
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /portuguese_example
{
"settings": {
"analysis": {
"filter": {
"portuguese_stop": {
"type": "stop",
"stopwords": "_portuguese_" (1)
},
"portuguese_keywords": {
"type": "keyword_marker",
"keywords": ["exemplo"] (2)
},
"portuguese_stemmer": {
"type": "stemmer",
"language": "light_portuguese"
}
},
"analyzer": {
"rebuilt_portuguese": {
"tokenizer": "standard",
"filter": [
"lowercase",
"portuguese_stop",
"portuguese_keywords",
"portuguese_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
romanian
analyzer
The romanian
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /romanian_example
{
"settings": {
"analysis": {
"filter": {
"romanian_stop": {
"type": "stop",
"stopwords": "_romanian_" (1)
},
"romanian_keywords": {
"type": "keyword_marker",
"keywords": ["exemplu"] (2)
},
"romanian_stemmer": {
"type": "stemmer",
"language": "romanian"
}
},
"analyzer": {
"rebuilt_romanian": {
"tokenizer": "standard",
"filter": [
"lowercase",
"romanian_stop",
"romanian_keywords",
"romanian_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
russian
analyzer
The russian
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /russian_example
{
"settings": {
"analysis": {
"filter": {
"russian_stop": {
"type": "stop",
"stopwords": "_russian_" (1)
},
"russian_keywords": {
"type": "keyword_marker",
"keywords": ["пример"] (2)
},
"russian_stemmer": {
"type": "stemmer",
"language": "russian"
}
},
"analyzer": {
"rebuilt_russian": {
"tokenizer": "standard",
"filter": [
"lowercase",
"russian_stop",
"russian_keywords",
"russian_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
sorani
analyzer
The sorani
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /sorani_example
{
"settings": {
"analysis": {
"filter": {
"sorani_stop": {
"type": "stop",
"stopwords": "_sorani_" (1)
},
"sorani_keywords": {
"type": "keyword_marker",
"keywords": ["mînak"] (2)
},
"sorani_stemmer": {
"type": "stemmer",
"language": "sorani"
}
},
"analyzer": {
"rebuilt_sorani": {
"tokenizer": "standard",
"filter": [
"sorani_normalization",
"lowercase",
"decimal_digit",
"sorani_stop",
"sorani_keywords",
"sorani_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
spanish
analyzer
The spanish
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /spanish_example
{
"settings": {
"analysis": {
"filter": {
"spanish_stop": {
"type": "stop",
"stopwords": "_spanish_" (1)
},
"spanish_keywords": {
"type": "keyword_marker",
"keywords": ["ejemplo"] (2)
},
"spanish_stemmer": {
"type": "stemmer",
"language": "light_spanish"
}
},
"analyzer": {
"rebuilt_spanish": {
"tokenizer": "standard",
"filter": [
"lowercase",
"spanish_stop",
"spanish_keywords",
"spanish_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
swedish
analyzer
The swedish
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /swedish_example
{
"settings": {
"analysis": {
"filter": {
"swedish_stop": {
"type": "stop",
"stopwords": "_swedish_" (1)
},
"swedish_keywords": {
"type": "keyword_marker",
"keywords": ["exempel"] (2)
},
"swedish_stemmer": {
"type": "stemmer",
"language": "swedish"
}
},
"analyzer": {
"rebuilt_swedish": {
"tokenizer": "standard",
"filter": [
"lowercase",
"swedish_stop",
"swedish_keywords",
"swedish_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
turkish
analyzer
The turkish
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /turkish_example
{
"settings": {
"analysis": {
"filter": {
"turkish_stop": {
"type": "stop",
"stopwords": "_turkish_" (1)
},
"turkish_lowercase": {
"type": "lowercase",
"language": "turkish"
},
"turkish_keywords": {
"type": "keyword_marker",
"keywords": ["örnek"] (2)
},
"turkish_stemmer": {
"type": "stemmer",
"language": "turkish"
}
},
"analyzer": {
"rebuilt_turkish": {
"tokenizer": "standard",
"filter": [
"apostrophe",
"turkish_lowercase",
"turkish_stop",
"turkish_keywords",
"turkish_stemmer"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
This filter should be removed unless there are words which should be excluded from stemming.
thai
analyzer
The thai
analyzer could be reimplemented as a custom
analyzer as follows:
PUT /thai_example
{
"settings": {
"analysis": {
"filter": {
"thai_stop": {
"type": "stop",
"stopwords": "_thai_" (1)
}
},
"analyzer": {
"rebuilt_thai": {
"tokenizer": "thai",
"filter": [
"lowercase",
"decimal_digit",
"thai_stop"
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters.
Pattern analyzer
The pattern
analyzer uses a regular expression to split the text into terms.
The regular expression should match the token separators not the tokens
themselves. The regular expression defaults to \W+
(or all non-word characters).
|
Warning
|
Beware of Pathological Regular Expressions
The pattern analyzer uses Java Regular Expressions. A badly written regular expression could run very slowly or even throw a StackOverflowError and cause the node it is running on to exit suddenly. Read more about pathological regular expressions and how to avoid them. |
Example output
POST _analyze
{
"analyzer": "pattern",
"text": "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone."
}
The above sentence would produce the following terms:
[ the, 2, quick, brown, foxes, jumped, over, the, lazy, dog, s, bone ]
Configuration
The pattern
analyzer accepts the following parameters:
pattern
|
A Java regular expression, defaults to |
flags
|
Java regular expression flags.
Flags should be pipe-separated, eg |
lowercase
|
Should terms be lowercased or not. Defaults to |
stopwords
|
A pre-defined stop words list like |
stopwords_path
|
The path to a file containing stop words. |
See the Stop Token Filter for more information about stop word configuration.
Example configuration
In this example, we configure the pattern
analyzer to split email addresses
on non-word characters or on underscores (\W|_
), and to lower-case the result:
PUT my-index-000001
{
"settings": {
"analysis": {
"analyzer": {
"my_email_analyzer": {
"type": "pattern",
"pattern": "\\W|_", (1)
"lowercase": true
}
}
}
}
}
POST my-index-000001/_analyze
{
"analyzer": "my_email_analyzer",
"text"<EMAIL_ADDRESS>}
-
The backslashes in the pattern need to be escaped when specifying the pattern as a JSON string.
The above example produces the following terms:
[ john, smith, foo, bar, com ]
CamelCase tokenizer
The following more complicated example splits CamelCase text into tokens:
PUT my-index-000001
{
"settings": {
"analysis": {
"analyzer": {
"camel": {
"type": "pattern",
"pattern": "([^\\p{L}\\d]+)|(?<=\\D)(?=\\d)|(?<=\\d)(?=\\D)|(?<=[\\p{L}&&[^\\p{Lu}]])(?=\\p{Lu})|(?<=\\p{Lu})(?=\\p{Lu}[\\p{L}&&[^\\p{Lu}]])"
}
}
}
}
}
GET my-index-000001/_analyze
{
"analyzer": "camel",
"text": "MooseX::FTPClass2_beta"
}
The above example produces the following terms:
[ moose, x, ftp, class, 2, beta ]
The regex above is easier to understand as:
([^\p{L}\d]+) # swallow non letters and numbers,
| (?<=\D)(?=\d) # or non-number followed by number,
| (?<=\d)(?=\D) # or number followed by non-number,
| (?<=[ \p{L} && [^\p{Lu}]]) # or lower case
(?=\p{Lu}) # followed by upper case,
| (?<=\p{Lu}) # or upper case
(?=\p{Lu} # followed by upper case
[\p{L}&&[^\p{Lu}]] # then lower case
)
Definition
The pattern
anlayzer consists of:
- Tokenizer
- Token Filters
-
-
Stop Token Filter (disabled by default)
If you need to customize the pattern
analyzer beyond the configuration
parameters then you need to recreate it as a custom
analyzer and modify
it, usually by adding token filters. This would recreate the built-in
pattern
analyzer and you can use it as a starting point for further
customization:
PUT /pattern_example
{
"settings": {
"analysis": {
"tokenizer": {
"split_on_non_word": {
"type": "pattern",
"pattern": "\\W+" (1)
}
},
"analyzer": {
"rebuilt_pattern": {
"tokenizer": "split_on_non_word",
"filter": [
"lowercase" (2)
]
}
}
}
}
}
-
The default pattern is
\W+
which splits on non-word characters and this is where you’d change it. -
You’d add other token filters after
lowercase
.
Simple analyzer
The simple
analyzer breaks text into tokens at any non-letter character, such
as numbers, spaces, hyphens and apostrophes, discards non-letter characters,
and changes uppercase to lowercase.
Example
POST _analyze
{
"analyzer": "simple",
"text": "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone."
}
The simple
analyzer parses the sentence and produces the following
tokens:
[ the, quick, brown, foxes, jumped, over, the, lazy, dog, s, bone ]
Customize
To customize the simple
analyzer, duplicate it to create the basis for
a custom analyzer. This custom analyzer can be modified as required, usually by
adding token filters.
PUT /my-index-000001
{
"settings": {
"analysis": {
"analyzer": {
"my_custom_simple_analyzer": {
"tokenizer": "lowercase",
"filter": [ (1)
]
}
}
}
}
}
-
Add token filters here.
Standard analyzer
The standard
analyzer is the default analyzer which is used if none is
specified. It provides grammar based tokenization (based on the Unicode Text
Segmentation algorithm, as specified in
Unicode Standard Annex #29) and works well
for most languages.
Example output
POST _analyze
{
"analyzer": "standard",
"text": "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone."
}
The above sentence would produce the following terms:
[ the, 2, quick, brown, foxes, jumped, over, the, lazy, dog's, bone ]
Configuration
The standard
analyzer accepts the following parameters:
max_token_length
|
The maximum token length. If a token is seen that exceeds this length then
it is split at |
stopwords
|
A pre-defined stop words list like |
stopwords_path
|
The path to a file containing stop words. |
See the Stop Token Filter for more information about stop word configuration.
Example configuration
In this example, we configure the standard
analyzer to have a
max_token_length
of 5 (for demonstration purposes), and to use the
pre-defined list of English stop words:
PUT my-index-000001
{
"settings": {
"analysis": {
"analyzer": {
"my_english_analyzer": {
"type": "standard",
"max_token_length": 5,
"stopwords": "_english_"
}
}
}
}
}
POST my-index-000001/_analyze
{
"analyzer": "my_english_analyzer",
"text": "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone."
}
The above example produces the following terms:
[ 2, quick, brown, foxes, jumpe, d, over, lazy, dog's, bone ]
Definition
The standard
analyzer consists of:
- Tokenizer
- Token Filters
-
-
Stop Token Filter (disabled by default)
If you need to customize the standard
analyzer beyond the configuration
parameters then you need to recreate it as a custom
analyzer and modify
it, usually by adding token filters. This would recreate the built-in
standard
analyzer and you can use it as a starting point:
PUT /standard_example
{
"settings": {
"analysis": {
"analyzer": {
"rebuilt_standard": {
"tokenizer": "standard",
"filter": [
"lowercase" (1)
]
}
}
}
}
}
-
You’d add any token filters after
lowercase
.
Stop analyzer
The stop
analyzer is the same as the simple
analyzer
but adds support for removing stop words. It defaults to using the
english
stop words.
Example output
POST _analyze
{
"analyzer": "stop",
"text": "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone."
}
The above sentence would produce the following terms:
[ quick, brown, foxes, jumped, over, lazy, dog, s, bone ]
Configuration
The stop
analyzer accepts the following parameters:
stopwords
|
A pre-defined stop words list like |
stopwords_path
|
The path to a file containing stop words. This path is relative to the
OpenSearch |
See the Stop Token Filter for more information about stop word configuration.
Example configuration
In this example, we configure the stop
analyzer to use a specified list of
words as stop words:
PUT my-index-000001
{
"settings": {
"analysis": {
"analyzer": {
"my_stop_analyzer": {
"type": "stop",
"stopwords": ["the", "over"]
}
}
}
}
}
POST my-index-000001/_analyze
{
"analyzer": "my_stop_analyzer",
"text": "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone."
}
The above example produces the following terms:
[ quick, brown, foxes, jumped, lazy, dog, s, bone ]
Definition
It consists of:
- Tokenizer
- Token filters
If you need to customize the stop
analyzer beyond the configuration
parameters then you need to recreate it as a custom
analyzer and modify
it, usually by adding token filters. This would recreate the built-in
stop
analyzer and you can use it as a starting point for further
customization:
PUT /stop_example
{
"settings": {
"analysis": {
"filter": {
"english_stop": {
"type": "stop",
"stopwords": "_english_" (1)
}
},
"analyzer": {
"rebuilt_stop": {
"tokenizer": "lowercase",
"filter": [
"english_stop" (2)
]
}
}
}
}
}
-
The default stopwords can be overridden with the
stopwords
orstopwords_path
parameters. -
You’d add any token filters after
english_stop
.
Whitespace analyzer
The whitespace
analyzer breaks text into terms whenever it encounters a
whitespace character.
Example output
POST _analyze
{
"analyzer": "whitespace",
"text": "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone."
}
The above sentence would produce the following terms:
[ The, 2, QUICK, Brown-Foxes, jumped, over, the, lazy, dog's, bone. ]
Configuration
The whitespace
analyzer is not configurable.
Definition
It consists of:
- Tokenizer
If you need to customize the whitespace
analyzer then you need to
recreate it as a custom
analyzer and modify it, usually by adding
token filters. This would recreate the built-in whitespace
analyzer
and you can use it as a starting point for further customization:
PUT /whitespace_example
{
"settings": {
"analysis": {
"analyzer": {
"rebuilt_whitespace": {
"tokenizer": "whitespace",
"filter": [ (1)
]
}
}
}
}
}
-
You’d add any token filters here.


We are hustlers, makers,
hipsters, and dreamers.
Startup Exchange is the entrepreneurial community that empowers students to support and inspire each other to build their dreams and live an alternative lifestyle.
Schedule of Events
{{ event.title }}
{{ event.subtitle }}
{{ event.description}}
What We Value
Hustle.
The people of Startup Exchange push to make ends meet and work to do what needs to be done. Exchangers are self-starters, persistent for what they want, and always hungry for more.
Innovation.
Startup Exchange thinks differently and challenges the status quo. We encourage people to be creative and disrupt the industries around us.
Empowerment.
Startup Exchange creates a place where people can learn and grow beyond even the Exchange by erasing the illusion of the startup world and making it accessible to anyone.
Community.
Startup Exchange fosters a community of like-minded people that supports and empowers each other to pursue their passions and live an alternative lifestyle.
Keep in Touch.
Sign up for our newsletter to hear about our meetings & events.
<EMAIL_ADDRESS>mentors@startup.exchange


Lesson 3
- Edit the 'users' factory to get real users from '/rest/users' '$http'
- Edit the 'UserCtrl' so:
- if the request is successful (using promises), the users should be printed out
-
if the request is fails (using promises), a proper message should be send to the user
The unhappy path can be tested by passing the query argument 'role=admin'
- Edit the existing tests in order to keep them valid
Go to the home page


We are interested in articulate, well-informed remarks that are relevant to the article. We welcome your advice, your criticism and your unique insights into the issues of the day.
To be approved for publication, your comments should be civil and avoid name-calling.
Our standards for taste are reflected in the articles we publish in the newspaper and on NYTimes.com; we expect your comments to follow that example. A few things we won't tolerate: personal attacks, obscenity, vulgarity, profanity (including expletives and letters followed by dashes), commercial promotion, impersonations, incoherence and SHOUTING.
Our goal is to provide substantive commentary for a general readership. By screening submissions, we have created a space where readers can exchange intelligent and informed commentary that enhances the quality of our news and information.
While most comments will be posted if they are on-topic and not abusive, moderating decisions are subjective. We will make them as carefully and consistently as we can. Because of the volume of reader comments, we cannot review individual moderation decisions with readers and generally cannot alter a comment once it is posted.
The first time you write a comment, you will be asked to complete a quick registration process. It should take you no more than one minute.
You will be asked for a display name and your location. Both of these items will be displayed publicly on NYTimes.com.
If you are writing a comment, please be thoughtful, civil and articulate. In the vast majority of cases, we only accept comments written in English; foreign language comments will be rejected.
The vast majority of comments are reviewed by a human moderator. Because of this, the number of comments that we are capable of moderating each day is limited. Typically, comment threads are open for 24 hours. After that point, we move to newer news. It is unfortunate that some of these discussions do not have the chance to further evolve, but the benefit is that we're able to host a civil comments section.
We ask you to complete the simple NYTimes.com registration process to ensure that you are a "real" person and that you accept our terms and conditions. The registration process serves to facilitate the development of our online community, and ensure that members take responsibility for their writings. For more information, please see our Registration FAQ.
Since comments are moderated, they do not appear on the site until they have been approved. Comments are typically posted faster during business hours. Moderation can be less frequent in the evening and on weekends. Comments by Verified Commenters appear on the site without prior moderation. Comments are displayed directly below articles and blog posts. We reserve the right to display comments in a variety of ways, including within the text of articles or in advertisements.
No. Comments are either approved or rejected. We reserve the right to edit a comment that is quoted or excerpted on NYTimes.com or on our affiliate blogs. In those cases, we may fix spelling, grammar or punctuation.
NYT Picks are a selection of comments that represent a range of views and are judged the most interesting and thoughtful. In some cases, NYT Picks may be selected to highlight comments from a particular region, or readers with first-hand knowledge of an issue.
Click Flag to the right of a comment, select the reasons you believe it should be removed and submit the report. Our moderation staff will review the comment again.
Using your real name is not required, but it is encouraged — we have found that people who use their names carry on more engaging, respectful conversations. We recommend against posting your full email address, for security reasons.
A reminder: Once your comment is published, it can be found in search results on websites like Google and Yahoo. A New York Times reporter may occasionally use the email address you have listed in the My Account area for a follow-up interview. For more information, please see our Privacy Policy.
Note: We no longer require Verified Commenters to use their real names. For more information, see the Verified Commenters FAQ.
We welcome strong opinions and criticism of our work, and do not hesitate to approve critical comments. However, personal attacks against our staff will not be permitted, and any criticism should relate to the article in question. Generally, we will not allow comments to become bogged down with discussions of our moderation policies, and we will moderate accordingly.
The confirmation email is sent automatically, but it may take a little while before it arrives in your Inbox. Also, you may want to make sure that we have your current email address associated with your registration. To check this, please go to www.nytimes.com/email in the My Account area. If the confirmation email was sent to an outdated address, or you did not receive it after 24 hours, please contact us.
If you think you are having problems receiving email from NYTimes.com, please read our Email FAQ.
You are free to change your display name, but these changes will apply to all of your past comments.
We appreciate it when readers and people quoted in articles or blog posts point out errors of fact or emphasis and will investigate all assertions. These suggestions should be sent by email. Many of our blogs provide specific email addresses for feedback and corrections. To avoid distracting other readers, we won't publish comments that suggest a correction. Instead, corrections will be made in a blog post or at the top of an article, and also can be found on the main Corrections page.
To send a message about news coverage, email<EMAIL_ADDRESS>or call toll-free at 1-888-NYT-NEWS (1-888-698-6397). Comments on editorials may be emailed to<EMAIL_ADDRESS>or faxed to<PHONE_NUMBER>. Readers dissatisfied with a response or concerned about the paper's journalistic integrity may reach the public editor at<EMAIL_ADDRESS>or<PHONE_NUMBER>.
If you have additional questions regarding comments on NYTimes.com, please contact us.


In 2021, the open source movement will be celebrating several important milestones, including the 20th Anniversary of Wikipedia and Creative Commons, as well as the 30th Anniversary of Linux and the World Wide Web.
Celebrating Anniversaries is much more than just throwing a party. It is also an opportunity to look back in history, to reevaluate the mission and values, and to build a new future for an organization or community.
With that in mind, Open Anniversary aims to provide an innovative and inclusive platform for the community to celebrate, remember, and reimagine the open source movement.
The 2021 Open Anniversary celebrations will revolve around 12 themes, one for each month of the year. A major milestone was selected for each theme. For example: the first month, January, will be dedicated to celebrating "Free Culture," when both Wikipedia and Creative Commons share their 20th Anniversary. The themes are presented below:
On January 15, 2001, Wikipedia was established. Creative Commons was coincidentally established on the same date. Wikipedia was created by Jimmy Wales and Larry Sanger, while Creative Commons was founded by Lawrence Lessig, Hal Abelson, and Eric Eldred. Both Creative Commons and Wikipedia are highlights from the free culture movement.
The "open source" label was created by Christine Peterson at a strategy session held on February 3rd, 1998 in Palo Alto, California. That same month, the Open Source Initiative (OSI) was founded by Bruce Perens and Eric Raymond as a general educational and advocacy organization. The group adopted the Open Source Definition for open-source software, based on the Debian Free Software Guidelines.
The book "The Open Organization" published by Jim Whitehurst, former CEO at Red Hat and current president at IBM, provides great insights on how to build a successful business based on many open source principles, including transparency, participation, and community. Red Had celebrates its anniversary in March--Bob Young founded ACC Corporation in March 1993 and the company was later renamed to Red Hat Software when merged with Marc Ewing's business.
The Sunlight Foundation, a leading nonprofit that advocates for open government, was founded in April 2006 by Ellen S. Miller and Michael R. Klein. Open government is the governing doctrine which holds that citizens have the right to access the documents and proceedings of the government to allow for effective public oversight.
The Internet Archive was founded by Brewster Kahle in May 1996 and the Wayback Machine was launched in 2001. The Internet Archive is a digital library with the stated mission of "universal access to all knowledge," providing free public access to collections of digitized materials, including websites, software applications, games, music, movies/videos, moving images, and books.
The Open Source Hardware Association (OSHWA) was established as an organization in June 2012 by engineer Alicia Gibb, who had been working on the Open Hardware Summit. The OSHWA is a non-profit organization that advocates for open-source hardware.
Open Educational Resources (OER), one of the key components of Open Education, was first coined in July 2002 at UNESCO's Forum on Open Courseware. OER are teaching, learning and research materials in any medium – digital or otherwise – that reside in the public domain or have been released under an open license that permits no-cost access, use, adaptation and redistribution by others with no or limited restrictions.
Tim Berners-Lee wrote a proposal to develop a distributed information system in March 1989, while working at CERN (European Organization for Nuclear Research). On August 6, 1991, he published the first website, dedicated to the World Wide Web project itself, and announced it on the Internet. In 1993, the World Wide Web project was made available with an open license, allowing the Open Web to flourish.
On August 25, 1991, Linus Torvalds, then a 21-year-old university student from Finland, announced a project on the comp.os.minix user group. The initial release (version 0.0.1) was created on September 17, 1991, and the official public release happened on October 5, 1991. In January 1992, Linux adopted the GNU GPLv2 license. This "hobby project" would later become one of the most prominent examples of free and open-source software collaboration.
The Free Software Foundation (FSF) is a non-profit organization founded by Richard Stallman on October 4, 1985 to support the free software movement, which promotes the universal freedom to study, distribute, create, and modify computer software. The GNU General Public License was written in 1989 for use with programs released as part of the GNU project.
Aaron Swartz was born November 8, 1986 and was one of the leading voices for open access. He published the "Guerilla Open Access Manifesto," helped launch the Progressive Change Campaign Committee, and founded the online group Demand Progress, known for its campaign against the Stop Online Piracy Act. His death was the result of years of disproportional prosecution from the government.
On December 7, 2007, a meeting held in Sebastopol, California, was designed to develop a set of principles of "open public data." Open data is data that can be freely used, re-used and redistributed by anyone - subject only, at most, to the requirement to attribute and sharealike.


|
LaraCraft
|
#include <Chunk.hpp>
Public Member Functions | |
| Chunk (const glm::vec2 &position, const HeightMap &heightMap, const OffsetTextureMap &textureMap) | |
| const ShapeVertex * | getDataPointer () const |
| std::vector< glimac::ShapeVertex > | getDataVector () const |
| GLsizei | getVertexCount () const |
Public Attributes | |
| std::vector< std::vector< std::vector< bool > > > | isFilled3DVector |
Represents a 16*256*16 bundle of cubes. In first approximation, a chunk section is made of only one texture. A coordinate (x,y,z) coordinate in the matrix can be either empty (no cube) or filled with a cube of the material.
|
inline |
Constructor for the Chunk
| position | Coordinates x and z on the map |
| heightMap | The HeightMap used |
| textureMap | OffsetTextureMap used |
| const ShapeVertex * Chunk::getDataPointer | ( | ) | const |
| std::vector< glimac::ShapeVertex > Chunk::getDataVector | ( | ) | const |
| GLsizei Chunk::getVertexCount | ( | ) | const |
| std::vector<std::vector<std::vector<bool> > > Chunk::isFilled3DVector |


module OpenSSL
OpenSSL provides SSL, TLS and general purpose cryptography. It wraps the OpenSSL library.
Examples¶ ↑
All examples assume you have loaded OpenSSL with:
require 'openssl'
These examples build atop each other. For example the key created in the next is used in throughout these examples.
Keys¶ ↑
Creating a Key¶ ↑
This example creates a 2048 bit RSA keypair and writes it to the current directory.
key = OpenSSL::PKey::RSA.new 2048 open 'private_key.pem', 'w' do |io| io.write key.to_pem end open 'public_key.pem', 'w' do |io| io.write key.public_key.to_pem end
Exporting a Key¶ ↑
Keys saved to disk without encryption are not secure as anyone who gets ahold of the key may use it unless it is encrypted. In order to securely export a key you may export it with a pass phrase.
cipher = OpenSSL::Cipher.new 'AES-128-CBC' pass_phrase = 'my secure pass phrase goes here' key_secure = key.export cipher, pass_phrase open 'private.secure.pem', 'w' do |io| io.write key_secure end
OpenSSL::Cipher.ciphers returns a list of available ciphers.
Loading a Key¶ ↑
A key can also be loaded from a file.
key2 = OpenSSL::PKey::RSA.new File.read 'private_key.pem' key2.public? # => true key2.private? # => true
or
key3 = OpenSSL::PKey::RSA.new File.read 'public_key.pem' key3.public? # => true key3.private? # => false
Loading an Encrypted Key¶ ↑
OpenSSL will prompt you for your pass phrase when loading an encrypted key. If you will not be able to type in the pass phrase you may provide it when loading the key:
key4_pem = File.read 'private.secure.pem' pass_phrase = 'my secure pass phrase goes here' key4 = OpenSSL::PKey::RSA.new key4_pem, pass_phrase
RSA Encryption¶ ↑
RSA provides encryption and decryption using the public and private keys. You can use a variety of padding methods depending upon the intended use of encrypted data.
Encryption & Decryption¶ ↑
Asymmetric public/private key encryption is slow and victim to attack in cases where it is used without padding or directly to encrypt larger chunks of data. Typical use cases for RSA encryption involve “wrapping” a symmetric key with the public key of the recipient who would “unwrap” that symmetric key again using their private key. The following illustrates a simplified example of such a key transport scheme. It shouldn't be used in practice, though, standardized protocols should always be preferred.
wrapped_key = key.public_encrypt key
A symmetric key encrypted with the public key can only be decrypted with the corresponding private key of the recipient.
original_key = key.private_decrypt wrapped_key
By default PKCS#1 padding will be used, but it is also possible to use other forms of padding, see PKey::RSA for further details.
Signatures¶ ↑
Using “private_encrypt” to encrypt some data with the private key is equivalent to applying a digital signature to the data. A verifying party may validate the signature by comparing the result of decrypting the signature with “public_decrypt” to the original data. However, OpenSSL::PKey already has methods “sign” and “verify” that handle digital signatures in a standardized way - “private_encrypt” and “public_decrypt” shouldn't be used in practice.
To sign a document, a cryptographically secure hash of the document is computed first, which is then signed using the private key.
digest = OpenSSL::Digest::SHA256.new signature = key.sign digest, document
To validate the signature, again a hash of the document is computed and the signature is decrypted using the public key. The result is then compared to the hash just computed, if they are equal the signature was valid.
digest = OpenSSL::Digest::SHA256.new if key.verify digest, signature, document puts 'Valid' else puts 'Invalid' end
PBKDF2 Password-based Encryption¶ ↑
If supported by the underlying OpenSSL version used, Password-based Encryption should use the features of PKCS5. If not supported or if required by legacy applications, the older, less secure methods specified in RFC 2898 are also supported (see below).
PKCS5 supports PBKDF2 as it was specified in PKCS#5 v2.0. It still uses a password, a salt, and additionally a number of iterations that will slow the key derivation process down. The slower this is, the more work it requires being able to brute-force the resulting key.
Encryption¶ ↑
The strategy is to first instantiate a Cipher for encryption, and then to generate a random IV plus a key derived from the password using PBKDF2. PKCS #5 v2.0 recommends at least 8 bytes for the salt, the number of iterations largely depends on the hardware being used.
cipher = OpenSSL::Cipher.new 'AES-128-CBC' cipher.encrypt iv = cipher.random_iv pwd = 'some hopefully not to easily guessable password' salt = OpenSSL::Random.random_bytes 16 iter = 20000 key_len = cipher.key_len digest = OpenSSL::Digest::SHA256.new key = OpenSSL::PKCS5.pbkdf2_hmac(pwd, salt, iter, key_len, digest) cipher.key = key Now encrypt the data: encrypted = cipher.update document encrypted << cipher.final
Decryption¶ ↑
Use the same steps as before to derive the symmetric AES key, this time setting the Cipher up for decryption.
cipher = OpenSSL::Cipher.new 'AES-128-CBC' cipher.decrypt cipher.iv = iv # the one generated with #random_iv pwd = 'some hopefully not to easily guessable password' salt = ... # the one generated above iter = 20000 key_len = cipher.key_len digest = OpenSSL::Digest::SHA256.new key = OpenSSL::PKCS5.pbkdf2_hmac(pwd, salt, iter, key_len, digest) cipher.key = key Now decrypt the data: decrypted = cipher.update encrypted decrypted << cipher.final
PKCS #5 Password-based Encryption¶ ↑
PKCS #5 is a password-based encryption standard documented at RFC2898. It allows a short password or passphrase to be used to create a secure encryption key. If possible, PBKDF2 as described above should be used if the circumstances allow it.
PKCS #5 uses a Cipher, a pass phrase and a salt to generate an encryption key.
pass_phrase = 'my secure pass phrase goes here' salt = '8 octets'
Encryption¶ ↑
First set up the cipher for encryption
encryptor = OpenSSL::Cipher.new 'AES-128-CBC' encryptor.encrypt encryptor.pkcs5_keyivgen pass_phrase, salt
Then pass the data you want to encrypt through
encrypted = encryptor.update 'top secret document' encrypted << encryptor.final
Decryption¶ ↑
Use a new Cipher instance set up for decryption
decryptor = OpenSSL::Cipher.new 'AES-128-CBC' decryptor.decrypt decryptor.pkcs5_keyivgen pass_phrase, salt
Then pass the data you want to decrypt through
plain = decryptor.update encrypted plain << decryptor.final
X509 Certificates¶ ↑
Creating a Certificate¶ ↑
This example creates a self-signed certificate using an RSA key and a SHA1 signature.
key = OpenSSL::PKey::RSA.new 2048 name = OpenSSL::X509::Name.parse 'CN=nobody/DC=example' cert = OpenSSL::X509::Certificate.new cert.version = 2 cert.serial = 0 cert.not_before = Time.now cert.not_after = Time.now + 3600 cert.public_key = key.public_key cert.subject = name
Certificate Extensions¶ ↑
You can add extensions to the certificate with OpenSSL::SSL::ExtensionFactory to indicate the purpose of the certificate.
extension_factory = OpenSSL::X509::ExtensionFactory.new nil, cert cert.add_extension extension_factory.create_extension('basicConstraints', 'CA:FALSE', true) cert.add_extension extension_factory.create_extension( 'keyUsage', 'keyEncipherment,dataEncipherment,digitalSignature') cert.add_extension extension_factory.create_extension('subjectKeyIdentifier', 'hash')
The list of supported extensions (and in some cases their possible values) can be derived from the “objects.h” file in the OpenSSL source code.
Signing a Certificate¶ ↑
To sign a certificate set the issuer and use OpenSSL::X509::Certificate#sign with a digest algorithm. This creates a self-signed cert because we're using the same name and key to sign the certificate as was used to create the certificate.
cert.issuer = name cert.sign key, OpenSSL::Digest::SHA1.new open 'certificate.pem', 'w' do |io| io.write cert.to_pem end
Loading a Certificate¶ ↑
Like a key, a cert can also be loaded from a file.
cert2 = OpenSSL::X509::Certificate.new File.read 'certificate.pem'
Verifying a Certificate¶ ↑
Certificate#verify will return true when a certificate was signed with the given public key.
raise 'certificate can not be verified' unless cert2.verify key
Certificate Authority¶ ↑
A certificate authority (CA) is a trusted third party that allows you to verify the ownership of unknown certificates. The CA issues key signatures that indicate it trusts the user of that key. A user encountering the key can verify the signature by using the CA's public key.
CA Key¶ ↑
CA keys are valuable, so we encrypt and save it to disk and make sure it is not readable by other users.
ca_key = OpenSSL::PKey::RSA.new 2048 pass_phrase = 'my secure pass phrase goes here' cipher = OpenSSL::Cipher.new 'AES-128-CBC' open 'ca_key.pem', 'w', 0400 do |io| io.write ca_key.export(cipher, pass_phrase) end
CA Certificate¶ ↑
A CA certificate is created the same way we created a certificate above, but with different extensions.
ca_name = OpenSSL::X509::Name.parse 'CN=ca/DC=example' ca_cert = OpenSSL::X509::Certificate.new ca_cert.serial = 0 ca_cert.version = 2 ca_cert.not_before = Time.now ca_cert.not_after = Time.now + 86400 ca_cert.public_key = ca_key.public_key ca_cert.subject = ca_name ca_cert.issuer = ca_name extension_factory = OpenSSL::X509::ExtensionFactory.new extension_factory.subject_certificate = ca_cert extension_factory.issuer_certificate = ca_cert ca_cert.add_extension extension_factory.create_extension('subjectKeyIdentifier', 'hash')
This extension indicates the CA's key may be used as a CA.
ca_cert.add_extension extension_factory.create_extension('basicConstraints', 'CA:TRUE', true)
This extension indicates the CA's key may be used to verify signatures on both certificates and certificate revocations.
ca_cert.add_extension extension_factory.create_extension( 'keyUsage', 'cRLSign,keyCertSign', true)
Root CA certificates are self-signed.
ca_cert.sign ca_key, OpenSSL::Digest::SHA1.new
The CA certificate is saved to disk so it may be distributed to all the users of the keys this CA will sign.
open 'ca_cert.pem', 'w' do |io| io.write ca_cert.to_pem end
Certificate Signing Request¶ ↑
The CA signs keys through a Certificate Signing Request (CSR). The CSR contains the information necessary to identify the key.
csr = OpenSSL::X509::Request.new csr.version = 0 csr.subject = name csr.public_key = key.public_key csr.sign key, OpenSSL::Digest::SHA1.new
A CSR is saved to disk and sent to the CA for signing.
open 'csr.pem', 'w' do |io| io.write csr.to_pem end
Creating a Certificate from a CSR¶ ↑
Upon receiving a CSR the CA will verify it before signing it. A minimal verification would be to check the CSR's signature.
csr = OpenSSL::X509::Request.new File.read 'csr.pem' raise 'CSR can not be verified' unless csr.verify csr.public_key
After verification a certificate is created, marked for various usages, signed with the CA key and returned to the requester.
csr_cert = OpenSSL::X509::Certificate.new csr_cert.serial = 0 csr_cert.version = 2 csr_cert.not_before = Time.now csr_cert.not_after = Time.now + 600 csr_cert.subject = csr.subject csr_cert.public_key = csr.public_key csr_cert.issuer = ca_cert.subject extension_factory = OpenSSL::X509::ExtensionFactory.new extension_factory.subject_certificate = csr_cert extension_factory.issuer_certificate = ca_cert csr_cert.add_extension extension_factory.create_extension('basicConstraints', 'CA:FALSE') csr_cert.add_extension extension_factory.create_extension( 'keyUsage', 'keyEncipherment,dataEncipherment,digitalSignature') csr_cert.add_extension extension_factory.create_extension('subjectKeyIdentifier', 'hash') csr_cert.sign ca_key, OpenSSL::Digest::SHA1.new open 'csr_cert.pem', 'w' do |io| io.write csr_cert.to_pem end
SSL and TLS Connections¶ ↑
Using our created key and certificate we can create an SSL or TLS connection. An SSLContext is used to set up an SSL session.
context = OpenSSL::SSL::SSLContext.new
SSL Server¶ ↑
An SSL server requires the certificate and private key to communicate securely with its clients:
context.cert = cert context.key = key
Then create an SSLServer with a TCP server socket and the context. Use the SSLServer like an ordinary TCP server.
require 'socket' tcp_server = TCPServer.new 5000 ssl_server = OpenSSL::SSL::SSLServer.new tcp_server, context loop do ssl_connection = ssl_server.accept data = connection.gets response = "I got #{data.dump}" puts response connection.puts "I got #{data.dump}" connection.close end
SSL client¶ ↑
An SSL client is created with a TCP socket and the context. SSLSocket#connect must be called to initiate the SSL handshake and start encryption. A key and certificate are not required for the client socket.
Note that SSLSocket#close doesn't close the underlying socket by default. Set SSLSocket#sync_close to true if you want.
require 'socket' tcp_socket = TCPSocket.new 'localhost', 5000 ssl_client = OpenSSL::SSL::SSLSocket.new tcp_socket, context ssl_client.sync_close = true ssl_client.connect ssl_client.puts "hello server!" puts ssl_client.gets ssl_client.close # shutdown the TLS connection and close tcp_socket
Peer Verification¶ ↑
An unverified SSL connection does not provide much security. For enhanced security the client or server can verify the certificate of its peer.
The client can be modified to verify the server's certificate against the certificate authority's certificate:
context.ca_file = 'ca_cert.pem' context.verify_mode = OpenSSL::SSL::VERIFY_PEER require 'socket' tcp_socket = TCPSocket.new 'localhost', 5000 ssl_client = OpenSSL::SSL::SSLSocket.new tcp_socket, context ssl_client.connect ssl_client.puts "hello server!" puts ssl_client.gets
If the server certificate is invalid or context.ca_file
is not
set when verifying peers an OpenSSL::SSL::SSLError will be raised.
Constants
- OPENSSL_FIPS
Boolean indicating whether OpenSSL is FIPS-enabled or not
- OPENSSL_LIBRARY_VERSION
Version of OpenSSL the ruby OpenSSL extension is running with
- OPENSSL_VERSION
Version of OpenSSL the ruby OpenSSL extension was built with
- OPENSSL_VERSION_NUMBER
Version number of OpenSSL the ruby OpenSSL extension was built with (base 16)
- VERSION
OpenSSL ruby extension version
Public Class Methods
Returns a Digest subclass by
name
.
require 'openssl' OpenSSL::Digest("MD5") # => OpenSSL::Digest::MD5 Digest("Foo") # => NameError: wrong constant name Foo
# File ext/openssl/lib/openssl/digest.rb, line 71 def Digest(name) OpenSSL::Digest.const_get(name) end
static VALUE ossl_debug_get(VALUE self) { return dOSSL; }
Turns on or off debug mode. With debug mode, all erros added to the OpenSSL error queue will be printed to stderr.
static VALUE ossl_debug_set(VALUE self, VALUE val) { dOSSL = RTEST(val) ? Qtrue : Qfalse; return val; }
See any remaining errors held in queue.
Any errors you see here are probably due to a bug in ruby's OpenSSL implementation.
VALUE ossl_get_errors(void) { VALUE ary; long e; ary = rb_ary_new(); while ((e = ERR_get_error()) != 0){ rb_ary_push(ary, rb_str_new2(ERR_error_string(e, NULL))); } return ary; }
Turns FIPS mode on or off. Turning on FIPS mode will obviously only have an effect for FIPS-capable installations of the OpenSSL library. Trying to do so otherwise will result in an error.
Examples¶ ↑
OpenSSL.fips_mode = true # turn FIPS mode on OpenSSL.fips_mode = false # and off again
static VALUE ossl_fips_mode_set(VALUE self, VALUE enabled) { #ifdef OPENSSL_FIPS if (RTEST(enabled)) { int mode = FIPS_mode(); if(!mode && !FIPS_mode_set(1)) /* turning on twice leads to an error */ ossl_raise(eOSSLError, "Turning on FIPS mode failed"); } else { if(!FIPS_mode_set(0)) /* turning off twice is OK */ ossl_raise(eOSSLError, "Turning off FIPS mode failed"); } return enabled; #else if (RTEST(enabled)) ossl_raise(eOSSLError, "This version of OpenSSL does not support FIPS mode"); return enabled; #endif }
Private Instance Methods
Returns a Digest subclass by
name
.
require 'openssl' OpenSSL::Digest("MD5") # => OpenSSL::Digest::MD5 Digest("Foo") # => NameError: wrong constant name Foo
# File ext/openssl/lib/openssl/digest.rb, line 71 def Digest(name) OpenSSL::Digest.const_get(name) end


const gauss = require("guassian")
const normalRandom = require("random-normal")
const cityBonus = .2;
class Tile {
/**
* Creates a new Tile object
*
* @param {Integer} x the x position in the game board
* @param {Integer} y the y position in the game board
* @param {Float} popMean the population mean of a normal random variable for population
* @param {Float} popDev the S.T. deviation of a normal random variable for population
* @param {Float} taxMean the population mean of a normal random variable for baseTax
* @param {Float} taxDev the S.T. deviation of a normal random variable for baseTax
*/
constructor(x, y, popMean, popDev, taxMean, taxDev) {
this.colonized = false;
// Tiles location in board
this.x = x;
this.y = y;
// Mean and STD for normal random variable representing population
this.popMean = popMean;
this.popDev = popDev;
this.popVar = this.popDev * this.popDev;
// Mean and STD for normal random variable representing baseTax
this.taxMean = taxMean;
this.taxDev = taxDev;
// Generation of baseTax and population as Normal random variables
this.baseTax = randomNormal({
mean: this.taxMean,
dev: this.taxDev
});
this.population = randomNormal({
mean: this.popMean,
dev: this.popDev
});
// Various inital calculations
if (this.baseTax > 0) this.baseTax = 1;
this.isCity();
this.localTaxEff = this.baseTax / this.taxMean;
if (this.isCity) this.localTaxEff += this.localTaxEff * cityBonus;
}
/**
* Calculates and returns the tax paid per turn. This is calculated by multiplying
* basetax by population and a localTaxEff, which is proportional to the inverse
* of size. Cities have an added bonus of 20%
*/
collectTaxes() {
return this.localTaxEff * this.population * this.baseTax;
}
/**
* Determins if the Tiles population is in the 80th percentile. If it is
* this tile becomes a city. Which has an added city tax bonus
*/
isCity() {
let dist = gaussian(this.popMean, this.popVar);
if (dist.cdf(this.population) > .8) {
this.isCity = true;
}
this.isCity = false;
}
}
module.exports = Tile;


Creates a full screen overlay containing a large block of formatted text that can be scrolled with buttons available.
If connecting to SDL Core v.6.0+, the scrollable message can be canceled programmatically using the cancelID
. On older versions of SDL Core, the scrollable message will persist until the user has interacted with the scrollable message or the specified timeout has elapsed.
@since SDL 2.0
Convenience init for creating a scrolling message with text.
- (nonnull instancetype)initWithMessage:(nonnull NSString *)message;
init(message: String)
Body of text that can include newlines and tabs
A SDLScrollableMessage object
Convenience init for creating a scrolling message with text and buttons.
- (nonnull instancetype)initWithMessage:(nonnull NSString *)message
timeout:(UInt16)timeout
softButtons:
(nullable NSArray<SDLSoftButton *> *)softButtons
cancelID:(UInt32)cancelID;
init(message: String, timeout: UInt16, softButtons: [SDLSoftButton]?, cancelID: UInt32)
Body of text that can include newlines and tabs
Indicates how long of a timeout from the last action (i.e. scrolling message resets timeout)
Buttons for the displayed scrollable message
An ID for this specific scrollable message to allow cancellation through the CancelInteraction
RPC
A SDLScrollableMessage object
Body of text that can include newlines and tabs.
String, Required, Max length 500 chars
@since SDL 2.0
@property (nonatomic, strong) NSString *_Nonnull scrollableMessageBody;
var scrollableMessageBody: String { get set }
App defined timeout. Indicates how long of a timeout from the last action (i.e. scrolling message resets timeout). If not set, a default value of 30 seconds is used by Core.
Integer, Optional, Min value: 1000, Max value: 65535, Default value: 30000
@since SDL 2.0
@property (nonatomic, strong, nullable) NSNumber<SDLInt> *timeout;
var timeout: (NSNumber & SDLInt)? { get set }
Buttons for the displayed scrollable message. If omitted on supported displays, only the system defined “Close” SoftButton will be displayed.
Array of SDLSoftButton, Optional, Array size: 0-8
Since SDL 2.0
@property (nonatomic, strong, nullable) NSArray<SDLSoftButton *> *softButtons;
var softButtons: [SDLSoftButton]? { get set }
An ID for this specific scrollable message to allow cancellation through the CancelInteraction
RPC.
Integer, Optional
See
SDLCancelInteraction @since SDL 6.0@property (nonatomic, strong, nullable) NSNumber<SDLInt> *cancelID;
var cancelID: (NSNumber & SDLInt)? { get set }


You may add collaborators on your booking to share resources with coworkers.


Note that due to copyright issues, we cannot post the actual slides. That said, please feel free to visit any of the course staff to go over the material.


Handcrafted, home-made masterpieces
Premium Materials
Our trombones use the shiniest brass which is sourced locally. This will increase the longevity of your purchase
Fast Shipping
We make sure you recieve your trombone as soon as we have finished making it. We also provide free returns if you are not satisfied.
Quality Assurance
For every purchase you make, we will ensure there are no damages or faults and we will check and test the pitch of your instrument.
JTenor Trombone
$600
Lorem ipsum
Lorem ipsum dolor
Lorem ipsum
JTenor Trombone
$600
Lorem ipsum
Lorem ipsum dolor
Lorem ipsum
JTenor Trombone
$600
Lorem ipsum
Lorem ipsum dolor
Lorem ipsum


Pregnant mum kills masked gunman who threatened daughter, 11, and husband in home
The woman, 31, opened fire after husband Jeremy King was pistol whipped during the terrifying incident in Florida
by Dave BurkeA pregnant mum shot an armed man dead after he and another broke into her home and grabbed her 11-year-old daughter.
The masked burglars also pistol-whipped her husband during the terrifying break-in, which happened in Florida.

Although the woman, who is eight months pregnant, has not been named, her husband has spoken about the incident.
Mr King told Bay News 9 : "Them guys came in with two normal pistols and my AR stopped it.
“[My wife] evened the playing field and kept them from killing me."
Mr King, 34, was left with a fractured eye socket, a fractured sinus cavity and concussion after what he described as a "severe beating".
He said he had explained to the men that he had nothing to give them.
His 31-year-old wife was in a back bedroom when she came out to see what happened, and one of the men opened fire.
She grabbed her husband's weapon and shot back, fatally injuring the man as he tried to escape.
He made it 200 yards before collapsing in a ditch, where his body was found.
His accomplice managed to get away, and police are still looking for him.
Hillsborough County Sheriff's Maj. Frank Losat told NBC News : "Two unknown males broke in and made demands of them.

"During that incident, the female homeowner retrieved a firearm, which was in the house legally, and fired one round which struck the male victim that was deceased in the ditch."
He said he did not think the family was targeted randomly, but Mr King said he has no idea who the men are.


HIV testing health test PITC []. users care migrants data offer study {border-style AIDS HPs Health people [User px;} treatment #ffffff;} consent sexual know participants collab: facilities qualitative HIV. MSM MSM] different populations Pathogens high acceptable don’t patients reported sex counselling countries ethnic generally new procedures results risk services time undocumented Available Public [HP adherence approach community-based confidentiality from: medical men migrant want cases collection concerns diagnosis did entitlements group guidelines information informed like minority particularly perceived persons prevalence primary px; result routine social stigma studies tests transgender {font-family However, Latin Medicine Sciences access ask asking attitudes community continued disease font-size importance included individuals levels minorities needed offered population principles process settings status systematic universal way Biology Ethical European Geneva: Health: Lancet: Sex Sexual Viral antiretroviral based challenges country double; endorsement ethical foreign-born identify implementation important interviews need negative policies response rights-based settings. study. terms treatment. vulnerable women work workers years yes, African American Barcelona Counselling HPs’ Life Policy Spanish TW] [,] [], and/or appreciated blood border-color border-width compulsory conducted dimensions epidemics explain feel generalised identified including infection likely majority mobile modalities. overlapping perceptions person positive practice provided public reach recommendations remain research rights screening solid; spite sub-Saharan targeted times underpinning uptake various views {font-style {font-weight (PITC) ./S-()- ./s--- BMC Behav: British Care ECDC Eur Grant HIV, Immunodeficiency Int Lentivirus: Maria Microbial Microbiology Organisms People Research Retroviruses Routine Spain Spain: Testing United Viruses Viruses: World [,]. applied approaches areas attention better bold case certain common consent. consequences considered contexts courier; developed discussions drugs. earlier em; embarrassment emerging evaluation exceptional good healthcare human increasingly internalised it. knowledge needs no, obtained offering operational opt-out origin potential prevention professionals provide provider-initiated providers pt;} rates received review said sciences security stigma. suggest suitability systematically tested testers think time-consuming transcripts treatment, units users, varied venues


Test passed in 4 seconds on T-Mobile myTouch 3G
| Timestamp | Level | Tag | Message |
|---|---|---|---|
| 02-12 12:28:15.021 | info | TestRunner | started: testValidValues_StartsNewActivity(com.example.spoon.ordering.tests.LoginActivityTest) |
| 02-12 12:28:15.161 | warn | IInputConnectionWrapper | showStatusIcon on inactive InputConnection |
| 02-12 12:28:15.791 | warn | KeyCharacterMap | No keyboard for id -1 |
| 02-12 12:28:15.791 | warn | KeyCharacterMap | Using default keymap: /system/usr/keychars/qwerty.kcm.bin |
| 02-12 12:28:16.061 | debug | dalvikvm | GC_EXTERNAL_ALLOC freed 1402 objects / 76896 bytes in 193ms |
| 02-12 12:28:16.711 | debug | dalvikvm | GC_EXTERNAL_ALLOC freed 1221 objects / 71816 bytes in 132ms |
| 02-12 12:28:16.951 | info | global | Default buffer size used in BufferedOutputStream constructor. It would be better to be explicit if an 8k buffer is required. |
| 02-12 12:28:17.711 | debug | dalvikvm | GC_EXTERNAL_ALLOC freed 487 objects / 39000 bytes in 149ms |
| 02-12 12:28:17.781 | info | global | Default buffer size used in BufferedOutputStream constructor. It would be better to be explicit if an 8k buffer is required. |
| 02-12 12:28:18.491 | info | dalvikvm | Jit: resizing JitTable from 1024 to 2048 |
| 02-12 12:28:19.251 | debug | dalvikvm | GC_EXTERNAL_ALLOC freed 1404 objects / 104640 bytes in 128ms |
| 02-12 12:28:19.291 | info | global | Default buffer size used in BufferedOutputStream constructor. It would be better to be explicit if an 8k buffer is required. |
| 02-12 12:28:19.821 | debug | dalvikvm | GC_EXPLICIT freed 565 objects / 39792 bytes in 112ms |
| 02-12 12:28:19.931 | debug | dalvikvm | GC_EXPLICIT freed 28 objects / 712 bytes in 98ms |
| 02-12 12:28:19.931 | info | TestRunner | finished: testValidValues_StartsNewActivity(com.example.spoon.ordering.tests.LoginActivityTest) |


--- layout: post status: publish published: true title: NTFS Data Streams and .NET author: display_name: Tomas Restrepo login: tomasr email<EMAIL_ADDRESS>url: http://winterdom.com/ author_login: tomasr author_email<EMAIL_ADDRESS>author_url: http://winterdom.com/ wordpress_id: 718 wordpress_url: http://winterdom.com/2008/03/ntfsdatastreamsandnet date: '2008-03-08 08:34:13 +0000' date_gmt: '2008-03-08 08:34:13 +0000' categories: - ".NET" tags: [] comments: [] ---
Several people have written in the past about accessing Alternate Data Streams in the NTFS file system from .NET code. The reason for this is that accessing streams is not natively supported in .NET. What you don't hear very often is exactly why this is so.
There are two things you might want to do with data streams:
Neither of these operations are not supported by System.IO. To be perfectly honest, I think not supporting (2) is an understandable option; after all, it requires exposing some fairly specific windows APIs that are really only useful with NTFS itself (as far as I know). There's an old MSDN Magazine article on this topic by Stephen Toub, by the way.
But I was always pretty surprised that (1) wasn't natively supported. After all, you don't really need special APIs to do most of that stuff; it's built into the native Win32 APIs that .NET has to call anyway to perform the basic file and I/O operations. Heck, it is so basic that you can create a new alternate stream using "echo" in a cmd prompt!
You can find some mentions out there about this being because complexities introduced by file name aliases and what not, but I'm not sure I buy it. Seriously, number (1) should just work; if it doesn't then something smells wrong. After all, it takes work to get something that you get for free to not work at all!
I think the overall reason this doesn't happen is just that the way paths are handled (and path canonicalization is done) in regards to FileIOPermissions and such, but frankly, I'm not sure why it is such a big deal. There are certainly a bunch of other file system features in NT, which .NET doesn't supports either, but those are a bit more understandable (to me, at least).
Other than that, there's also the fact that System.IO.Path isn't particularly bright about how it handles paths. For example, it is hard to argue that this could be proper behavior:
Fortunately, using alternate data streams isn't so common, so this isn't such a big deal. Still, it is a curious bit.


{{ 'Join coordinated Kaart mapping projects by taking a task and mapping a part of the world in '}} OpenStreetMap{{', the free and editable map of the world. Communities, organizations and governments across the world have access to these maps to address local development challenges. You can join thousands of others to map '}}OpenStreetMap {{'and support these communities in need.' | translate }}
{{ 'Even if you are a new mapper, anyone who can use a computer to draw lines and boxes or use a phone to swipe and tap can meaningfully contribute to humanitarian mapping through' | translate }} OpenStreetMap. {{ 'If you have never mapped before and would like to help, visit our Learn page to get started mapping in less than 30 minutes.' | translate }}
{{ 'Seasoned mapper? Pick up one of our many projects and start contributing.' | translate }}


, you should study...
Ruby on Rails. The Language is ideal for building interactive web applications. It is popular with startups and smaller companies. Ruby is a object-oriented programming language and uses mainly the Rails framework that makes many common tasks easier.
front end interfaces, like HTML, Avanced CSS, JAVASCRIPT and DESIGN, because these languages and tools are executed directly in the browser and allows you to make an enjoyable experience for the user. It is widely used by startups and large companies
C# and the .NET. framework. The C# language is used by bigger, established Companies. The language also boasts a framework specially designed to work with it, to quickly create software and web apps. It's called the .NET framework from Microsoft.


home
Events
Categories
Software
Hardware
PenTest
pages
bi0s wiki
blog
Members
Achievements
CVEs' and Bug Bounties
Tools And Talks
Interships
Cyber awareness
Cyber Awareness
Posts
Cyber predatory
Fake news
Online bullying
Online Stalking
Social Media: Pros and Cons
Malware: Types and How do you recognise
Password creation, Password Manager and 2FA
Cyber Ethics
Dark Net and Child Sexual Abuse Materail
Career Opportunities in Cyber Security
Application Permissions on Android Devices


Section 6 Mismatch between conservation funding and drivers of biodiversity loss
We estimate if the amount of funding to drivers of biodiversity loss tracks the number of species threatened by each of these drivers
Importing data
# Read in data of post-processed grants
data <- read_csv("data\\post-processed-grants-for-analysis-unique.csv") %>%
as.data.frame() %>%
dplyr::select(c(1:42, 49:109, 43:48))
# Read in data on the number of species threatened (Maxwell et al. 2016)
maxwell.tab.s2 <- read_csv("data\\maxwell-tab-s2.csv") %>%
as.data.frame()
Preparing data to match our list of threat categories
maxwell.tab.s2$Abbreviated_clasfication
# Our threat categories: Climate Change, Habitat Loss, Overexploitation, Energy/Transportation, Pollution, Invasive Species
maxwell.data_threats <- maxwell.tab.s2 %>%
dplyr::select(3:4) %>%
group_by(Category) %>%
summarise(Total = sum(n_species_threatened)) %>%
mutate(Total.adjusted = replace(Total, Total %in% c(2132,11509), c(1910,7406)),
adj.prop = Total.adjusted/Total,
prop.total = Total.adjusted/sum(Total.adjusted))
# Habitat Loss and Energy/Transportation adjusted for compilation of multiple threats from Maxwell et al 2016. To do so, we looked at the number of species shared between threats and derived a conservative estimate accounting for overlap (species threatened by multiple threats).
Data on funding
# Single Category Data
money.data <- data[,11:(length(data)-6)]*data[,7]
money.data <- mutate(money.data,
Year.Authorized = data$Year.Authorized,
Num.Thr = data$Num.Thr,
Num.Tax = data$Num.Tax)
by.year.money <- money.data %>%
group_by(Year.Authorized) %>%
summarize_at(vars(cat_act_area:cat_taxa_other), sum, na.rm = TRUE) %>%
pivot_longer(-Year.Authorized, names_to = "Category", values_to = "Amount") %>%
group_by(Year.Authorized) %>%
mutate(ann.percent = Amount/sum(Amount)) %>%
group_by(Category) %>%
mutate(percent = Amount/sum(Amount))
Categories used for analysis
u.threat_cat <- names(data)[37:42]
u.threat_list <- c("Climate Change",
"Energy/Transportation",
"Habitat Loss",
"Invasive Species",
"Overexploitation",
"Pollution")
Fig. 1: Mismatch between conservation funding and threats to biodiversity
fig.1.money <- by.year.money %>%
filter(Category %in% u.threat_cat) %>%
group_by(Category) %>%
summarise(Total = sum(Amount))
rawFrame <- left_join(fig.1.money, maxwell.data_threats, by = "Category") %>%
mutate(rawMoney = Total.x) %>% # Total Money
mutate(rawSpecies = Total.adjusted) %>% # Total Species
dplyr::select(Category, rawMoney, rawSpecies)
fig.1.data <- left_join(fig.1.money, maxwell.data_threats, by = "Category") %>%
mutate(Money = (Total.x/sum(Total.x))*100) %>% # Percentage of Total Money
mutate(Species = (Total.adjusted/sum(Total.adjusted))*100) %>% # Percentage of Total Species
dplyr::select(Category, Money, Species) %>%
pivot_longer(c(Money, Species), names_to = "is.Money", values_to = "Amount") %>%
left_join(rawFrame, by = "Category") %>%
mutate(rawMoney = ifelse(is.Money == "Species", NA, rawMoney)) %>%
mutate(rawSpecies= ifelse(is.Money == "Money", NA, rawSpecies)) %>%
replace_na(list(rawMoney = "", rawSpecies = "")) %>%
unite("rawValue", rawMoney:rawSpecies, sep = "", remove = FALSE) %>%
mutate(rawValue = as.numeric(rawValue),
rawLabels = comma(rawValue))
fig.1.data$rawLabels[c(seq(1,11,2))] <- str_glue("${round(rawFrame$rawMoney/1e6)}mil")
fig.1.data$rawLabels[c(seq(2,12,2))] <- str_glue("n = {rawFrame$rawSpecies}")
fig_threats_funding<- ggplot(fig.1.data,
aes(fill=is.Money,
y=Amount,
x=Category)) +
geom_col(position="dodge") +
labs(x="Threat Category",
y="Proportion") +
scale_x_discrete(labels=u.threat_list) +
scale_fill_manual(values = c("slategray4", alpha("orangered4",.9)),
labels = c("Money (USD Million)", "Threatened Species")) +
geom_text(position = position_dodge(width= 1),
aes(label = rawLabels, hjust = 0.55, vjust = -0.5), angle=0, size=5) +
ylim(0, 40) +
theme_minimal() +
theme(axis.title = element_text(size = 16, face = "bold"),
axis.ticks.length.x = unit(.5, "cm"),
axis.text = element_text(size = 14),
legend.title = element_blank(),
legend.key.size = unit(1,"cm"),
legend.text = element_text(size = 12))
ggsave(fig_threats_funding, filename = "figs\\Figure1.png", width=17, height=7,
device = png(), units="in", dpi = 300)


PubNub’s Connected Car Solution Kit provides the key building blocks for developers to deliver scalable, real-time transportation management and dispatch applications, including taxi, fleet management and delivery service. Connected cars require real-time signaling for effective dispatch and tracking, yet vehicles constantly change networks and cell towers, lose connectivity, and drop dedicated network connections. Built on its global Real-Time Network and supported on over 50+ SDKs, PubNub’s Connected Car Solution Kit addresses the inherent challenges posed by connected vehicles, ensuring reliable, real-time message streaming and other key services like dispatch, location tracking, and geo-fencing.
Provides a real-time map view of all vehicle locations by streaming location data from each vehicle for delivery or arrival information. Stream real-time geo coordinates using PubNub’s Data Stream technology.
Read More About PubNub Data Stream Technology ›
// Publishing location data
pubnub.publish({
channel: 'vehicle_locations',
message: {
vehicle_id: 10,
lat: -37.123,
lng: 120.456
}
});
// Subscribing to location updates
pubnub.subscribe({
channel: 'vehicle_loations',
message: function (message) {
console.log("Vehicle: " + message.id);
console.log("Is located at: " + message.lat + ", " + message.lng);
}
});
Responds to transportation demand in real-time by managing messaging and response between vehicles and dispatch. Works seamlessly across all device platforms from desktop to mobile. All powered by PubNub Data Stream Technology.
// Listening for dispatch data
pubnub.subscribe({
channel: 'dispatch',
message: function (message) {
console.log("Picking up user at: " + message.lat + ", " + message.lng);
// Business logic...
}
});
// Requesting a ride
pubnub.publish({
channel: 'dispatch',
message: {
user_id: 42,
lat: -37.945,
lng: 122.574
}
});
Delivers a real-time view of fleet status and availability by detecting when vehicles are available or offline. Additionally, you can store custom data about each vehicle such as booked, fully-loaded and other status using PubNub Presence.
// Giving the vehicle status
pubnub.subscribe({
channel: 'vehicle_status',
message: function (message) {
// App code...
},
metadata: {
vehicle_id: 22,
state: {
type: "in_transit",
time: 12345
}
}
});
// Getting the current status of vehicles
pubnub.here_now({
channel: 'vehicle_status',
metadata: 1,
callback: function (data) {
console.log("There are " + data.uuids.length + " online.");
}
});
Trigger actions and alerts based on vehicle location and geographic boundaries (e.g. only vehicles within five miles get dispatch requests, customized traffic alerts, etc.). Create specific channels for individual geographic regions to target communications.
Read more about PubNub and Geolocation ›
// Publishing location data
navigator.geolocation.getCurrentPosition(function (position) {
pubnub.publish({
channel: "geofencing",
message: {
user_id: 42,
lat: position.coords.latitude,
lng: position.coords.longitude
}
});
});
// Geofencing user locations
pubnub.subscribe({
channel: "geofencing",
message: function (message) {
if (insideGeoNode(message.lat, message.lng)) {
console.log("The user " + message.user_id + " is in the node!");
}
}
});
Cell tower switching, 4G/3G/2G toggling, and tunnels ensure that persistent connections will drop frequently. PubNub provides server-side caching, auto-reconnection, and “catch-up” of missed messages upon reconnect to ensure that messages are delivered reliably in all network conditions.
The PubNub Network manages the real time connectivity for your application while working seamlessly with your application architecture. Whether you're using Backbone.JS, NodeJS or Scala - PubNub provides client SDKs for all major frameworks.
The first step to building your own transportation solution is signing up for PubNub. It is free and easy to get started and you can sign up using the link on the right.
Getting started is as easy as embedding the PubNub library into your HTML file. This is done via a script tag and also allows for selecting the specific version you would like to work with. Our library is served via a CDN for the guaranteed fastest performance download.
Publishing messages allows you to notify other users in real-time. This can be used for geolocation updates or even dispatch!
Subscribing to a channel allows you to listen for real-time updates from anyone who is publishing to the same channel!
Using PubNub Presence you will be able to detect when vehicles come online or go offline as well as store metadata about each of them.


Last Five Stocks Added
{% for stock in last_five_added_stocks %}
{{stock.id}}.
{{stock.stock_name}}
{{stock.initial_price}}
{{stock.maket_type}}
{{stock.available_no_units}}
{{stock.conversion_rate}}
Delete
{% endfor %}
Last Five NewsPosts Added
{% for newspost in last_five_added_newsposts %}
{{newspost.id}}.
{{newspost.headline}}
Delete
{% endfor %}