Docs: Added IDs to the highlighters for linking

[[search-request-highlighting]]
=== Highlighting

Allows to highlight search results on one or more fields. The
implementation uses either the lucene `highlighter`, `fast-vector-highlighter`
or `postings-highlighter`. The following is an example of the  search request
body:

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "fields" : {
            "content" : {}
        }
    }
}
--------------------------------------------------

In the above case, the `content` field will be highlighted for each
search hit (there will be another element in each search hit, called
`highlight`, which includes the highlighted fields and the highlighted
fragments).

In order to perform highlighting, the actual content of the field is
required. If the field in question is stored (has `store` set to `true`
in the mapping) it will be used, otherwise, the actual `_source` will
be loaded and the relevant field will be extracted from it.

The field name supports wildcard notation. For example, using `comment_*`
will cause all fields that match the expression to be highlighted.

[[postings-highlighter]]
==== Postings highlighter

If `index_options` is set to `offsets` in the mapping the postings highlighter
will be used instead of the plain highlighter. The postings highlighter:

* Is faster since it doesn't require to reanalyze the text to be highlighted:
the larger the documents the better the performance gain should be
* Requires less disk space than term_vectors, needed for the fast vector
highlighter
* Breaks the text into sentences and highlights them. Plays really well with
natural languages, not as well with fields containing for instance html markup
* Treats the document as the whole corpus, and scores individual sentences as
if they were documents in this corpus, using the  BM25 algorithm

Here is an example of setting the `content` field to allow for
highlighting using the postings highlighter on it:

[source,js]
--------------------------------------------------
{
    "type_name" : {
        "content" : {"index_options" : "offsets"}
    }
}
--------------------------------------------------

[NOTE]
Note that the postings highlighter is meant to perform simple query terms
highlighting, regardless of their positions. That means that when used for
instance in combination with a phrase query, it will highlight all the terms
that the query is composed of, regardless of whether they are actually part of
a query match, effectively ignoring their positions.

[WARNING]
The postings highlighter does support highlighting of multi term queries, like
prefix queries, wildcard queries and so on. On the other hand, this requires
the queries to be rewritten using a proper
<<query-dsl-multi-term-rewrite,rewrite method>> that supports multi term
extraction, which is a potentially expensive operation.

[[fast-vector-highlighter]]
==== Fast vector highlighter

If `term_vector` information is provided by setting `term_vector` to
`with_positions_offsets` in the mapping then the fast vector highlighter
will be used instead of the plain highlighter.  The fast vector highlighter:

* Is faster especially for large fields (> `1MB`)
* Can be customized with `boundary_chars`, `boundary_max_scan`, and
 `fragment_offset` (see <<boundary-characters,below>>)
* Requires setting `term_vector` to `with_positions_offsets` which
  increases the size of the index
* Can combine matches from multiple fields into one result.  See
  `matched_fields`
* Can assign different weights to matches at different positions allowing
  for things like phrase matches being sorted above term matches when
  highlighting a Boosting Query that boosts phrase matches over term matches

Here is an example of setting the `content` field to allow for
highlighting using the fast vector highlighter on it (this will cause
the index to be bigger):

[source,js]
--------------------------------------------------
{
    "type_name" : {
        "content" : {"term_vector" : "with_positions_offsets"}
    }
}
--------------------------------------------------

==== Force highlighter type

The `type` field allows to force a specific highlighter type. This is useful
for instance when needing to use the plain highlighter on a field that has
`term_vectors` enabled. The allowed values are: `plain`, `postings` and `fvh`.
The following is an example that forces the use of the plain highlighter:

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "fields" : {
            "content" : {"type" : "plain"}
        }
    }
}
--------------------------------------------------

==== Force highlighting on source

added[1.0.0.RC1]

Forces the highlighting to highlight fields based on the source even if fields are
stored separately. Defaults to `false`.

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "fields" : {
            "content" : {"force_source" : true}
        }
    }
}
--------------------------------------------------

[[tags]]
==== Highlighting Tags

By default, the highlighting will wrap highlighted text in `<em>` and
`</em>`. This can be controlled by setting `pre_tags` and `post_tags`,
for example:

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "pre_tags" : ["<tag1>"],
        "post_tags" : ["</tag1>"],
        "fields" : {
            "_all" : {}
        }
    }
}
--------------------------------------------------

Using the fast vector highlighter there can be more tags, and the "importance"
is ordered.

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "pre_tags" : ["<tag1>", "<tag2>"],
        "post_tags" : ["</tag1>", "</tag2>"],
        "fields" : {
            "_all" : {}
        }
    }
}
--------------------------------------------------

There are also built in "tag" schemas, with currently a single schema
called `styled` with the following `pre_tags`:

[source,js]
--------------------------------------------------
, , ,
, , ,
, , ,

--------------------------------------------------

and `</em>` as `post_tags`. If you think of more nice to have built in tag
schemas, just send an email to the mailing list or open an issue. Here
is an example of switching tag schemas:

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "tags_schema" : "styled",
        "fields" : {
            "content" : {}
        }
    }
}
--------------------------------------------------

==== Encoder

An `encoder` parameter can be used to define how highlighted text will
be encoded. It can be either `default` (no encoding) or `html` (will
escape html, if you use html highlighting tags).

==== Highlighted Fragments

Each field highlighted can control the size of the highlighted fragment
in characters (defaults to `100`), and the maximum number of fragments
to return (defaults to `5`).
For example:

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "fields" : {
            "content" : {"fragment_size" : 150, "number_of_fragments" : 3}
        }
    }
}
--------------------------------------------------

The `fragment_size` is ignored when using the postings highlighter, as it
outputs sentences regardless of their length.

On top of this it is possible to specify that highlighted fragments need
to be sorted by score:

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "order" : "score",
        "fields" : {
            "content" : {"fragment_size" : 150, "number_of_fragments" : 3}
        }
    }
}
--------------------------------------------------

If the `number_of_fragments` value is set to `0` then no fragments are
produced, instead the whole content of the field is returned, and of
course it is highlighted. This can be very handy if short texts (like
document title or address) need to be highlighted but no fragmentation
is required. Note that `fragment_size` is ignored in this case.

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "fields" : {
            "_all" : {},
            "bio.title" : {"number_of_fragments" : 0}
        }
    }
}
--------------------------------------------------

When using `fast-vector-highlighter` one can use `fragment_offset`
parameter to control the margin to start highlighting from.

In the case where there is no matching fragment to highlight, the default is
to not return anything. Instead, we can return a snippet of text from the
beginning of the field by setting `no_match_size` (default `0`) to the length
of the text that you want returned. The actual length may be shorter than
specified as it tries to break on a word boundary. When using the postings
highlighter it is not possible to control the actual size of the snippet,
therefore the first sentence gets returned whenever `no_match_size` is
greater than `0`.

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "fields" : {
            "content" : {
                "fragment_size" : 150,
                "number_of_fragments" : 3,
                "no_match_size": 150
            }
        }
    }
}
--------------------------------------------------

==== Highlight query

It is also possible to highlight against a query other than the search
query by setting `highlight_query`.  This is especially useful if you
use a rescore query because those are not taken into account by
highlighting by default.  Elasticsearch does not validate that
`highlight_query` contains the search query in any way so it is possible
to define it so legitimate query results aren't highlighted at all.
Generally it is better to include the search query in the
`highlight_query`.  Here is an example of including both the search
query and the rescore query in `highlight_query`.
[source,js]
--------------------------------------------------
{
    "fields": [ "_id" ],
    "query" : {
        "match": {
            "content": {
                "query": "foo bar"
            }
        }
    },
    "rescore": {
        "window_size": 50,
        "query": {
            "rescore_query" : {
                "match_phrase": {
                    "content": {
                        "query": "foo bar",
                        "phrase_slop": 1
                    }
                }
            },
            "rescore_query_weight" : 10
        }
    },
    "highlight" : {
        "order" : "score",
        "fields" : {
            "content" : {
                "fragment_size" : 150,
                "number_of_fragments" : 3,
                "highlight_query": {
                    "bool": {
                        "must": {
                            "match": {
                                "content": {
                                    "query": "foo bar"
                                }
                            }
                        },
                        "should": {
                            "match_phrase": {
                                "content": {
                                    "query": "foo bar",
                                    "phrase_slop": 1,
                                    "boost": 10.0
                                }
                            }
                        },
                        "minimum_should_match": 0
                    }
                }
            }
        }
    }
}
--------------------------------------------------

Note that the score of text fragment in this case is calculated by the Lucene
highlighting framework. For implementation details you can check the
`ScoreOrderFragmentsBuilder.java` class. On the other hand when using the
postings highlighter the fragments are scored using, as mentioned above,
the BM25 algorithm.

[[highlighting-settings]]
==== Global Settings

Highlighting settings can be set on a global level and then overridden
at the field level.

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "number_of_fragments" : 3,
        "fragment_size" : 150,
        "tag_schema" : "styled",
        "fields" : {
            "_all" : { "pre_tags" : [""], "post_tags" : [""] },
            "bio.title" : { "number_of_fragments" : 0 },
            "bio.author" : { "number_of_fragments" : 0 },
            "bio.content" : { "number_of_fragments" : 5, "order" : "score" }
        }
    }
}
--------------------------------------------------

[[field-match]]
==== Require Field Match

`require_field_match` can be set to `true` which will cause a field to
be highlighted only if a query matched that field. `false` means that
terms are highlighted on all requested fields regardless if the query
matches specifically on them.

[[boundary-characters]]
==== Boundary Characters

When highlighting a field using the fast vector highlighter,
`boundary_chars` can be configured to define what constitutes a boundary
for highlighting. It's a single string with each boundary character
defined in it. It defaults to `.,!? \t\n`.

The `boundary_max_scan` allows to control how far to look for boundary
characters, and defaults to `20`.

[[matched-fields]]
==== Matched Fields
The Fast Vector Highlighter can combine matches on multiple fields to
highlight a single field using `matched_fields`.  This is most
intuitive for multifields that analyze the same string in different
ways.  All `matched_fields` must have `term_vector` set to
`with_positions_offsets` but only the field to which the matches are
combined is loaded so only that field would benefit from having
`store` set to `yes`.

In the following examples `content` is analyzed by the `english`
analyzer and `content.plain` is analyzed by the `standard` analyzer.

[source,js]
--------------------------------------------------
{
    "query": {
        "query_string": {
            "query": "content.plain:running scissors",
            "fields": ["content"]
        }
    },
    "highlight": {
        "order": "score",
        "fields": {
            "content": {
                "matched_fields": ["content", "content.plain"],
                "type" : "fvh"
            }
        }
    }
}
--------------------------------------------------
The above matches both "run with scissors" and "running with scissors"
and would highlight "running" and "scissors" but not "run". If both
phrases appear in a large document then "running with scissors" is
sorted above "run with scissors" in the fragments list because there
are more matches in that fragment.

[source,js]
--------------------------------------------------
{
    "query": {
        "query_string": {
            "query": "running scissors",
            "fields": ["content", "content.plain^10"]
        }
    },
    "highlight": {
        "order": "score",
        "fields": {
            "content": {
                "matched_fields": ["content", "content.plain"],
                "type" : "fvh"
            }
        }
    }
}
--------------------------------------------------
The above highlights "run" as well as "running" and "scissors" but
still sorts "running with scissors" above "run with scissors" because
the plain match ("running") is boosted.

[source,js]
--------------------------------------------------
{
    "query": {
        "query_string": {
            "query": "running scissors",
            "fields": ["content", "content.plain^10"]
        }
    },
    "highlight": {
        "order": "score",
        "fields": {
            "content": {
                "matched_fields": ["content.plain"],
                "type" : "fvh"
            }
        }
    }
}
--------------------------------------------------
The above query wouldn't highlight "run" or "scissor" but shows that
it is just fine not to list the field to which the matches are combined
(`content`) in the matched fields.

[NOTE]
Technically it is also fine to add fields to `matched_fields` that
don't share the same underlying string as the field to which the matches
are combined.  The results might not make much sense and if one of the
matches is off the end of the text then the whole query will fail.

[NOTE]
===================================================================
There is a small amount of overhead involved with setting
`matched_fields` to a non-empty array so always prefer
[source,js]
--------------------------------------------------
    "highlight": {
        "fields": {
            "content": {}
        }
    }
--------------------------------------------------
to
[source,js]
--------------------------------------------------
    "highlight": {
        "fields": {
            "content": {
                "matched_fields": ["content"],
                "type" : "fvh"
            }
        }
    }
--------------------------------------------------
===================================================================

[[phrase-limit]]
==== Phrase Limit
The `fast-vector-highlighter` has a `phrase_limit` parameter that prevents
it from analyzing too many phrases and eating tons of memory.  It defaults
to 256 so only the first 256 matching phrases in the document scored
considered.  You can raise the limit with the `phrase_limit` parameter but
keep in mind that scoring more phrases consumes more time and memory.

If using `matched_fields` keep in mind that `phrase_limit` phrases per
matched field are considered.

[[explicit-field-order]]
=== Field Highlight Order
Elasticsearch highlights the fields in the order that they are sent.  Per the
json spec objects are unordered but if you need to be explicit about the order
that fields are highlighted then you can use an array for `fields` like this:
[source,js]
--------------------------------------------------
    "highlight": {
        "fields": [
            {"title":{ /*params*/ }},
            {"text":{ /*params*/ }}
        ]
    }
--------------------------------------------------
None of the highlighters built into Elasticsearch care about the order that the
fields are highlighted but a plugin may.

[[search-request-highlighting]]
=== Highlighting

Allows to highlight search results on one or more fields. The
implementation uses either the lucene `highlighter`, `fast-vector-highlighter`
or `postings-highlighter`. The following is an example of the  search request
body:

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "fields" : {
            "content" : {}
        }
    }
}
--------------------------------------------------

In the above case, the `content` field will be highlighted for each
search hit (there will be another element in each search hit, called
`highlight`, which includes the highlighted fields and the highlighted
fragments).

In order to perform highlighting, the actual content of the field is
required. If the field in question is stored (has `store` set to `true`
in the mapping) it will be used, otherwise, the actual `_source` will
be loaded and the relevant field will be extracted from it.

The field name supports wildcard notation. For example, using `comment_*`
will cause all fields that match the expression to be highlighted.

==== Postings highlighter

If `index_options` is set to `offsets` in the mapping the postings highlighter
will be used instead of the plain highlighter. The postings highlighter:

* Is faster since it doesn't require to reanalyze the text to be highlighted:
the larger the documents the better the performance gain should be
* Requires less disk space than term_vectors, needed for the fast vector
highlighter
* Breaks the text into sentences and highlights them. Plays really well with
natural languages, not as well with fields containing for instance html markup
* Treats the document as the whole corpus, and scores individual sentences as
if they were documents in this corpus, using the  BM25 algorithm

Here is an example of setting the `content` field to allow for
highlighting using the postings highlighter on it:

[source,js]
--------------------------------------------------
{
    "type_name" : {
        "content" : {"index_options" : "offsets"}
    }
}
--------------------------------------------------

[NOTE]
Note that the postings highlighter is meant to perform simple query terms
highlighting, regardless of their positions. That means that when used for
instance in combination with a phrase query, it will highlight all the terms
that the query is composed of, regardless of whether they are actually part of
a query match, effectively ignoring their positions.

[WARNING]
The postings highlighter does support highlighting of multi term queries, like
prefix queries, wildcard queries and so on. On the other hand, this requires
the queries to be rewritten using a proper
<<query-dsl-multi-term-rewrite,rewrite method>> that supports multi term
extraction, which is a potentially expensive operation.

==== Fast vector highlighter

If `term_vector` information is provided by setting `term_vector` to
`with_positions_offsets` in the mapping then the fast vector highlighter
will be used instead of the plain highlighter.  The fast vector highlighter:

* Is faster especially for large fields (> `1MB`)
* Can be customized with `boundary_chars`, `boundary_max_scan`, and
 `fragment_offset` (see <<boundary-characters,below>>)
* Requires setting `term_vector` to `with_positions_offsets` which
  increases the size of the index
* Can combine matches from multiple fields into one result.  See
  `matched_fields`
* Can assign different weights to matches at different positions allowing
  for things like phrase matches being sorted above term matches when
  highlighting a Boosting Query that boosts phrase matches over term matches

Here is an example of setting the `content` field to allow for
highlighting using the fast vector highlighter on it (this will cause
the index to be bigger):

[source,js]
--------------------------------------------------
{
    "type_name" : {
        "content" : {"term_vector" : "with_positions_offsets"}
    }
}
--------------------------------------------------

==== Force highlighter type

The `type` field allows to force a specific highlighter type. This is useful
for instance when needing to use the plain highlighter on a field that has
`term_vectors` enabled. The allowed values are: `plain`, `postings` and `fvh`.
The following is an example that forces the use of the plain highlighter:

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "fields" : {
            "content" : {"type" : "plain"}
        }
    }
}
--------------------------------------------------

==== Force highlighting on source

added[1.0.0.RC1]

Forces the highlighting to highlight fields based on the source even if fields are
stored separately. Defaults to `false`.

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "fields" : {
            "content" : {"force_source" : true}
        }
    }
}
--------------------------------------------------

[[tags]]
==== Highlighting Tags

By default, the highlighting will wrap highlighted text in `<em>` and
`</em>`. This can be controlled by setting `pre_tags` and `post_tags`,
for example:

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "pre_tags" : ["<tag1>"],
        "post_tags" : ["</tag1>"],
        "fields" : {
            "_all" : {}
        }
    }
}
--------------------------------------------------

Using the fast vector highlighter there can be more tags, and the "importance"
is ordered.

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "pre_tags" : ["<tag1>", "<tag2>"],
        "post_tags" : ["</tag1>", "</tag2>"],
        "fields" : {
            "_all" : {}
        }
    }
}
--------------------------------------------------

There are also built in "tag" schemas, with currently a single schema
called `styled` with the following `pre_tags`:

[source,js]
--------------------------------------------------
, , ,
, , ,
, , ,

--------------------------------------------------

and `</em>` as `post_tags`. If you think of more nice to have built in tag
schemas, just send an email to the mailing list or open an issue. Here
is an example of switching tag schemas:

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "tags_schema" : "styled",
        "fields" : {
            "content" : {}
        }
    }
}
--------------------------------------------------

==== Encoder

An `encoder` parameter can be used to define how highlighted text will
be encoded. It can be either `default` (no encoding) or `html` (will
escape html, if you use html highlighting tags).

==== Highlighted Fragments

Each field highlighted can control the size of the highlighted fragment
in characters (defaults to `100`), and the maximum number of fragments
to return (defaults to `5`).
For example:

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "fields" : {
            "content" : {"fragment_size" : 150, "number_of_fragments" : 3}
        }
    }
}
--------------------------------------------------

The `fragment_size` is ignored when using the postings highlighter, as it
outputs sentences regardless of their length.

On top of this it is possible to specify that highlighted fragments need
to be sorted by score:

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "order" : "score",
        "fields" : {
            "content" : {"fragment_size" : 150, "number_of_fragments" : 3}
        }
    }
}
--------------------------------------------------

If the `number_of_fragments` value is set to `0` then no fragments are
produced, instead the whole content of the field is returned, and of
course it is highlighted. This can be very handy if short texts (like
document title or address) need to be highlighted but no fragmentation
is required. Note that `fragment_size` is ignored in this case.

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "fields" : {
            "_all" : {},
            "bio.title" : {"number_of_fragments" : 0}
        }
    }
}
--------------------------------------------------

When using `fast-vector-highlighter` one can use `fragment_offset`
parameter to control the margin to start highlighting from.

In the case where there is no matching fragment to highlight, the default is
to not return anything. Instead, we can return a snippet of text from the
beginning of the field by setting `no_match_size` (default `0`) to the length
of the text that you want returned. The actual length may be shorter than
specified as it tries to break on a word boundary. When using the postings
highlighter it is not possible to control the actual size of the snippet,
therefore the first sentence gets returned whenever `no_match_size` is
greater than `0`.

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "fields" : {
            "content" : {
                "fragment_size" : 150,
                "number_of_fragments" : 3,
                "no_match_size": 150
            }
        }
    }
}
--------------------------------------------------

==== Highlight query

It is also possible to highlight against a query other than the search
query by setting `highlight_query`.  This is especially useful if you
use a rescore query because those are not taken into account by
highlighting by default.  Elasticsearch does not validate that
`highlight_query` contains the search query in any way so it is possible
to define it so legitimate query results aren't highlighted at all.
Generally it is better to include the search query in the
`highlight_query`.  Here is an example of including both the search
query and the rescore query in `highlight_query`.
[source,js]
--------------------------------------------------
{
    "fields": [ "_id" ],
    "query" : {
        "match": {
            "content": {
                "query": "foo bar"
            }
        }
    },
    "rescore": {
        "window_size": 50,
        "query": {
            "rescore_query" : {
                "match_phrase": {
                    "content": {
                        "query": "foo bar",
                        "phrase_slop": 1
                    }
                }
            },
            "rescore_query_weight" : 10
        }
    },
    "highlight" : {
        "order" : "score",
        "fields" : {
            "content" : {
                "fragment_size" : 150,
                "number_of_fragments" : 3,
                "highlight_query": {
                    "bool": {
                        "must": {
                            "match": {
                                "content": {
                                    "query": "foo bar"
                                }
                            }
                        },
                        "should": {
                            "match_phrase": {
                                "content": {
                                    "query": "foo bar",
                                    "phrase_slop": 1,
                                    "boost": 10.0
                                }
                            }
                        },
                        "minimum_should_match": 0
                    }
                }
            }
        }
    }
}
--------------------------------------------------

Note that the score of text fragment in this case is calculated by the Lucene
highlighting framework. For implementation details you can check the
`ScoreOrderFragmentsBuilder.java` class. On the other hand when using the
postings highlighter the fragments are scored using, as mentioned above,
the BM25 algorithm.

[[highlighting-settings]]
==== Global Settings

Highlighting settings can be set on a global level and then overridden
at the field level.

[source,js]
--------------------------------------------------
{
    "query" : {...},
    "highlight" : {
        "number_of_fragments" : 3,
        "fragment_size" : 150,
        "tag_schema" : "styled",
        "fields" : {
            "_all" : { "pre_tags" : [""], "post_tags" : [""] },
            "bio.title" : { "number_of_fragments" : 0 },
            "bio.author" : { "number_of_fragments" : 0 },
            "bio.content" : { "number_of_fragments" : 5, "order" : "score" }
        }
    }
}
--------------------------------------------------

[[field-match]]
==== Require Field Match

`require_field_match` can be set to `true` which will cause a field to
be highlighted only if a query matched that field. `false` means that
terms are highlighted on all requested fields regardless if the query
matches specifically on them.

[[boundary-characters]]
==== Boundary Characters

When highlighting a field using the fast vector highlighter,
`boundary_chars` can be configured to define what constitutes a boundary
for highlighting. It's a single string with each boundary character
defined in it. It defaults to `.,!? \t\n`.

The `boundary_max_scan` allows to control how far to look for boundary
characters, and defaults to `20`.

[[matched-fields]]
==== Matched Fields
The Fast Vector Highlighter can combine matches on multiple fields to
highlight a single field using `matched_fields`.  This is most
intuitive for multifields that analyze the same string in different
ways.  All `matched_fields` must have `term_vector` set to
`with_positions_offsets` but only the field to which the matches are
combined is loaded so only that field would benefit from having
`store` set to `yes`.

In the following examples `content` is analyzed by the `english`
analyzer and `content.plain` is analyzed by the `standard` analyzer.

[source,js]
--------------------------------------------------
{
    "query": {
        "query_string": {
            "query": "content.plain:running scissors",
            "fields": ["content"]
        }
    },
    "highlight": {
        "order": "score",
        "fields": {
            "content": {
                "matched_fields": ["content", "content.plain"],
                "type" : "fvh"
            }
        }
    }
}
--------------------------------------------------
The above matches both "run with scissors" and "running with scissors"
and would highlight "running" and "scissors" but not "run". If both
phrases appear in a large document then "running with scissors" is
sorted above "run with scissors" in the fragments list because there
are more matches in that fragment.

[source,js]
--------------------------------------------------
{
    "query": {
        "query_string": {
            "query": "running scissors",
            "fields": ["content", "content.plain^10"]
        }
    },
    "highlight": {
        "order": "score",
        "fields": {
            "content": {
                "matched_fields": ["content", "content.plain"],
                "type" : "fvh"
            }
        }
    }
}
--------------------------------------------------
The above highlights "run" as well as "running" and "scissors" but
still sorts "running with scissors" above "run with scissors" because
the plain match ("running") is boosted.

[source,js]
--------------------------------------------------
{
    "query": {
        "query_string": {
            "query": "running scissors",
            "fields": ["content", "content.plain^10"]
        }
    },
    "highlight": {
        "order": "score",
        "fields": {
            "content": {
                "matched_fields": ["content.plain"],
                "type" : "fvh"
            }
        }
    }
}
--------------------------------------------------
The above query wouldn't highlight "run" or "scissor" but shows that
it is just fine not to list the field to which the matches are combined
(`content`) in the matched fields.

[NOTE]
Technically it is also fine to add fields to `matched_fields` that
don't share the same underlying string as the field to which the matches
are combined.  The results might not make much sense and if one of the
matches is off the end of the text then the whole query will fail.

[NOTE]
===================================================================
There is a small amount of overhead involved with setting
`matched_fields` to a non-empty array so always prefer
[source,js]
--------------------------------------------------
    "highlight": {
        "fields": {
            "content": {}
        }
    }
--------------------------------------------------
to
[source,js]
--------------------------------------------------
    "highlight": {
        "fields": {
            "content": {
                "matched_fields": ["content"],
                "type" : "fvh"
            }
        }
    }
--------------------------------------------------
===================================================================

[[phrase-limit]]
==== Phrase Limit
The `fast-vector-highlighter` has a `phrase_limit` parameter that prevents
it from analyzing too many phrases and eating tons of memory.  It defaults
to 256 so only the first 256 matching phrases in the document scored
considered.  You can raise the limit with the `phrase_limit` parameter but
keep in mind that scoring more phrases consumes more time and memory.

If using `matched_fields` keep in mind that `phrase_limit` phrases per
matched field are considered.

[[explicit-field-order]]
=== Field Highlight Order
Elasticsearch highlights the fields in the order that they are sent.  Per the
json spec objects are unordered but if you need to be explicit about the order
that fields are highlighted then you can use an array for `fields` like this:
[source,js]
--------------------------------------------------
    "highlight": {
        "fields": [
            {"title":{ /*params*/ }},
            {"text":{ /*params*/ }}
        ]
    }
--------------------------------------------------
None of the highlighters built into Elasticsearch care about the order that the
fields are highlighted but a plugin may.


Show KStreams binder as a top-level section in the docs (#44)

include::{stream-docs-basedir}/core/spring-cloud-stream-core-docs/src/main/asciidoc/spring-cloud-stream-overview.adoc[]

= Binder Implementations

== Apache Kafka Binder

include::{stream-docs-basedir}/kafka/spring-cloud-stream-binder-kafka-docs/src/main/asciidoc/spring-cloud-stream-binder-kafka-aggregate.adoc[leveloffset=+1]

== Apache Kafka Streams Binder

include::{stream-docs-basedir}/kafka/spring-cloud-stream-binder-kafka-docs/src/main/asciidoc/kafka-streams.adoc[leveloffset=+1]

== RabbitMQ Binder

include::{stream-docs-basedir}/rabbit/spring-cloud-stream-binder-rabbit-docs/src/main/asciidoc/spring-cloud-stream-binder-rabbit-aggregate.adoc[leveloffset=+1]

include::{stream-docs-basedir}/core/spring-cloud-stream-core-docs/src/main/asciidoc/spring-cloud-stream-overview.adoc[]

= Binder Implementations

== Apache Kafka Binder

include::{stream-docs-basedir}/kafka/spring-cloud-stream-binder-kafka-docs/src/main/asciidoc/spring-cloud-stream-binder-kafka-aggregate.adoc[leveloffset=+1]

== RabbitMQ Binder

include::{stream-docs-basedir}/rabbit/spring-cloud-stream-binder-rabbit-docs/src/main/asciidoc/spring-cloud-stream-binder-rabbit-aggregate.adoc[leveloffset=+1]


Fixing typo

Fixing typo

//
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements.  See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership.  The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License.  You may obtain a copy of the License at
//
//   http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied.  See the License for the
// specific language governing permissions and limitations
// under the License.
//
=== Customization

[CAUTION]
Only Maven projects can be customized: if using Standalone, Debian packages or the GUI installer, none of the
customizations discussed below can be applied.

Apache Syncope is designed to be as flexible as possible, to best suit the various environments
in which it can be deployed. Besides other aspects, this means that every feature and component can be extended or
replaced.

Once the project has been created from the provided Maven archetype, the generated source tree is available for either
adding new features or replacing existing components.

[[override-behavior]]
[TIP]
.Override behavior
====
As a rule of thumb, any file of the local project will take precedence over a file with the same name in the same
directory of the standard Apache Syncope release.

For example, if you place

 core/spring/src/main/java/org/apache/syncope/core/spring/security/SyncopeAuthenticationProvider.java

in the local project, this file will be picked up instead of
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/spring/src/main/java/org/apache/syncope/core/spring/security/SyncopeAuthenticationProvider.java[SyncopeAuthenticationProvider^].
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/master/core/spring/src/main/java/org/apache/syncope/core/spring/security/SyncopeAuthenticationProvider.java[SyncopeAuthenticationProvider^].
endif::[]

The same happens with resources as images or HTML files; if you place

 console/src/main/resources/org/apache/syncope/client/console/pages/BasePage.html

in the local project, this file will be picked up instead of
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/client/console/src/main/resources/org/apache/syncope/client/console/pages/BasePage.html[BasePage.html^].
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/master/client/console/src/main/resources/org/apache/syncope/client/console/pages/BasePage.html[BasePage.html^].
endif::[]

This general behavior might have exceptions, as highlighted below.
====

In general, the Embedded Mode (see the
ifeval::["{backend}" == "html5"]
http://syncope.apache.org/docs/getting-started.html[Apache Syncope Getting Started Guide]
endif::[]
ifeval::["{backend}" == "pdf"]
http://syncope.apache.org/docs/getting-started.pdf[Apache Syncope Getting Started Guide]
endif::[]
for details) allows the user to work comfortably from a single workstation, with no need of additional setup; it is
effectively implemented as the `all`
https://maven.apache.org/guides/introduction/introduction-to-profiles.html[Maven profile^], where the available optional
components and extensions are enabled. +
When deploying the generated WAR artifacts into an external <<javaee-container>> however, the required components and
extensions need to be explicitly selected and enabled, as shown in the following text.

[[deployment-directories]]
.Deployment directories
****
Apache Syncope needs three base directories to be defined:

* bundles - where the <<external-resources,connector bundles>> are stored;
* log - where all the system logs are written;
* conf (optional) - where configuration files are located, if overriding the default values is needed.

[WARNING]
The `bundles` directory should only contain connector bundle JAR files. +
The presence of any other file might cause the unavailability of any connector bundle in Apache Syncope.

For reference, the suggested directory layout can be created as follows:

....
$ mkdir /opt/syncope
$ mkdir /opt/syncope/bundles
$ mkdir /opt/syncope/log
$ mkdir /opt/syncope/conf
....
****

The WAR artifacts are generated by running the Maven command (with reference to the suggested directory layout):

....
mvn clean verify \
  -Dconf.directory=/opt/syncope/conf \
  -Dbundles.directory=/opt/syncope/bundles \
  -Dlog.directory=/opt/syncope/log
....

After downloading all of the dependencies that are needed, three WAR files will be produced:

. `core/target/syncope.war`
. `console/target/syncope-console.war`
. `enduser/target/syncope-enduser.war`

If no failures are encountered, your basic Apache Syncope project is now ready to be deployed.

[[embedded-debug]]
[TIP]
.JPDA Debug in Embedded Mode
====
The Java™ Platform Debugger Architecture (http://docs.oracle.com/javase/8/docs/technotes/guides/jpda/index.html[JPDA^])
is a collection of APIs aimed to help with debugging Java code.

Enhancing the `embedded` profile of the `enduser` module to enable the JPDA socket is quite
straightforward: just add the `<profile>` below to `enduser/pom.xml`:

[source,xml,subs="verbatim,attributes"]
----
<profile>
  <id>debug</id>

  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.cargo</groupId>
        <artifactId>cargo-maven2-plugin</artifactId>
        <inherited>true</inherited>
        <configuration>
          <configuration>
            <properties>
              <cargo.jvmargs>
              -Xdebug
              -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n
              -noverify -XX:+CMSClassUnloadingEnabled
              -XX:+UseConcMarkSweepGC -Xmx1024m -Xms512m
              </cargo.jvmargs>
            </properties>
          </configuration>
        </configuration>
      </plugin>
    </plugins>
  </build>
</profile>
----

Now, from the `enduser` subdirectory, execute:

[source,bash]
mvn -P embedded,debug

At this point your favorite IDE can be attached to the port `8000`; please note that you might need to add
`-XX:MaxPermSize=512m` to `<cargo.jvmargs>` in order to run with JDK 7.
====

[[customization-core]]
==== Core

[CAUTION]
When providing custom Java classes implementing the defined interfaces or extending the existing
implementations, their package *must* be rooted under `org.apache.syncope.core`, otherwise they will not be available
at runtime.

Besides replacing existing classes as explained <<override-behavior,above>>, new implementations can be provided under
`core/src/main/java` for the following components:

* <<propagationactions,propagation>>, <<pushactions,push>>, <<pullactions,pull>> and <<logicactions,logic>> actions
* <<push-correlation-rules,push>> / <<pull-correlation-rules,pull>> correlation rules
* <<pull-mode,reconciliation filter builders>>
* <<tasks-custom,custom tasks>>
* <<reportlets,reportlets>>
* <<account-rules,account>> and <<password-rules,password>> rules for policies
* <<plain,plain schema validators>>
* <<mapping,mapping item transformers>>
* <<virtual-attribute-cache,virtual attribute cache>>
* <<workflow-adapters,workflow adapters>>
* <<provisioning-managers,provisioning managers>>
* <<notifications,notification recipient providers>>

[[new-rest-endpoints]]
[TIP]
.New REST endpoints
====
Adding a new REST endpoint involves several operations:

. create - in an extension's `rest-api` module or under `common` otherwise - a Java interface with package
`org.apache.syncope.common.rest.api.service` and proper JAX-RS annotations; check
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/ext/camel/rest-api/src/main/java/org/apache/syncope/common/rest/api/service/CamelRouteService.java[CamelRouteService^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/master/ext/camel/rest-api/src/main/java/org/apache/syncope/common/rest/api/service/CamelRouteService.java[CamelRouteService^]
endif::[]
for reference;
. if needed, define supporting payload objects - in an extension's `common-lib` module or under `common` otherwise;
check
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/ext/camel/common-lib/src/main/java/org/apache/syncope/common/lib/to/CamelRouteTO.java[CamelRouteTO^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/master/ext/camel/common-lib/src/main/java/org/apache/syncope/common/lib/to/CamelRouteTO.java[CamelRouteTO^]
endif::[]
for reference;
. implement - in an extension's `rest-cxf` module or under `core` otherwise -  the interface defined above in a Java
class with package `org.apache.syncope.core.rest.cxf.service`; check
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/ext/camel/rest-cxf/src/main/java/org/apache/syncope/core/rest/cxf/service/CamelRouteServiceImpl.java[CamelRouteServiceImpl^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/master/ext/camel/rest-cxf/src/main/java/org/apache/syncope/core/rest/cxf/service/CamelRouteServiceImpl.java[CamelRouteServiceImpl^]
endif::[]
for reference.

By following such conventions, the new REST endpoint will be automatically picked up alongside the default services.
====

[WARNING]
====
The <<override-behavior,override behavior>> might have exceptions; if you need to customize one of the
Spring context definitions. For example, if you want to customize
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/spring/src/main/resources/securityContext.xml[securityContext.xml^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/master/core/spring/src/main/resources/securityContext.xml[securityContext.xml^]
endif::[]
, you will also need to replace the following text in `core/src/main/webapp/WEB-INF/web.xml`,

....
classpath*:/*Context.xml
....

with

....
classpath:/coreContext.xml
classpath:/securityContext.xml
classpath*:/logicContext.xml
classpath*:/restCXFContext.xml
classpath*:/persistenceContext.xml
classpath*:/provisioningContext.xml
classpath*:/workflowContext.xml
....

to be sure that `core/src/main/resources/securityContext.xml` is picked up. +
Please also note that the actual list of Spring context files to include might depend on the configured extensions.
====

[discrete]
===== Select the <<activiti-user-workflow-adapter>>

Add the following dependency to `core/pom.xml`:

[source,xml,subs="verbatim,attributes"]
----
<dependency>
  <groupId>org.apache.syncope.core</groupId>
  <artifactId>syncope-core-workflow-activiti</artifactId>
  <version>${syncope.version}</version>
</dependency>
----

Copy `core/src/main/resources/all/workflow.properties` to `core/src/main/resources/workflow.properties`.

[discrete]
===== Enable the <<apache-camel-provisioning-manager>>

Add the following dependencies to `core/pom.xml`:

[source,xml,subs="verbatim,attributes"]
----
<dependency>
  <groupId>org.apache.syncope.ext.camel</groupId>
  <artifactId>syncope-ext-camel-rest-cxf</artifactId>
  <version>${syncope.version}</version>
</dependency>
<dependency>
  <groupId>org.apache.syncope.ext.camel</groupId>
  <artifactId>syncope-ext-camel-persistence-jpa</artifactId>
  <version>${syncope.version}</version>
</dependency>
<dependency>
  <groupId>org.apache.syncope.ext.camel</groupId>
  <artifactId>syncope-ext-camel-provisioning</artifactId>
  <version>${syncope.version}</version>
</dependency>
----

Copy `core/src/main/resources/all/provisioning.properties` to `core/src/main/resources/provisioning.properties`.

[discrete]
===== Enable the <<swagger>> extension

Add the following dependency to `core/pom.xml`:

[source,xml,subs="verbatim,attributes"]
----
<dependency>
  <groupId>org.apache.syncope.ext</groupId>
  <artifactId>syncope-ext-swagger-ui</artifactId>
  <version>${syncope.version}</version>
</dependency>
----

[[customization-console]]
==== Console

[CAUTION]
When providing custom Java classes implementing the defined interfaces or extending the existing
implementations, their package *must* be rooted under `org.apache.syncope.client.console`, otherwise they will not be
available at runtime.

[discrete]
===== Enable the <<apache-camel-provisioning-manager>>

Add the following dependency to `console/pom.xml`:

[source,xml,subs="verbatim,attributes"]
----
<dependency>
  <groupId>org.apache.syncope.ext.camel</groupId>
  <artifactId>syncope-ext-camel-client-console</artifactId>
  <version>${syncope.version}</version>
</dependency>
----

[[customization-enduser]]
==== Enduser

Given the nature of the <<enduser-application>>, all the files required by the AngularJS-based frontend to run are
generated under the local project's `enduser/src/main/webapp/app/` directory and are available for full customization.

The files in use by the Apache Wicket-based backend are still subject to the general
<<override-behavior,override behavior>>, instead.

[[customization-extensions]]
==== Extensions

<<extensions>> can be part of a local project, to encapsulate special features which are specific to a given deployment.

For example, the http://www.chorevolution.eu/[CHOReVOLUTION^] IdM - based on Apache Syncope - provides
https://tuleap.ow2.org/plugins/git/chorevolution/syncope?p=syncope.git&a=tree&f=ext/choreography[an extension^]
for managing via the <<core>> and visualizing via the <<admin-console-component>> the running choreography instances.

//
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements.  See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership.  The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License.  You may obtain a copy of the License at
//
//   http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied.  See the License for the
// specific language governing permissions and limitations
// under the License.
//
=== Customization

[CAUTION]
Only Maven projects can be customized: if using Standalone, Debian packages or the GUI installer, none of the
customizations discussed below can be applied.

Apache Syncope is designed to be as flexible as possible, to best suit the various environments
in which it can be deployed. Besides other aspects, this means that every feature and component can be extended or
replaced.

Once the project has been created from the provided Maven archetype, the generated source tree is available for either
adding new features or replacing existing components.

[[override-behavior]]
[TIP]
.Override behavior
====
As a rule of thumb, any file of the local project will take precedence over a file with the same name in the same
directory of the standard Apache Syncope release.

For example, if you place

 core/spring/src/main/java/org/apache/syncope/core/spring/security/SyncopeAuthenticationProvider.java

in the local project, this file will be picked up instead of
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/spring/src/main/java/org/apache/syncope/core/spring/security/SyncopeAuthenticationProvider.java[SyncopeAuthenticationProvider^].
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/master/core/spring/src/main/java/org/apache/syncope/core/spring/security/SyncopeAuthenticationProvider.java[SyncopeAuthenticationProvider^].
endif::[]

The same happens with resources as images or HTML files; if you place

 console/src/main/resources/org/apache/syncope/client/console/pages/BasePage.html

in the local project, this file will be picked up instead of
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/client/console/src/main/resources/org/apache/syncope/client/console/pages/BasePage.html[BasePage.html^].
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/master/client/console/src/main/resources/org/apache/syncope/client/console/pages/BasePage.html[BasePage.html^].
endif::[]

This general behavior might have exceptions, as highlighted below.
====

In general, the Embedded Mode (see the
ifeval::["{backend}" == "html5"]
http://syncope.apache.org/docs/getting-started.html[Apache Syncope Getting Started Guide]
endif::[]
ifeval::["{backend}" == "pdf"]
http://syncope.apache.org/docs/getting-started.pdf[Apache Syncope Getting Started Guide]
endif::[]
for details) allows the user to work comfortably from a single workstation, with no need of additional setup; it is
effectively implemented as the `all`
https://maven.apache.org/guides/introduction/introduction-to-profiles.html[Maven profile^], where the available optional
components and extensions are enabled. +
When deploying the generated WAR artifacts into an external <<javaee-container>> however, the required components and
extensions need to be explicitly selected and enabled, as shown in the following text.

[[deployment-directories]]
.Deployment directories
****
Apache Syncope needs three base directories to be defined:

* bundles - where the <<external-resources,connector bundles>> are stored;
* log - where all the system logs are written;
* conf (optional) - where configuration files are located, if overriding the default values is needed.

[WARNING]
The `bundles` directory should only contain connector bundle JAR files. +
The presence of any other file might cause the unavailability of any connector bundle in Apache Syncope.

For reference, the suggested directory layout can be created as follows:

....
$ mkdir /opt/syncope
$ mkdir /opt/syncope/bundles
$ mkdir /opt/syncope/log
$ mkdir /opt/syncope/conf
....
****

The WAR artifacts are generated by running the Maven command (with reference to the suggested directory layout):

....
mvn clean verify \
  -Dconf.directory=/opt/syncope/conf \
  -Dbundles.directory=/opt/syncope/bundles \
  -Dlog.directory=/opt/syncope/log
....

After downloading all of the dependencies that are needed, three WAR files will be produced:

. `core/target/syncope.war`
. `console/target/syncope-console.war`
. `enduser/target/syncope-enduser.war`

If no failures are encountered, your basic Apache Syncope project is now ready to be deployed.

[[embedded-debug]]
[TIP]
.JPDA Debug in Embedded Mode
====
The Java™ Platform Debugger Architecture (http://docs.oracle.com/javase/8/docs/technotes/guides/jpda/index.html[JPDA^])
is a collection of APIs aimed to help with debugging Java code.

Enhancing the `embedded` profile of the `enduser` module to enable the JPDA socket is quite
straightforward: just add the `<profile>` below to `enduser/pom.xml`:

[source,xml,subs="verbatim,attributes"]
----
<profile>
  <id>debug</id>

  <build>
    <plugin>
      <plugin>
        <groupId>org.codehaus.cargo</groupId>
        <artifactId>cargo-maven2-plugin</artifactId>
        <inherited>true</inherited>
        <configuration>
          <configuration>
            <properties>
              <cargo.jvmargs>
              -Xdebug
              -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n
              -noverify -XX:+CMSClassUnloadingEnabled
              -XX:+UseConcMarkSweepGC -Xmx1024m -Xms512m
              </cargo.jvmargs>
            </properties>
          </configuration>
        </configuration>
      </plugin>
    </plugins>
  </build>
</profile>
----

Now, from the `enduser` subdirectory, execute:

[source,bash]
mvn -P embedded,debug

At this point your favorite IDE can be attached to the port `8000`; please note that you might need to add
`-XX:MaxPermSize=512m` to `<cargo.jvmargs>` in order to run with JDK 7.
====

[[customization-core]]
==== Core

[CAUTION]
When providing custom Java classes implementing the defined interfaces or extending the existing
implementations, their package *must* be rooted under `org.apache.syncope.core`, otherwise they will not be available
at runtime.

Besides replacing existing classes as explained <<override-behavior,above>>, new implementations can be provided under
`core/src/main/java` for the following components:

* <<propagationactions,propagation>>, <<pushactions,push>>, <<pullactions,pull>> and <<logicactions,logic>> actions
* <<push-correlation-rules,push>> / <<pull-correlation-rules,pull>> correlation rules
* <<pull-mode,reconciliation filter builders>>
* <<tasks-custom,custom tasks>>
* <<reportlets,reportlets>>
* <<account-rules,account>> and <<password-rules,password>> rules for policies
* <<plain,plain schema validators>>
* <<mapping,mapping item transformers>>
* <<virtual-attribute-cache,virtual attribute cache>>
* <<workflow-adapters,workflow adapters>>
* <<provisioning-managers,provisioning managers>>
* <<notifications,notification recipient providers>>

[[new-rest-endpoints]]
[TIP]
.New REST endpoints
====
Adding a new REST endpoint involves several operations:

. create - in an extension's `rest-api` module or under `common` otherwise - a Java interface with package
`org.apache.syncope.common.rest.api.service` and proper JAX-RS annotations; check
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/ext/camel/rest-api/src/main/java/org/apache/syncope/common/rest/api/service/CamelRouteService.java[CamelRouteService^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/master/ext/camel/rest-api/src/main/java/org/apache/syncope/common/rest/api/service/CamelRouteService.java[CamelRouteService^]
endif::[]
for reference;
. if needed, define supporting payload objects - in an extension's `common-lib` module or under `common` otherwise;
check
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/ext/camel/common-lib/src/main/java/org/apache/syncope/common/lib/to/CamelRouteTO.java[CamelRouteTO^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/master/ext/camel/common-lib/src/main/java/org/apache/syncope/common/lib/to/CamelRouteTO.java[CamelRouteTO^]
endif::[]
for reference;
. implement - in an extension's `rest-cxf` module or under `core` otherwise -  the interface defined above in a Java
class with package `org.apache.syncope.core.rest.cxf.service`; check
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/ext/camel/rest-cxf/src/main/java/org/apache/syncope/core/rest/cxf/service/CamelRouteServiceImpl.java[CamelRouteServiceImpl^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/master/ext/camel/rest-cxf/src/main/java/org/apache/syncope/core/rest/cxf/service/CamelRouteServiceImpl.java[CamelRouteServiceImpl^]
endif::[]
for reference.

By following such conventions, the new REST endpoint will be automatically picked up alongside the default services.
====

[WARNING]
====
The <<override-behavior,override behavior>> might have exceptions; if you need to customize one of the
Spring context definitions. For example, if you want to customize
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/spring/src/main/resources/securityContext.xml[securityContext.xml^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/master/core/spring/src/main/resources/securityContext.xml[securityContext.xml^]
endif::[]
, you will also need to replace the following text in `core/src/main/webapp/WEB-INF/web.xml`,

....
classpath*:/*Context.xml
....

with

....
classpath:/coreContext.xml
classpath:/securityContext.xml
classpath*:/logicContext.xml
classpath*:/restCXFContext.xml
classpath*:/persistenceContext.xml
classpath*:/provisioningContext.xml
classpath*:/workflowContext.xml
....

to be sure that `core/src/main/resources/securityContext.xml` is picked up. +
Please also note that the actual list of Spring context files to include might depend on the configured extensions.
====

[discrete]
===== Select the <<activiti-user-workflow-adapter>>

Add the following dependency to `core/pom.xml`:

[source,xml,subs="verbatim,attributes"]
----
<dependency>
  <groupId>org.apache.syncope.core</groupId>
  <artifactId>syncope-core-workflow-activiti</artifactId>
  <version>${syncope.version}</version>
</dependency>
----

Copy `core/src/main/resources/all/workflow.properties` to `core/src/main/resources/workflow.properties`.

[discrete]
===== Enable the <<apache-camel-provisioning-manager>>

Add the following dependencies to `core/pom.xml`:

[source,xml,subs="verbatim,attributes"]
----
<dependency>
  <groupId>org.apache.syncope.ext.camel</groupId>
  <artifactId>syncope-ext-camel-rest-cxf</artifactId>
  <version>${syncope.version}</version>
</dependency>
<dependency>
  <groupId>org.apache.syncope.ext.camel</groupId>
  <artifactId>syncope-ext-camel-persistence-jpa</artifactId>
  <version>${syncope.version}</version>
</dependency>
<dependency>
  <groupId>org.apache.syncope.ext.camel</groupId>
  <artifactId>syncope-ext-camel-provisioning</artifactId>
  <version>${syncope.version}</version>
</dependency>
----

Copy `core/src/main/resources/all/provisioning.properties` to `core/src/main/resources/provisioning.properties`.

[discrete]
===== Enable the <<swagger>> extension

Add the following dependency to `core/pom.xml`:

[source,xml,subs="verbatim,attributes"]
----
<dependency>
  <groupId>org.apache.syncope.ext</groupId>
  <artifactId>syncope-ext-swagger-ui</artifactId>
  <version>${syncope.version}</version>
</dependency>
----

[[customization-console]]
==== Console

[CAUTION]
When providing custom Java classes implementing the defined interfaces or extending the existing
implementations, their package *must* be rooted under `org.apache.syncope.client.console`, otherwise they will not be
available at runtime.

[discrete]
===== Enable the <<apache-camel-provisioning-manager>>

Add the following dependency to `console/pom.xml`:

[source,xml,subs="verbatim,attributes"]
----
<dependency>
  <groupId>org.apache.syncope.ext.camel</groupId>
  <artifactId>syncope-ext-camel-client-console</artifactId>
  <version>${syncope.version}</version>
</dependency>
----

[[customization-enduser]]
==== Enduser

Given the nature of the <<enduser-application>>, all the files required by the AngularJS-based frontend to run are
generated under the local project's `enduser/src/main/webapp/app/` directory and are available for full customization.

The files in use by the Apache Wicket-based backend are still subject to the general
<<override-behavior,override behavior>>, instead.

[[customization-extensions]]
==== Extensions

<<extensions>> can be part of a local project, to encapsulate special features which are specific to a given deployment.

For example, the http://www.chorevolution.eu/[CHOReVOLUTION^] IdM - based on Apache Syncope - provides
https://tuleap.ow2.org/plugins/git/chorevolution/syncope?p=syncope.git&a=tree&f=ext/choreography[an extension^]
for managing via the <<core>> and visualizing via the <<admin-console-component>> the running choreography instances.


Fix typo

Fix typo

- https://github.com/openshift/openshift-docs/issues/21094

:context: nodes-pods-device
[id="nodes-pods-device"]
= Using device plug-ins to access external resources with pods
include::modules/common-attributes.adoc[]

toc::[]

Device plug-ins allow you to use a particular device type (GPU, InfiniBand,
or other similar computing resources that require vendor-specific initialization
and setup) in your {product-title} pod without needing to write custom code.

// The following include statements pull in the module files that comprise
// the assembly. Include any combination of concept, procedure, or reference
// modules required to cover the user story. You can also include other
// assemblies.

include::modules/nodes-pods-plugins-about.adoc[leveloffset=+1]

include::modules/nodes-pods-plugins-device-mgr.adoc[leveloffset=+1]

include::modules/nodes-pods-plugins-install.adoc[leveloffset=+1]

:context: nodes-pods-device
[id="nodes-pods-device"]
= Using device plug-ins to access external resouces with pods
include::modules/common-attributes.adoc[]

toc::[]

Device plug-ins allow you to use a particular device type (GPU, InfiniBand,
or other similar computing resources that require vendor-specific initialization
and setup) in your {product-title} pod without needing to write custom code.

// The following include statements pull in the module files that comprise
// the assembly. Include any combination of concept, procedure, or reference
// modules required to cover the user story. You can also include other
// assemblies.

include::modules/nodes-pods-plugins-about.adoc[leveloffset=+1]

include::modules/nodes-pods-plugins-device-mgr.adoc[leveloffset=+1]

include::modules/nodes-pods-plugins-install.adoc[leveloffset=+1]


Replaced all the neo4j stuff

Replaced all the neo4j stuff

= Documentation Overview
// Authors, seperated by;. They might not be shown in the overview but will be generated
Ronny Trommer <ronny@opennms.org>; Markus von Rüden <mvr@opennms.com>
:ascii-ids:

[[community-docs]]
== Writing OpenNMS Documentation

NOTE: Other than writing documentation, you can help out by providing comments about improvements or reporting bugs - head over to the http://issues.opennms.org/browse/NMS/component/10011[issue tracker for documentation] to do that!

For how to build the manual see:
https://github.com/OpenNMS/opennms/blob/development/docs/opennms-doc/doc-overview/README.adoc[readme]

The documents use the AsciiDoc format, see:

* http://www.methods.co.nz/asciidoc/[Aciidoc Reference]
* http://www.methods.co.nz/asciidoc/faq.html[AsciiDoc FAQ]
* http://powerman.name/doc/asciidoc[AsciiDoc cheatsheet]
* http://xpt.sourceforge.net/techdocs/nix/tool/asciidoc-syn/ascs01-AsciiDocMarkupSyntaxQuickSummary/single/[AsciiDoc Cheatsheet]

Here you can find other resources to get familiar with AsciiDoc, see:

 * http://asciidoctor.org/docs/user-manual[AsciiDoc User Manual]
 * http://asciidoctor.org/docs/install-and-use-asciidoctor-maven-plugin/[AsciiDoc Maven Plugin]
 * https://groups.google.com/forum/?fromgroups#!forum/asciidoc[AsciiDoc discussion list]
 * http://code.google.com/p/asciidoc/issues/list[AsciiDoc issue tracker]
 * https://github.com/oreillymedia/docbook2asciidoc[Docbook to AsciiDoc]
 * http://blog.rainwebs.net/2010/02/25/how-to-create-handsome-pdf-documents-without-frustration/[How to create handsome PDF documents without frustration]

The cheatsheets are really useful!

[[community-docs-overall-flow]]
== Overall Flow ==

Each (sub)project represents a part of the documentation, which will produce a HTML output in the file system.
The output is generated in the `target/generated` sources folder.
Asciidoc documents have the +.asciidoc+ file extension.

Note that different ways to add documentation works best for different cases:

* Tutorials and How To's should be published on the http://wiki.opennms.org[OpenNMS Wiki].
For example you want to describe how to use the Net-SNMP agent and the SNMP monitor from OpenNMS to solve a special use case with OpenNMS.

* The documentation you can find in the source code can be characterized as a non-emotional technical documentation which explains concepts in detail and should be complete.

== File Structure in 'opennms-doc' ==

[options="header", cols="e,d"]
|========================
| Directory | Contents
| doc-overview/ | module with this overview documentation
| guide-user/ | module with the guide for people how to use and configure OpenNMS
| guide-admin/ | module with the guide for administrators optimizing and running OpenNMS
| guide-development/ | module with the guide for people who want to develop within OpenNMS
| guide-install/ | module with the guide how to install OpenNMS on different operating systems
| releasenotes/ | module with the changelog and release notes
|========================

== Headings and document structure ==

Each document starts over with headings from level zero (the document title).
Each document should have an id.
In some cases sections in the document need to have id's as well, this depends on where they fit in the overall structure.
To be able to link to content, it has to have an id. Missing id's in mandatory places will fail the build.

This is how a document should start:

[source]
----
[[unique-id-verbose-is-ok]]
= The Document Title =
----

To push the headings down to the right level in the output, the +leveloffset+
attribute is used when including the document inside of another document.

Subsequent headings in a document should use the following syntax:

[source]
----
== Subheading ==

... content here ...

=== Subsubheading ===

content here ...

----

== Writing ==

Put one sentence on each line.
This makes it easy to move content around, and also easy to spot (too) long sentences.

== Gotchas ==

* Always leave a blank line at the end of documents
  (or the title of the next document might end up in the last
  paragraph of the document)
* As +{}+ are used for Asciidoc attributes, everything inside will be treated as an attribute.
  What you have to do is to escape the opening brace: +\\{+.
  If you don't, the braces and the text inside them will be removed without any warning being issued!

== Links ==

To link to other parts of the manual the id of the target is used.
This is how such a reference looks:

[source]
----
<<community-docs-overall-flow>>
----

Which will render like: <<community-docs-overall-flow>>

[NOTE]
Just write "see \<<target-id>>" and similar, that should suffice in most cases.

If you need to link to another document with your own link text, this is what to do:

[source]
----
<<target-id, link text that fits in the context>>
----

NOTE: Having lots of linked text may work well in a web context but is a pain in print, and we aim for both!

External links are added like this:

[source]
----
http://www.opennms.org/[Link text here]
----

Which renders like: http://www.opennms.org/[Link text here]

For short links it may be better not to add a link text, just do:

[source]
----
http://www.opennms.org/
----

Which renders like: http://www.opennms.org/

NOTE: It's ok to have a dot right after the URL, it won't be part of the link.

== Text Formatting ==

* \_Italics_ is rendered as _Italics_ and used for emphasis.
* \*Bold* is rendered as *Bold* and used sparingly, for strong emphasis only.
* \+methodName()+ is rendered as +methodName()+ and is used for literals as well
  (note: the content between the `+` signs _will_ be parsed).
* \`command` is rendered as `command` (typically used for command-line)
  (note: the content between the +`+ signs _will not_ be parsed).
* Mono\+\+space\++d is rendered as Mono++space++d and is used for monospaced letters.
* \'my/path/' is rendered as 'my/path/' (used for file names and paths).
* \\``Double quoted'' (that is two grave accents to the left and two acute accents to the right) renders as ``Double quoted''.
* \`Single quoted' (that is a single grave accent to the left and a single acute accent to the right) renders as `Single quoted'.

== Admonitions ==

These are very useful and should be used where appropriate.
Choose from the following (write all caps and no, we can't easily add new ones):

NOTE: Note.

TIP: Tip.

IMPORTANT: Important

CAUTION: Caution

WARNING: Warning

Here's how it's done:

[source]
----
NOTE: Note.
----

A multiline variation:

[source]
----
[TIP]
Tiptext.
Line 2.
----

Which is rendered as:

[TIP]
Tiptext.
Line 2.

== Images ==

IMPORTANT: _All images in the entire manual share the same namespace._
  You know how to handle that.

=== Images Files ===

To include an image file, make sure it resides in the 'images/' directory relative to the document you're including it from. Then go:

[source]
----
image::opennms-logo.png[]
----

Which is rendered as:

image::opennms-logo.png[]

=== Static Graphviz/DOT ===

We use the Graphviz/DOT language to describe graphs.
For documentation see http://graphviz.org/.

This is how to include a simple example graph:

[source]
----
 ["dot", "community-docs-graphdb-rels.svg"]
 ----
 "Start node" -> "End node" [label="relationship"]
 ----
----

Which is rendered as:

["dot", "community-docs-graphdb-rels.svg"]
----
"Start node" -> "End node" [label="relationship"]
----

Here's an example using some predefined variables available in the build:

[source]
----
 ["dot", "community-docs-graphdb-rels-overview.svg", "meta"]
 ----
 "A Relationship" [fillcolor="NODEHIGHLIGHT"]
 "Start node" [fillcolor="NODE2HIGHLIGHT"]
 "A Relationship" -> "Start node" [label="has a"]
 "A Relationship" -> "End node" [label="has a"]
 "A Relationship" -> "Relationship type" [label="has a"]
 "Name" [TEXTNODE]
 "Relationship type" -> "Name" [label="uniquely identified by" color="EDGEHIGHLIGHT" fontcolor="EDGEHIGHLIGHT"]
 ----
----

Which is rendered as:

["dot", "community-docs-graphdb-rels-overview.svg", "meta"]
----
"A Relationship" [fillcolor="NODEHIGHLIGHT"]
"Start node" [fillcolor="NODE2HIGHLIGHT"]
"A Relationship" -> "Start node" [label="has a"]
"A Relationship" -> "End node" [label="has a"]
"A Relationship" -> "Relationship type" [label="has a"]
"Name" [TEXTNODE]
"Relationship type" -> "Name" [label="uniquely identified by" color="EDGEHIGHLIGHT" fontcolor="EDGEHIGHLIGHT"]
----

The optional second argument given to the dot filter defines the style to use:

* when not defined: Default styling for nodespace examples.
* +neoviz+: Nodespace view generated by Neoviz.
* +meta+: Graphs that don't resemble db contents, but rather concepts.

CAUTION: Keywords of the DOT language have to be surrounded by double quotes when used for other purposes.
  The keywords include _node, edge, graph, digraph, subgraph,_ and _strict_.

== Attributes ==

Common attributes you can use in documents:

* \{opennms-version} - rendered as "{opennms-version}"
* \{opennms-git-tag} - rendered as "{opennms-git-tag}"

These can substitute part of URLs that point to for example APIdocs or source code.
Note that opennms-git-tag also handles the case of snapshot/master.

Sample Asciidoc attributes which can be used:

* \{docdir} - root directory of the documents
* \{nbsp} - non-breaking space

== Comments ==

There's a separate build including comments.
The comments show up with a yellow background.
This build doesn't run by default, but after a normal build, you can use `make annotated` to build it.
You can also use the resulting page to search for content, as the full manual is on a single page.

Here's how to write a comment:

[source]
----
// this is a comment
----

The comments are not visible in the normal build.
Comment blocks won't be included in the output of any build at all.
Here's a comment block:

[source]
----
////
Note that includes in here will still be processed, but not make it into the output.
That is, missing includes here will still break the build!
////
----

== Code Snippets ==

=== Explicitly defined in the document ===

WARNING: Use this kind of code snippets as little as possible.
  They are well known to get out of sync with reality after a while.

This is how to do it:

[source,xml]
----
<service name="DNS" interval="300000" user-defined="false" status="on">
  <parameter key="retry" value="2" />
  <parameter key="timeout" value="5000" />
  <parameter key="port" value="53" />
  <parameter key="lookup" value="localhost" />
  <parameter key="fatal-response-codes" value="2,3,5" /><!-- ServFail, NXDomain, Refused -->
  <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
  <parameter key="rrd-base-name" value="dns" />
  <parameter key="ds-name" value="dns" />
</service>
----

If there's no suitable syntax highlighter, just omit the language: +[source]+.

Currently the following syntax highlighters are enabled:

* Bash
* Groovy
* Java
* JavaScript
* Python
* XML

For other highlighters we could add see https://code.google.com/p/google-code-prettify/.

=== Fetched from source code ===

Code can be automatically fetched from source files.
You need to define:

* component: the +artifactId+ of the Maven coordinates,
* source: path to the file inside the jar it's deployed to,
* classifier: +sources+ or +test-sources+ or any other classifier pointing to the artifact,
* tag: tag name to search the file for,
* the language of the code, if a corresponding syntax highlighter is available.

Note that the artifact has to be included as a Maven dependency of the Manual project so that the files can be found.

The file will be searched for lines including +START SNIPPET: {tag}+ and +END SNIPPET: {tag}+, the lines between those will go into the output.
Be aware of that the tag "abc" will match "abcd" as well.
It's a simple on/off switch, meaning that multiple occurrences will be assembled into a single code snippet in the output.
This behavior can be user to hide away assertions from code examples sourced from tests.

This is how to define a code snippet inclusion:

[source]
----
 [snippet,java]
 ----
 component=opennms-examples
 source=org/opennms/examples/JmxDocTest.java
 classifier=test-sources
 tag=getStartTime
 ----
----

This is how it renders:

[snippet,java]
----
component=opennms-examples
source=org/opennms/examples/JmxDocTest.java
classifier=test-sources
tag=getStartTime
----

=== Query Results ===

There's a special filter for Cypher query results.
This is how to tag a query result:

[source]
----
 .Result
 [queryresult]
 ----
 +----------------------------------+
 | friend_of_friend.name | count(*) |
 +----------------------------------+
 | Ian                   | 2        |
 | Derrick               | 1        |
 | Jill                  | 1        |
 +----------------------------------+
 3 rows, 12 ms
 ----
----

This is how it renders:

.Result
[queryresult]
----
+----------------------------------+
| friend_of_friend.name | count(*) |
+----------------------------------+
| Ian                   | 2        |
| Derrick               | 1        |
| Jill                  | 1        |
+----------------------------------+
3 rows, 12 ms
----

== A sample Java based documentation test ==

For Java, there are a couple of premade utilities that keep code and documentation together in
Javadocs and code snippets that generate Asciidoc for the rest of the toolchain.

To illustrate this, look at the following documentation that generates the Asciidoc file +hello-world-title.asciidoc+ with a content of:

[source]
------------------------------
include::{importdir}/opennms-examples-docs-jar/dev/examples/hello-world-sample-chapter.asciidoc[]
------------------------------

this file is included in this documentation via

[source]
----
  :leveloffset: 3
  include::{importdir}/opennms-examples-docs-jar/dev/examples/hello-world-sample-chapter.asciidoc[]
----

which renders the following chapter:

:leveloffset: 3

include::{importdir}/opennms-examples-docs-jar/dev/examples/hello-world-sample-chapter.asciidoc[]

:leveloffset: 2

== Images

.pris-overview.graphml
image::images/pris-overview.png[pris-overview.graphml]

.example.odp
image::images/example.png[example.odp]

= Documentation Overview
// Authors, seperated by;. They might not be shown in the overview but will be generated
Ronny Trommer <ronny@opennms.org>; Markus von Rüden <mvr@opennms.com>
:ascii-ids:

[[community-docs]]
== Writing OpenNMS Documentation

NOTE: Other than writing documentation, you can help out by providing comments about improvements or reporting bugs - head over to the http://issues.opennms.org/browse/NMS/component/10011[issue tracker for documentation] to do that!

For how to build the manual see:
https://github.com/OpenNMS/opennms/blob/development/docs/opennms-doc/doc-overview/README.adoc[readme]

The documents use the AsciiDoc format, see:

* http://www.methods.co.nz/asciidoc/[Aciidoc Reference]
* http://www.methods.co.nz/asciidoc/faq.html[AsciiDoc FAQ]
* http://powerman.name/doc/asciidoc[AsciiDoc cheatsheet]
* http://xpt.sourceforge.net/techdocs/nix/tool/asciidoc-syn/ascs01-AsciiDocMarkupSyntaxQuickSummary/single/[AsciiDoc Cheatsheet]

Here you can find other resources to get familiar with AsciiDoc, see:

 * http://asciidoctor.org/docs/user-manual[AsciiDoc User Manual]
 * http://asciidoctor.org/docs/install-and-use-asciidoctor-maven-plugin/[AsciiDoc Maven Plugin]
 * https://groups.google.com/forum/?fromgroups#!forum/asciidoc[AsciiDoc discussion list]
 * http://code.google.com/p/asciidoc/issues/list[AsciiDoc issue tracker]
 * https://github.com/oreillymedia/docbook2asciidoc[Docbook to AsciiDoc]
 * http://blog.rainwebs.net/2010/02/25/how-to-create-handsome-pdf-documents-without-frustration/[How to create handsome PDF documents without frustration]

The cheatsheets are really useful!

[[community-docs-overall-flow]]
== Overall Flow ==

Each (sub)project represents a part of the documentation, which will produce a HTML output in the file system.
The output is generated in the `target/generated` sources folder.
Asciidoc documents have the +.asciidoc+ file extension.

Note that different ways to add documentation works best for different cases:

* Tutorials and How To's should be published on the http://wiki.opennms.org[OpenNMS Wiki].
For example you want to describe how to use the Net-SNMP agent and the SNMP monitor from OpenNMS to solve a special use case with OpenNMS.

* The documentation you can find in the source code can be characterized as a non-emotional technical documentation which explains concepts in detail and should be complete.

== File Structure in 'opennms-doc' ==

[options="header", cols="e,d"]
|========================
| Directory | Contents
| doc-overview/ | module with this overview documentation
| guide-user/ | module with the guide for people how to use and configure OpenNMS
| guide-admin/ | module with the guide for administrators optimizing and running OpenNMS
| guide-development/ | module with the guide for people who want to develop within OpenNMS
| guide-install/ | module with the guide how to install OpenNMS on different operating systems
| releasenotes/ | module with the changelog and release notes
|========================

== Headings and document structure ==

Each document starts over with headings from level zero (the document title).
Each document should have an id.
In some cases sections in the document need to have id's as well, this depends on where they fit in the overall structure.
To be able to link to content, it has to have an id. Missing id's in mandatory places will fail the build.

This is how a document should start:

[source]
----
[[unique-id-verbose-is-ok]]
= The Document Title =
----

To push the headings down to the right level in the output, the +leveloffset+
attribute is used when including the document inside of another document.

Subsequent headings in a document should use the following syntax:

[source]
----
== Subheading ==

... content here ...

=== Subsubheading ===

content here ...

----

== Writing ==

Put one sentence on each line.
This makes it easy to move content around, and also easy to spot (too) long sentences.

== Gotchas ==

* Always leave a blank line at the end of documents
  (or the title of the next document might end up in the last
  paragraph of the document)
* As +{}+ are used for Asciidoc attributes, everything inside will be treated as an attribute.
  What you have to do is to escape the opening brace: +\\{+.
  If you don't, the braces and the text inside them will be removed without any warning being issued!

== Links ==

To link to other parts of the manual the id of the target is used.
This is how such a reference looks:

[source]
----
<<community-docs-overall-flow>>
----

Which will render like: <<community-docs-overall-flow>>

[NOTE]
Just write "see \<<target-id>>" and similar, that should suffice in most cases.

If you need to link to another document with your own link text, this is what to do:

[source]
----
<<target-id, link text that fits in the context>>
----

NOTE: Having lots of linked text may work well in a web context but is a pain in print, and we aim for both!

External links are added like this:

[source]
----
http://www.opennms.org/[Link text here]
----

Which renders like: http://www.opennms.org/[Link text here]

For short links it may be better not to add a link text, just do:

[source]
----
http://www.opennms.org/
----

Which renders like: http://www.opennms.org/

NOTE: It's ok to have a dot right after the URL, it won't be part of the link.

== Text Formatting ==

* \_Italics_ is rendered as _Italics_ and used for emphasis.
* \*Bold* is rendered as *Bold* and used sparingly, for strong emphasis only.
* \+methodName()+ is rendered as +methodName()+ and is used for literals as well
  (note: the content between the `+` signs _will_ be parsed).
* \`command` is rendered as `command` (typically used for command-line)
  (note: the content between the +`+ signs _will not_ be parsed).
* Mono\+\+space\++d is rendered as Mono++space++d and is used for monospaced letters.
* \'my/path/' is rendered as 'my/path/' (used for file names and paths).
* \\``Double quoted'' (that is two grave accents to the left and two acute accents to the right) renders as ``Double quoted''.
* \`Single quoted' (that is a single grave accent to the left and a single acute accent to the right) renders as `Single quoted'.

== Admonitions ==

These are very useful and should be used where appropriate.
Choose from the following (write all caps and no, we can't easily add new ones):

NOTE: Note.

TIP: Tip.

IMPORTANT: Important

CAUTION: Caution

WARNING: Warning

Here's how it's done:

[source]
----
NOTE: Note.
----

A multiline variation:

[source]
----
[TIP]
Tiptext.
Line 2.
----

Which is rendered as:

[TIP]
Tiptext.
Line 2.

== Images ==

IMPORTANT: _All images in the entire manual share the same namespace._
  You know how to handle that.

=== Images Files ===

To include an image file, make sure it resides in the 'images/' directory relative to the document you're including it from. Then go:

[source]
----
image::neo4j-logo.png[]
----

Which is rendered as:

image::neo4j-logo.png[]

=== Static Graphviz/DOT ===

We use the Graphviz/DOT language to describe graphs.
For documentation see http://graphviz.org/.

This is how to include a simple example graph:

[source]
----
 ["dot", "community-docs-graphdb-rels.svg"]
 ----
 "Start node" -> "End node" [label="relationship"]
 ----
----

Which is rendered as:

["dot", "community-docs-graphdb-rels.svg"]
----
"Start node" -> "End node" [label="relationship"]
----

Here's an example using some predefined variables available in the build:

[source]
----
 ["dot", "community-docs-graphdb-rels-overview.svg", "meta"]
 ----
 "A Relationship" [fillcolor="NODEHIGHLIGHT"]
 "Start node" [fillcolor="NODE2HIGHLIGHT"]
 "A Relationship" -> "Start node" [label="has a"]
 "A Relationship" -> "End node" [label="has a"]
 "A Relationship" -> "Relationship type" [label="has a"]
 "Name" [TEXTNODE]
 "Relationship type" -> "Name" [label="uniquely identified by" color="EDGEHIGHLIGHT" fontcolor="EDGEHIGHLIGHT"]
 ----
----

Which is rendered as:

["dot", "community-docs-graphdb-rels-overview.svg", "meta"]
----
"A Relationship" [fillcolor="NODEHIGHLIGHT"]
"Start node" [fillcolor="NODE2HIGHLIGHT"]
"A Relationship" -> "Start node" [label="has a"]
"A Relationship" -> "End node" [label="has a"]
"A Relationship" -> "Relationship type" [label="has a"]
"Name" [TEXTNODE]
"Relationship type" -> "Name" [label="uniquely identified by" color="EDGEHIGHLIGHT" fontcolor="EDGEHIGHLIGHT"]
----

The optional second argument given to the dot filter defines the style to use:

* when not defined: Default styling for nodespace examples.
* +neoviz+: Nodespace view generated by Neoviz.
* +meta+: Graphs that don't resemble db contents, but rather concepts.

CAUTION: Keywords of the DOT language have to be surrounded by double quotes when used for other purposes.
  The keywords include _node, edge, graph, digraph, subgraph,_ and _strict_.

== Attributes ==

Common attributes you can use in documents:

* \{opennms-version} - rendered as "{opennms-version}"
* \{opennms-git-tag} - rendered as "{opennms-git-tag}"

These can substitute part of URLs that point to for example APIdocs or source code.
Note that opennms-git-tag also handles the case of snapshot/master.

Sample Asciidoc attributes which can be used:

* \{docdir} - root directory of the documents
* \{nbsp} - non-breaking space

== Comments ==

There's a separate build including comments.
The comments show up with a yellow background.
This build doesn't run by default, but after a normal build, you can use `make annotated` to build it.
You can also use the resulting page to search for content, as the full manual is on a single page.

Here's how to write a comment:

[source]
----
// this is a comment
----

The comments are not visible in the normal build.
Comment blocks won't be included in the output of any build at all.
Here's a comment block:

[source]
----
////
Note that includes in here will still be processed, but not make it into the output.
That is, missing includes here will still break the build!
////
----

== Code Snippets ==

=== Explicitly defined in the document ===

WARNING: Use this kind of code snippets as little as possible.
  They are well known to get out of sync with reality after a while.

This is how to do it:

[source,xml]
----
<service name="DNS" interval="300000" user-defined="false" status="on">
  <parameter key="retry" value="2" />
  <parameter key="timeout" value="5000" />
  <parameter key="port" value="53" />
  <parameter key="lookup" value="localhost" />
  <parameter key="fatal-response-codes" value="2,3,5" /><!-- ServFail, NXDomain, Refused -->
  <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
  <parameter key="rrd-base-name" value="dns" />
  <parameter key="ds-name" value="dns" />
</service>
----

If there's no suitable syntax highlighter, just omit the language: +[source]+.

Currently the following syntax highlighters are enabled:

* Bash
* Groovy
* Java
* JavaScript
* Python
* XML

For other highlighters we could add see https://code.google.com/p/google-code-prettify/.

=== Fetched from source code ===

Code can be automatically fetched from source files.
You need to define:

* component: the +artifactId+ of the Maven coordinates,
* source: path to the file inside the jar it's deployed to,
* classifier: +sources+ or +test-sources+ or any other classifier pointing to the artifact,
* tag: tag name to search the file for,
* the language of the code, if a corresponding syntax highlighter is available.

Note that the artifact has to be included as a Maven dependency of the Manual project so that the files can be found.

The file will be searched for lines including +START SNIPPET: {tag}+ and +END SNIPPET: {tag}+, the lines between those will go into the output.
Be aware of that the tag "abc" will match "abcd" as well.
It's a simple on/off switch, meaning that multiple occurrences will be assembled into a single code snippet in the output.
This behavior can be user to hide away assertions from code examples sourced from tests.

This is how to define a code snippet inclusion:

[source]
----
 [snippet,java]
 ----
 component=opennms-examples
 source=org/opennms/examples/JmxDocTest.java
 classifier=test-sources
 tag=getStartTime
 ----
----

This is how it renders:

[snippet,java]
----
component=neo4j-examples
source=org/neo4j/examples/JmxDocTest.java
classifier=test-sources
tag=getStartTime
----

=== Query Results ===

There's a special filter for Cypher query results.
This is how to tag a query result:

[source]
----
 .Result
 [queryresult]
 ----
 +----------------------------------+
 | friend_of_friend.name | count(*) |
 +----------------------------------+
 | Ian                   | 2        |
 | Derrick               | 1        |
 | Jill                  | 1        |
 +----------------------------------+
 3 rows, 12 ms
 ----
----

This is how it renders:

.Result
[queryresult]
----
+----------------------------------+
| friend_of_friend.name | count(*) |
+----------------------------------+
| Ian                   | 2        |
| Derrick               | 1        |
| Jill                  | 1        |
+----------------------------------+
3 rows, 12 ms
----

== A sample Java based documentation test ==

For Java, there are a couple of premade utilities that keep code and documentation together in
Javadocs and code snippets that generate Asciidoc for the rest of the toolchain.

To illustrate this, look at the following documentation that generates the Asciidoc file +hello-world-title.asciidoc+ with a content of:

[source]
------------------------------
include::{importdir}/neo4j-examples-docs-jar/dev/examples/hello-world-sample-chapter.asciidoc[]
------------------------------

this file is included in this documentation via

[source]
----
  :leveloffset: 3
  include::{importdir}/neo4j-examples-docs-jar/dev/examples/hello-world-sample-chapter.asciidoc[]
----

which renders the following chapter:

:leveloffset: 3

include::{importdir}/neo4j-examples-docs-jar/dev/examples/hello-world-sample-chapter.asciidoc[]

:leveloffset: 2

== Images

.pris-overview.graphml
image::images/pris-overview.png[pris-overview.graphml]

.example.odp
image::images/example.png[example.odp]