Regen docs

Regen docs

[[aws2-lambda-component]]
= AWS Lambda Component
:page-source: components/camel-aws2-lambda/src/main/docs/aws2-lambda-component.adoc

*Since Camel 3.2*

*Since Camel 3.2*

// HEADER START
*Only producer is supported*
// HEADER END

The Lambda component supports create, get, list, delete and invoke
https://aws.amazon.com/lambda/[AWS Lambda] functions.

*Prerequisites*

You must have a valid Amazon Web Services developer account, and be
signed up to use Amazon Lambda. More information is available at
https://aws.amazon.com/lambda/[AWS Lambda].

When creating a Lambda function, you need to specify a IAM role which has at least the AWSLambdaBasicExecuteRole policy attached.

[NOTE]
====
The AWS2 Lambda component is not supported in OSGI
====

== URI Format

[source,java]
-------------------------
aws2-lambda://functionName[?options]
-------------------------

You can append query options to the URI in the following format,
?options=value&option2=value&...

== URI Options

// component options: START
The AWS Lambda component supports 6 options, which are listed below.

[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *accessKey* (producer) | Amazon AWS Access Key |  | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean
| *region* (producer) | Amazon AWS Region |  | String
| *secretKey* (producer) | Amazon AWS Secret Key |  | String
| *basicPropertyBinding* (advanced) | Whether the component should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | boolean
| *configuration* (advanced) | The AWS Lambda default configuration |  | Lambda2Configuration
|===
// component options: END

// endpoint options: START
The AWS Lambda endpoint is configured using URI syntax:

----
aws2-lambda:function
----

with the following path and query parameters:

=== Path Parameters (1 parameters):

[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *function* | *Required* Name of the Lambda function. |  | String
|===

=== Query Parameters (11 parameters):

[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean
| *operation* (producer) | The operation to perform. It can be listFunctions, getFunction, createFunction, deleteFunction or invokeFunction. The value can be one of: listFunctions, getFunction, createAlias, deleteAlias, getAlias, listAliases, createFunction, deleteFunction, invokeFunction, updateFunction, createEventSourceMapping, deleteEventSourceMapping, listEventSourceMapping, listTags, tagResource, untagResource, publishVersion, listVersions | invokeFunction | Lambda2Operations
| *region* (producer) | Amazon AWS Region. When using this parameter, the configuration will expect the capitalized name of the region (for example AP_EAST_1) You'll need to use the name Regions.EU_WEST_1.name() |  | String
| *awsLambdaClient* (advanced) | To use a existing configured AwsLambdaClient as client |  | LambdaClient
| *basicPropertyBinding* (advanced) | Whether the endpoint should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | boolean
| *synchronous* (advanced) | Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported). | false | boolean
| *proxyHost* (proxy) | To define a proxy host when instantiating the Lambda client |  | String
| *proxyPort* (proxy) | To define a proxy port when instantiating the Lambda client |  | Integer
| *proxyProtocol* (proxy) | To define a proxy protocol when instantiating the Lambda client. The value can be one of: HTTP, HTTPS | HTTPS | Protocol
| *accessKey* (security) | Amazon AWS Access Key |  | String
| *secretKey* (security) | Amazon AWS Secret Key |  | String
|===
// endpoint options: END

Required Lambda component options

You have to provide the awsLambdaClient in the
Registry or your accessKey and secretKey to access
the https://aws.amazon.com/lambda/[Amazon Lambda] service.

== Usage

=== Message headers evaluated by the Lambda producer

[width="100%",cols="5%,5%,10%,75%,5%",options="header",]
|=======================================================================
|Operation |Header |Type |Description |Required

|All |`CamelAwsLambdaOperation` |`String` |The operation we want to perform. Override operation passed as query parameter| Yes

|createFunction |`CamelAwsLambdaS3Bucket` |`String` |Amazon S3 bucket name where the .zip file containing
your deployment package is stored. This bucket must reside in the same AWS region where you are creating the Lambda function.| No

|createFunction |`CamelAwsLambdaS3Key` |`String` |The Amazon S3 object (the deployment package) key name
you want to upload.| No

|createFunction |`CamelAwsLambdaS3ObjectVersion` |String |The Amazon S3 object (the deployment package) version
you want to upload.| No

|createFunction |`CamelAwsLambdaZipFile` |`String` |The local path of the zip file (the deployment package).
 Content of zip file can also be put in Message body.| No

|createFunction |`CamelAwsLambdaRole` |`String` |The Amazon Resource Name (ARN) of the IAM role that Lambda assumes
 when it executes your function to access any other Amazon Web Services (AWS) resources. |Yes

|createFunction |`CamelAwsLambdaRuntime` |String |The runtime environment for the Lambda function you are uploading.
 (nodejs, nodejs4.3, nodejs6.10, java8, python2.7, python3.6, dotnetcore1.0, odejs4.3-edge) |Yes

|createFunction |`CamelAwsLambdaHandler` |`String` |The function within your code that Lambda calls to begin execution.
 For Node.js, it is the module-name.export value in your function.
 For Java, it can be package.class-name::handler or package.class-name.|Yes

|createFunction |`CamelAwsLambdaDescription` |`String` |The user-provided description.|No

|createFunction |`CamelAwsLambdaTargetArn` |`String` |The parent object that contains the target ARN (Amazon Resource Name)
of an Amazon SQS queue or Amazon SNS topic.|No

|createFunction |`CamelAwsLambdaMemorySize` |`Integer` |The memory size, in MB, you configured for the function.
Must be a multiple of 64 MB.|No

|createFunction |`CamelAwsLambdaKMSKeyArn` |`String` |The Amazon Resource Name (ARN) of the KMS key used to encrypt your function's environment variables.
If not provided, AWS Lambda will use a default service key.|No

|createFunction |`CamelAwsLambdaPublish` |`Boolean` |This boolean parameter can be used to request AWS Lambda
to create the Lambda function and publish a version as an atomic operation.|No

|createFunction |`CamelAwsLambdaTimeout` |`Integer` |The function execution time at which Lambda should terminate the function.
The default is 3 seconds.|No

|createFunction |`CamelAwsLambdaTracingConfig` |`String` |Your function's tracing settings (Active or PassThrough).|No

|createFunction |`CamelAwsLambdaEnvironmentVariables` |`Map<String, String>` |The key-value pairs that represent your environment's configuration settings.|No

|createFunction |`CamelAwsLambdaEnvironmentTags` |`Map<String, String>` |The list of tags (key-value pairs) assigned to the new function.|No

|createFunction |`CamelAwsLambdaSecurityGroupIds` |`List<String>` |If your Lambda function accesses resources in a VPC, a list of one or more security groups IDs in your VPC.|No

|createFunction |`CamelAwsLambdaSubnetIds` |`List<String>` |If your Lambda function accesses resources in a VPC, a list of one or more subnet IDs in your VPC.|No

|createAlias |`CamelAwsLambdaFunctionVersion` |`String` |The function version to set in the alias|Yes

|createAlias |`CamelAwsLambdaAliasFunctionName` |`String` |The function name to set in the alias|Yes

|createAlias |`CamelAwsLambdaAliasFunctionDescription` |`String` |The function description to set in the alias|No

|deleteAlias |`CamelAwsLambdaAliasFunctionName` |`String` |The function name of the alias|Yes

|getAlias |`CamelAwsLambdaAliasFunctionName` |`String` |The function name of the alias|Yes

|listAliases |`CamelAwsLambdaFunctionVersion` |`String` |The function version to set in the alias|Yes

|=======================================================================

== List of Avalaible Operations

- listFunctions
- getFunction
- createFunction
- deleteFunction
- invokeFunction
- updateFunction
- createEventSourceMapping
- deleteEventSourceMapping
- listEventSourceMapping
- listTags
- tagResource
- untagResource
- publishVersion
- listVersions
- createAlias
- deleteAlias
- getAlias
- listAliases

== Example

To have a full understanding of how the component works, you may have a look at this https://github.com/apache/camel/blob/master/components/camel-aws/src/test/java/org/apache/camel/component/aws/lambda/integration/LambdaComponentIntegrationTest.java[integration test]

== Automatic detection of LambdaClient client in registry

The component is capable of detecting the presence of an LambdaClient bean into the registry.
If it's the only instance of that type it will be used as client and you won't have to define it as uri parameter.
This may be really useful for smarter configuration of the endpoint.

== Dependencies

Maven users will need to add the following dependency to their pom.xml.

*pom.xml*

[source,xml]
---------------------------------------
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-aws2-lambda</artifactId>
    <version>${camel-version}</version>
</dependency>
---------------------------------------

where `$\{camel-version\}` must be replaced by the actual version of Camel.

include::camel-spring-boot::page$aws-lambda-starter.adoc[]

[[aws2-lambda-component]]
= AWS Lambda Component
:page-source: components/camel-aws2-lambda/src/main/docs/aws2-lambda-component.adoc

*Since Camel 3.2*

*Since Camel 3.2*

// HEADER START
*Only producer is supported*
// HEADER END

The Lambda component supports create, get, list, delete and invoke
https://aws.amazon.com/lambda/[AWS Lambda] functions.

*Prerequisites*

You must have a valid Amazon Web Services developer account, and be
signed up to use Amazon Lambda. More information is available at
https://aws.amazon.com/lambda/[AWS Lambda].

When creating a Lambda function, you need to specify a IAM role which has at least the AWSLambdaBasicExecuteRole policy attached.

*Warning*

Lambda is regional service. Unlike S3 bucket, Lambda function created in a given region is not available on other regions.

== URI Format

[source,java]
-------------------------
aws2-lambda://functionName[?options]
-------------------------

You can append query options to the URI in the following format,
?options=value&option2=value&...

== URI Options

// component options: START
The AWS Lambda component supports 6 options, which are listed below.

[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *accessKey* (producer) | Amazon AWS Access Key |  | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean
| *region* (producer) | Amazon AWS Region |  | String
| *secretKey* (producer) | Amazon AWS Secret Key |  | String
| *basicPropertyBinding* (advanced) | Whether the component should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | boolean
| *configuration* (advanced) | The AWS Lambda default configuration |  | Lambda2Configuration
|===
// component options: END

// endpoint options: START
The AWS Lambda endpoint is configured using URI syntax:

----
aws2-lambda:function
----

with the following path and query parameters:

=== Path Parameters (1 parameters):

[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *function* | *Required* Name of the Lambda function. |  | String
|===

=== Query Parameters (11 parameters):

[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean
| *operation* (producer) | The operation to perform. It can be listFunctions, getFunction, createFunction, deleteFunction or invokeFunction. The value can be one of: listFunctions, getFunction, createAlias, deleteAlias, getAlias, listAliases, createFunction, deleteFunction, invokeFunction, updateFunction, createEventSourceMapping, deleteEventSourceMapping, listEventSourceMapping, listTags, tagResource, untagResource, publishVersion, listVersions | invokeFunction | Lambda2Operations
| *region* (producer) | Amazon AWS Region. When using this parameter, the configuration will expect the capitalized name of the region (for example AP_EAST_1) You'll need to use the name Regions.EU_WEST_1.name() |  | String
| *awsLambdaClient* (advanced) | To use a existing configured AwsLambdaClient as client |  | LambdaClient
| *basicPropertyBinding* (advanced) | Whether the endpoint should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | boolean
| *synchronous* (advanced) | Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported). | false | boolean
| *proxyHost* (proxy) | To define a proxy host when instantiating the Lambda client |  | String
| *proxyPort* (proxy) | To define a proxy port when instantiating the Lambda client |  | Integer
| *proxyProtocol* (proxy) | To define a proxy protocol when instantiating the Lambda client. The value can be one of: HTTP, HTTPS | HTTPS | Protocol
| *accessKey* (security) | Amazon AWS Access Key |  | String
| *secretKey* (security) | Amazon AWS Secret Key |  | String
|===
// endpoint options: END

Required Lambda component options

You have to provide the awsLambdaClient in the
Registry or your accessKey and secretKey to access
the https://aws.amazon.com/lambda/[Amazon Lambda] service.

== Usage

=== Message headers evaluated by the Lambda producer

[width="100%",cols="5%,5%,10%,75%,5%",options="header",]
|=======================================================================
|Operation |Header |Type |Description |Required

|All |`CamelAwsLambdaOperation` |`String` |The operation we want to perform. Override operation passed as query parameter| Yes

|createFunction |`CamelAwsLambdaS3Bucket` |`String` |Amazon S3 bucket name where the .zip file containing
your deployment package is stored. This bucket must reside in the same AWS region where you are creating the Lambda function.| No

|createFunction |`CamelAwsLambdaS3Key` |`String` |The Amazon S3 object (the deployment package) key name
you want to upload.| No

|createFunction |`CamelAwsLambdaS3ObjectVersion` |String |The Amazon S3 object (the deployment package) version
you want to upload.| No

|createFunction |`CamelAwsLambdaZipFile` |`String` |The local path of the zip file (the deployment package).
 Content of zip file can also be put in Message body.| No

|createFunction |`CamelAwsLambdaRole` |`String` |The Amazon Resource Name (ARN) of the IAM role that Lambda assumes
 when it executes your function to access any other Amazon Web Services (AWS) resources. |Yes

|createFunction |`CamelAwsLambdaRuntime` |String |The runtime environment for the Lambda function you are uploading.
 (nodejs, nodejs4.3, nodejs6.10, java8, python2.7, python3.6, dotnetcore1.0, odejs4.3-edge) |Yes

|createFunction |`CamelAwsLambdaHandler` |`String` |The function within your code that Lambda calls to begin execution.
 For Node.js, it is the module-name.export value in your function.
 For Java, it can be package.class-name::handler or package.class-name.|Yes

|createFunction |`CamelAwsLambdaDescription` |`String` |The user-provided description.|No

|createFunction |`CamelAwsLambdaTargetArn` |`String` |The parent object that contains the target ARN (Amazon Resource Name)
of an Amazon SQS queue or Amazon SNS topic.|No

|createFunction |`CamelAwsLambdaMemorySize` |`Integer` |The memory size, in MB, you configured for the function.
Must be a multiple of 64 MB.|No

|createFunction |`CamelAwsLambdaKMSKeyArn` |`String` |The Amazon Resource Name (ARN) of the KMS key used to encrypt your function's environment variables.
If not provided, AWS Lambda will use a default service key.|No

|createFunction |`CamelAwsLambdaPublish` |`Boolean` |This boolean parameter can be used to request AWS Lambda
to create the Lambda function and publish a version as an atomic operation.|No

|createFunction |`CamelAwsLambdaTimeout` |`Integer` |The function execution time at which Lambda should terminate the function.
The default is 3 seconds.|No

|createFunction |`CamelAwsLambdaTracingConfig` |`String` |Your function's tracing settings (Active or PassThrough).|No

|createFunction |`CamelAwsLambdaEnvironmentVariables` |`Map<String, String>` |The key-value pairs that represent your environment's configuration settings.|No

|createFunction |`CamelAwsLambdaEnvironmentTags` |`Map<String, String>` |The list of tags (key-value pairs) assigned to the new function.|No

|createFunction |`CamelAwsLambdaSecurityGroupIds` |`List<String>` |If your Lambda function accesses resources in a VPC, a list of one or more security groups IDs in your VPC.|No

|createFunction |`CamelAwsLambdaSubnetIds` |`List<String>` |If your Lambda function accesses resources in a VPC, a list of one or more subnet IDs in your VPC.|No

|createAlias |`CamelAwsLambdaFunctionVersion` |`String` |The function version to set in the alias|Yes

|createAlias |`CamelAwsLambdaAliasFunctionName` |`String` |The function name to set in the alias|Yes

|createAlias |`CamelAwsLambdaAliasFunctionDescription` |`String` |The function description to set in the alias|No

|deleteAlias |`CamelAwsLambdaAliasFunctionName` |`String` |The function name of the alias|Yes

|getAlias |`CamelAwsLambdaAliasFunctionName` |`String` |The function name of the alias|Yes

|listAliases |`CamelAwsLambdaFunctionVersion` |`String` |The function version to set in the alias|Yes

|=======================================================================

== List of Avalaible Operations

- listFunctions
- getFunction
- createFunction
- deleteFunction
- invokeFunction
- updateFunction
- createEventSourceMapping
- deleteEventSourceMapping
- listEventSourceMapping
- listTags
- tagResource
- untagResource
- publishVersion
- listVersions
- createAlias
- deleteAlias
- getAlias
- listAliases

== Example

To have a full understanding of how the component works, you may have a look at this https://github.com/apache/camel/blob/master/components/camel-aws/src/test/java/org/apache/camel/component/aws/lambda/integration/LambdaComponentIntegrationTest.java[integration test]

== Automatic detection of LambdaClient client in registry

The component is capable of detecting the presence of an LambdaClient bean into the registry.
If it's the only instance of that type it will be used as client and you won't have to define it as uri parameter.
This may be really useful for smarter configuration of the endpoint.

== Dependencies

Maven users will need to add the following dependency to their pom.xml.

*pom.xml*

[source,xml]
---------------------------------------
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-aws2-lambda</artifactId>
    <version>${camel-version}</version>
</dependency>
---------------------------------------

where `$\{camel-version\}` must be replaced by the actual version of Camel.

include::camel-spring-boot::page$aws-lambda-starter.adoc[]


fix no=>not typo (#32463)

fix no=>not typo (#32463)

Found a tiny typo while reading the docs

[[search-aggregations-bucket-terms-aggregation]]
=== Terms Aggregation

A multi-bucket value source based aggregation where buckets are dynamically built - one per unique value.

//////////////////////////

[source,js]
--------------------------------------------------
PUT /products
{
    "mappings": {
        "product": {
            "properties": {
                "genre": {
                    "type": "keyword"
                },
                "product": {
                    "type": "keyword"
                }
            }
        }
    }
}

POST /products/product/_bulk?refresh
{"index":{"_id":0}}
{"genre": "rock", "product": "Product A"}
{"index":{"_id":1}}
{"genre": "rock"}
{"index":{"_id":2}}
{"genre": "rock"}
{"index":{"_id":3}}
{"genre": "jazz", "product": "Product Z"}
{"index":{"_id":4}}
{"genre": "jazz"}
{"index":{"_id":5}}
{"genre": "electronic"}
{"index":{"_id":6}}
{"genre": "electronic"}
{"index":{"_id":7}}
{"genre": "electronic"}
{"index":{"_id":8}}
{"genre": "electronic"}
{"index":{"_id":9}}
{"genre": "electronic"}
{"index":{"_id":10}}
{"genre": "electronic"}

-------------------------------------------------
// NOTCONSOLE
// TESTSETUP

//////////////////////////

Example:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : { "field" : "genre" }
        }
    }
}
--------------------------------------------------
// CONSOLE
// TEST[s/_search/_search\?filter_path=aggregations/]

Response:

[source,js]
--------------------------------------------------
{
    ...
    "aggregations" : {
        "genres" : {
            "doc_count_error_upper_bound": 0, <1>
            "sum_other_doc_count": 0, <2>
            "buckets" : [ <3>
                {
                    "key" : "electronic",
                    "doc_count" : 6
                },
                {
                    "key" : "rock",
                    "doc_count" : 3
                },
                {
                    "key" : "jazz",
                    "doc_count" : 2
                }
            ]
        }
    }
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\.//]
<1> an upper bound of the error on the document counts for each term, see <<search-aggregations-bucket-terms-aggregation-approximate-counts,below>>
<2> when there are lots of unique terms, Elasticsearch only returns the top terms; this number is the sum of the document counts for all buckets that are not part of the response
<3> the list of the top buckets, the meaning of `top` being defined by the <<search-aggregations-bucket-terms-aggregation-order,order>>

By default, the `terms` aggregation will return the buckets for the top ten terms ordered by the `doc_count`. One can
change this default behaviour by setting the `size` parameter.

[[search-aggregations-bucket-terms-aggregation-size]]
==== Size

The `size` parameter can be set to define how many term buckets should be returned out of the overall terms list. By
default, the node coordinating the search process will request each shard to provide its own top `size` term buckets
and once all shards respond, it will reduce the results to the final list that will then be returned to the client.
This means that if the number of unique terms is greater than `size`, the returned list is slightly off and not accurate
(it could be that the term counts are slightly off and it could even be that a term that should have been in the top
size buckets was not returned).

NOTE: If you want to retrieve **all** terms or all combinations of terms in a nested `terms` aggregation
      you should use the <<search-aggregations-bucket-composite-aggregation,Composite>> aggregation which
      allows to paginate over all possible terms rather than setting a size greater than the cardinality of the field in the
      `terms` aggregation. The `terms` aggregation is meant to return the `top` terms and does not allow pagination.

[[search-aggregations-bucket-terms-aggregation-approximate-counts]]
==== Document counts are approximate

As described above, the document counts (and the results of any sub aggregations) in the terms aggregation are not always
accurate.  This is because each shard provides its own view of what the ordered list of terms should be and these are
combined to give a final view. Consider the following scenario:

A request is made to obtain the top 5 terms in the field product, ordered by descending document count from an index with
3 shards. In this case each shard is asked to give its top 5 terms.

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "products" : {
            "terms" : {
                "field" : "product",
                "size" : 5
            }
        }
    }
}
--------------------------------------------------
// CONSOLE
// TEST[s/_search/_search\?filter_path=aggregations/]

The terms for each of the three shards are shown below with their
respective document counts in brackets:

[width="100%",cols="^2,^2,^2,^2",options="header"]
|=========================================================
|    | Shard A        | Shard B        | Shard C

| 1  | Product A (25) | Product A (30) | Product A (45)
| 2  | Product B (18) | Product B (25) | Product C (44)
| 3  | Product C (6)  | Product F (17) | Product Z (36)
| 4  | Product D (3)  | Product Z (16) | Product G (30)
| 5  | Product E (2)  | Product G (15) | Product E (29)
| 6  | Product F (2)  | Product H (14) | Product H (28)
| 7  | Product G (2)  | Product I (10) | Product Q (2)
| 8  | Product H (2)  | Product Q (6)  | Product D (1)
| 9  | Product I (1)  | Product J (8)  |
| 10 | Product J (1)  | Product C (4)  |

|=========================================================

The shards will return their top 5 terms so the results from the shards will be:

[width="100%",cols="^2,^2,^2,^2",options="header"]
|=========================================================
|    | Shard A        | Shard B        | Shard C

| 1  | Product A (25) | Product A (30) | Product A (45)
| 2  | Product B (18) | Product B (25) | Product C (44)
| 3  | Product C (6)  | Product F (17) | Product Z (36)
| 4  | Product D (3)  | Product Z (16) | Product G (30)
| 5  | Product E (2)  | Product G (15) | Product E (29)

|=========================================================

Taking the top 5 results from each of the shards (as requested) and combining them to make a final top 5 list produces
the following:

[width="40%",cols="^2,^2"]
|=========================================================

| 1  | Product A (100)
| 2  | Product Z (52)
| 3  | Product C (50)
| 4  | Product G (45)
| 5  | Product B (43)

|=========================================================

Because Product A was returned from all shards we know that its document count value is accurate. Product C was only
returned by shards A and C so its document count is shown as 50 but this is not an accurate count. Product C exists on
shard B, but its count of 4 was not high enough to put Product C into the top 5 list for that shard. Product Z was also
returned only by 2 shards but the third shard does not contain the term. There is no way of knowing, at the point of
combining the results to produce the final list of terms, that there is an error in the document count for Product C and
not for Product Z. Product H has a document count of 44 across all 3 shards but was not included in the final list of
terms because it did not make it into the top five terms on any of the shards.

==== Shard Size

The higher the requested `size` is, the more accurate the results will be, but also, the more expensive it will be to
compute the final results (both due to bigger priority queues that are managed on a shard level and due to bigger data
transfers between the nodes and the client).

The `shard_size` parameter can be  used to minimize the extra work that comes with bigger requested `size`. When defined,
it will determine how many terms the coordinating node will request from each shard. Once all the shards responded, the
coordinating node will then reduce them to a final result which will be based on the `size` parameter - this way,
one can increase the accuracy of the returned terms and avoid the overhead of streaming a big list of buckets back to
the client.

NOTE:   `shard_size` cannot be smaller than `size` (as it doesn't make much sense). When it is, Elasticsearch will
        override it and reset it to be equal to `size`.

The default `shard_size` will be `size` if the search request needs to go to a single shard, and `(size * 1.5 + 10)`
otherwise.

==== Calculating Document Count Error

There are two error values which can be shown on the terms aggregation. The first gives a value for the aggregation as
a whole which represents the maximum potential document count for a term which did not make it into the final list of
terms. This is calculated as the sum of the document count from the last term returned from each shard. For the example
given above the value would be 46 (2 + 15 + 29). This means that in the worst case scenario a term which was not returned
could have the 4th highest document count.

[source,js]
--------------------------------------------------
{
    ...
    "aggregations" : {
        "products" : {
            "doc_count_error_upper_bound" : 46,
            "sum_other_doc_count" : 79,
            "buckets" : [
                {
                    "key" : "Product A",
                    "doc_count" : 100
                },
                {
                    "key" : "Product Z",
                    "doc_count" : 52
                }
                ...
            ]
        }
    }
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\.//]
// TESTRESPONSE[s/: (\-)?[0-9]+/: $body.$_path/]

==== Per bucket document count error

The second error value can be enabled by setting the `show_term_doc_count_error` parameter to true:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "products" : {
            "terms" : {
                "field" : "product",
                "size" : 5,
                "show_term_doc_count_error": true
            }
        }
    }
}
--------------------------------------------------
// CONSOLE
// TEST[s/_search/_search\?filter_path=aggregations/]

This shows an error value for each term returned by the aggregation which represents the 'worst case' error in the document count
and can be useful when deciding on a value for the `shard_size` parameter. This is calculated by summing the document counts for
the last term returned by all shards which did not return the term. In the example above the error in the document count for Product C
would be 15 as Shard B was the only shard not to return the term and the document count of the last term it did return was 15.
The actual document count of Product C was 54 so the document count was only actually off by 4 even though the worst case was that
it would be off by 15. Product A, however has an error of 0 for its document count, since every shard returned it we can be confident
that the count returned is accurate.

[source,js]
--------------------------------------------------
{
    ...
    "aggregations" : {
        "products" : {
            "doc_count_error_upper_bound" : 46,
            "sum_other_doc_count" : 79,
            "buckets" : [
                {
                    "key" : "Product A",
                    "doc_count" : 100,
                    "doc_count_error_upper_bound" : 0
                },
                {
                    "key" : "Product Z",
                    "doc_count" : 52,
                    "doc_count_error_upper_bound" : 2
                }
                ...
            ]
        }
    }
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\.//]
// TESTRESPONSE[s/: (\-)?[0-9]+/: $body.$_path/]

These errors can only be calculated in this way when the terms are ordered by descending document count. When the aggregation is
ordered by the terms values themselves (either ascending or descending) there is no error in the document count since if a shard
does not return a particular term which appears in the results from another shard, it must not have that term in its index. When the
aggregation is either sorted by a sub aggregation or in order of ascending document count, the error in the document counts cannot be
determined and is given a value of -1 to indicate this.

[[search-aggregations-bucket-terms-aggregation-order]]
==== Order

The order of the buckets can be customized by setting the `order` parameter. By default, the buckets are ordered by
their `doc_count` descending.  It is possible to change this behaviour as documented below:

WARNING: Sorting by ascending `_count` or by sub aggregation is discouraged as it increases the
<<search-aggregations-bucket-terms-aggregation-approximate-counts,error>> on document counts.
It is fine when a single shard is queried, or when the field that is being aggregated was used
as a routing key at index time: in these cases results will be accurate since shards have disjoint
values. However otherwise, errors are unbounded. One particular case that could still be useful
is sorting by <<search-aggregations-metrics-min-aggregation,`min`>> or
<<search-aggregations-metrics-max-aggregation,`max`>> aggregation: counts will not be accurate
but at least the top buckets will be correctly picked.

Ordering the buckets by their doc `_count` in an ascending manner:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : {
                "field" : "genre",
                "order" : { "_count" : "asc" }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

Ordering the buckets alphabetically by their terms in an ascending manner:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : {
                "field" : "genre",
                "order" : { "_key" : "asc" }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

deprecated[6.0.0, Use `_key` instead of `_term` to order buckets by their term]

Ordering the buckets by single value metrics sub-aggregation (identified by the aggregation name):

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : {
                "field" : "genre",
                "order" : { "max_play_count" : "desc" }
            },
            "aggs" : {
                "max_play_count" : { "max" : { "field" : "play_count" } }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

Ordering the buckets by multi value metrics sub-aggregation (identified by the aggregation name):

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : {
                "field" : "genre",
                "order" : { "playback_stats.max" : "desc" }
            },
            "aggs" : {
                "playback_stats" : { "stats" : { "field" : "play_count" } }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

[NOTE]
.Pipeline aggs cannot be used for sorting
=======================================

<<search-aggregations-pipeline,Pipeline aggregations>> are run during the
reduce phase after all other aggregations have already completed.  For this
reason, they cannot be used for ordering.

=======================================

It is also possible to order the buckets based on a "deeper" aggregation in the hierarchy. This is supported as long
as the aggregations path are of a single-bucket type, where the last aggregation in the path may either be a single-bucket
one or a metrics one. If it's a single-bucket type, the order will be defined by the number of docs in the bucket (i.e. `doc_count`),
in case it's a metrics one, the same rules as above apply (where the path must indicate the metric name to sort by in case of
a multi-value metrics aggregation, and in case of a single-value metrics aggregation the sort will be applied on that value).

The path must be defined in the following form:

// https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_Form
[source,ebnf]
--------------------------------------------------
AGG_SEPARATOR       =  '>' ;
METRIC_SEPARATOR    =  '.' ;
AGG_NAME            =  <the name of the aggregation> ;
METRIC              =  <the name of the metric (in case of multi-value metrics aggregation)> ;
PATH                =  <AGG_NAME> [ <AGG_SEPARATOR>, <AGG_NAME> ]* [ <METRIC_SEPARATOR>, <METRIC> ] ;
--------------------------------------------------

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "countries" : {
            "terms" : {
                "field" : "artist.country",
                "order" : { "rock>playback_stats.avg" : "desc" }
            },
            "aggs" : {
                "rock" : {
                    "filter" : { "term" : { "genre" :  "rock" }},
                    "aggs" : {
                        "playback_stats" : { "stats" : { "field" : "play_count" }}
                    }
                }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

The above will sort the artist's countries buckets based on the average play count among the rock songs.

Multiple criteria can be used to order the buckets by providing an array of order criteria such as the following:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "countries" : {
            "terms" : {
                "field" : "artist.country",
                "order" : [ { "rock>playback_stats.avg" : "desc" }, { "_count" : "desc" } ]
            },
            "aggs" : {
                "rock" : {
                    "filter" : { "term" : { "genre" : "rock" }},
                    "aggs" : {
                        "playback_stats" : { "stats" : { "field" : "play_count" }}
                    }
                }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

The above will sort the artist's countries buckets based on the average play count among the rock songs and then by
their `doc_count` in descending order.

NOTE: In the event that two buckets share the same values for all order criteria the bucket's term value is used as a
tie-breaker in ascending alphabetical order to prevent non-deterministic ordering of buckets.

==== Minimum document count

It is possible to only return terms that match more than a configured number of hits using the `min_doc_count` option:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "tags" : {
            "terms" : {
                "field" : "tags",
                "min_doc_count": 10
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

The above aggregation would only return tags which have been found in 10 hits or more. Default value is `1`.

Terms are collected and ordered on a shard level and merged with the terms collected from other shards in a second step. However, the shard does not have the information about the global document count available. The decision if a term is added to a candidate list depends only on the order computed on the shard using local shard frequencies. The `min_doc_count` criterion is only applied after merging local terms statistics of all shards. In a way the decision to add the term as a candidate is made without being very _certain_ about if the term will actually reach the required `min_doc_count`. This might cause many (globally) high frequent terms to be missing in the final result if low frequent terms populated the candidate lists. To avoid this, the `shard_size` parameter can be increased to allow more candidate terms on the shards. However, this increases memory consumption and network traffic.

`shard_min_doc_count` parameter

The parameter `shard_min_doc_count` regulates the _certainty_ a shard has if the term should actually be added to the candidate list or not with respect to the `min_doc_count`. Terms will only be considered if their local shard frequency within the set is higher than the `shard_min_doc_count`. If your dictionary contains many low frequent terms and you are not interested in those (for example misspellings), then you can set the `shard_min_doc_count` parameter to filter out candidate terms on a shard level that will with a reasonable certainty not reach the required `min_doc_count` even after merging the local counts. `shard_min_doc_count` is set to `0` per default and has no effect unless you explicitly set it.

NOTE:    Setting `min_doc_count`=`0` will also return buckets for terms that didn't match any hit. However, some of
         the returned terms which have a document count of zero might only belong to deleted documents or documents
         from other types, so there is no warranty that a `match_all` query would find a positive document count for
         those terms.

WARNING: When NOT sorting on `doc_count` descending, high values of `min_doc_count` may return a number of buckets
         which is less than `size` because not enough data was gathered from the shards. Missing buckets can be
         back by increasing `shard_size`.
         Setting `shard_min_doc_count` too high will cause terms to be filtered out on a shard level. This value should be set much lower than `min_doc_count/#shards`.

[[search-aggregations-bucket-terms-aggregation-script]]
==== Script

Generating the terms using a script:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : {
                "script" : {
                    "source": "doc['genre'].value",
                    "lang": "painless"
                }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

This will interpret the `script` parameter as an `inline` script with the default script language and no script parameters. To use a stored script use the following syntax:

//////////////////////////

[source,js]
--------------------------------------------------
POST /_scripts/my_script
{
  "script": {
    "lang": "painless",
    "source": "doc[params.field].value"
  }
}
--------------------------------------------------
// CONSOLE

//////////////////////////

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : {
                "script" : {
                    "id": "my_script",
                    "params": {
                        "field": "genre"
                    }
                }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE
// TEST[continued]

==== Value Script

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : {
                "field" : "gender",
                "script" : {
                    "source" : "'Genre: ' +_value",
                    "lang" : "painless"
                }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

==== Filtering Values

It is possible to filter the values for which buckets will be created. This can be done using the `include` and
`exclude` parameters which are based on regular expression strings or arrays of exact values. Additionally,
`include` clauses can filter using `partition` expressions.

===== Filtering Values with regular expressions

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "tags" : {
            "terms" : {
                "field" : "tags",
                "include" : ".*sport.*",
                "exclude" : "water_.*"
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

In the above example, buckets will be created for all the tags that has the word `sport` in them, except those starting
with `water_` (so the tag `water_sports` will not be aggregated). The `include` regular expression will determine what
values are "allowed" to be aggregated, while the `exclude` determines the values that should not be aggregated. When
both are defined, the `exclude` has precedence, meaning, the `include` is evaluated first and only then the `exclude`.

The syntax is the same as <<regexp-syntax,regexp queries>>.

===== Filtering Values with exact values

For matching based on exact values the `include` and `exclude` parameters can simply take an array of
strings that represent the terms as they are found in the index:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "JapaneseCars" : {
             "terms" : {
                 "field" : "make",
                 "include" : ["mazda", "honda"]
             }
         },
        "ActiveCarManufacturers" : {
             "terms" : {
                 "field" : "make",
                 "exclude" : ["rover", "jensen"]
             }
         }
    }
}
--------------------------------------------------
// CONSOLE

===== Filtering Values with partitions

Sometimes there are too many unique terms to process in a single request/response pair so
it can be useful to break the analysis up into multiple requests.
This can be achieved by grouping the field's values into a number of partitions at query-time and processing
only one partition in each request.
Consider this request which is looking for accounts that have not logged any access recently:

[source,js]
--------------------------------------------------
GET /_search
{
   "size": 0,
   "aggs": {
      "expired_sessions": {
         "terms": {
            "field": "account_id",
            "include": {
               "partition": 0,
               "num_partitions": 20
            },
            "size": 10000,
            "order": {
               "last_access": "asc"
            }
         },
         "aggs": {
            "last_access": {
               "max": {
                  "field": "access_date"
               }
            }
         }
      }
   }
}
--------------------------------------------------
// CONSOLE

This request is finding the last logged access date for a subset of customer accounts because we
might want to expire some customer accounts who haven't been seen for a long while.
The `num_partitions` setting has requested that the unique account_ids are organized evenly into twenty
partitions (0 to 19). and the `partition` setting in this request filters to only consider account_ids falling
into partition 0. Subsequent requests should ask for partitions 1 then 2 etc to complete the expired-account analysis.

Note that the `size` setting for the number of results returned needs to be tuned with the `num_partitions`.
For this particular account-expiration example the process for balancing values for `size` and `num_partitions` would be as follows:

1. Use the `cardinality` aggregation to estimate the total number of unique account_id values
2. Pick a value for `num_partitions` to break the number from 1) up into more manageable chunks
3. Pick a `size` value for the number of responses we want from each partition
4. Run a test request

If we have a circuit-breaker error we are trying to do too much in one request and must increase `num_partitions`.
If the request was successful but the last account ID in the date-sorted test response was still an account we might want to
expire then we may be missing accounts of interest and have set our numbers too low. We must either

* increase the `size` parameter to return more results per partition (could be heavy on memory) or
* increase the `num_partitions` to consider less accounts per request (could increase overall processing time as we need to make more requests)

Ultimately this is a balancing act between managing the Elasticsearch resources required to process a single request and the volume
of requests that the client application must issue to complete a task.

==== Multi-field terms aggregation

The `terms` aggregation does not support collecting terms from multiple fields
in the same document.  The reason is that the `terms` agg doesn't collect the
string term values themselves, but rather uses
<<search-aggregations-bucket-terms-aggregation-execution-hint,global ordinals>>
to produce a list of all of the unique values in the field.  Global ordinals
results in an important performance boost which would not be possible across
multiple fields.

There are two approaches that you can use to perform a `terms` agg across
multiple fields:

<<search-aggregations-bucket-terms-aggregation-script,Script>>::

Use a script to retrieve terms from multiple fields.  This disables the global
ordinals optimization and will be slower than collecting terms from a single
field, but it gives you the flexibility to implement this option at search
time.

<<copy-to,`copy_to` field>>::

If you know ahead of time that you want to collect the terms from two or more
fields, then use `copy_to` in your mapping to create a new dedicated field at
index time which contains the values from both fields.  You can aggregate on
this single field, which will benefit from the global ordinals optimization.

==== Collect mode

Deferring calculation of child aggregations

For fields with many unique terms and a small number of required results it can be more efficient to delay the calculation
of child aggregations until the top parent-level aggs have been pruned. Ordinarily, all branches of the aggregation tree
are expanded in one depth-first pass and only then any pruning occurs.
In some scenarios this can be very wasteful and can hit memory constraints.
An example problem scenario is querying a movie database for the 10 most popular actors and their 5 most common co-stars:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "actors" : {
             "terms" : {
                 "field" : "actors",
                 "size" : 10
             },
            "aggs" : {
                "costars" : {
                     "terms" : {
                         "field" : "actors",
                         "size" : 5
                     }
                 }
            }
         }
    }
}
--------------------------------------------------
// CONSOLE

Even though the number of actors may be comparatively small and we want only 50 result buckets there is a combinatorial explosion of buckets
during calculation - a single actor can produce n² buckets where n is the number of actors. The sane option would be to first determine
the 10 most popular actors and only then examine the top co-stars for these 10 actors. This alternative strategy is what we call the `breadth_first` collection
mode as opposed to the `depth_first` mode.

NOTE: The `breadth_first` is the default mode for fields with a cardinality bigger than the requested size or when the cardinality is unknown (numeric fields or scripts for instance).
It is possible to override the default heuristic and to provide a collect mode directly in the request:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "actors" : {
             "terms" : {
                 "field" : "actors",
                 "size" : 10,
                 "collect_mode" : "breadth_first" <1>
             },
            "aggs" : {
                "costars" : {
                     "terms" : {
                         "field" : "actors",
                         "size" : 5
                     }
                 }
            }
         }
    }
}
--------------------------------------------------
// CONSOLE

<1> the possible values are `breadth_first` and `depth_first`

When using `breadth_first` mode the set of documents that fall into the uppermost buckets are
cached for subsequent replay so there is a memory overhead in doing this which is linear with the number of matching documents.
Note that the `order` parameter can still be used to refer to data from a child aggregation when using the `breadth_first` setting - the parent
aggregation understands that this child aggregation will need to be called first before any of the other child aggregations.

WARNING: Nested aggregations such as `top_hits` which require access to score information under an aggregation that uses the `breadth_first`
collection mode need to replay the query on the second pass but only for the documents belonging to the top buckets.

[[search-aggregations-bucket-terms-aggregation-execution-hint]]
==== Execution hint

There are different mechanisms by which terms aggregations can be executed:

 - by using field values directly in order to aggregate data per-bucket (`map`)
 - by using global ordinals of the field and allocating one bucket per global ordinal (`global_ordinals`)

Elasticsearch tries to have sensible defaults so this is something that generally doesn't need to be configured.

`global_ordinals` is the default option for `keyword` field, it uses global ordinals to allocates buckets dynamically
so memory usage is linear to the number of values of the documents that are part of the aggregation scope.

`map` should only be considered when very few documents match a query. Otherwise the ordinals-based execution mode
is significantly faster. By default, `map` is only used when running an aggregation on scripts, since they don't have
ordinals.

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "tags" : {
             "terms" : {
                 "field" : "tags",
                 "execution_hint": "map" <1>
             }
         }
    }
}
--------------------------------------------------
// CONSOLE

<1> The possible values are `map`, `global_ordinals`

Please note that Elasticsearch will ignore this execution hint if it is not applicable and that there is no backward compatibility guarantee on these hints.

==== Missing value

The `missing` parameter defines how documents that are missing a value should be treated.
By default they will be ignored but it is also possible to treat them as if they
had a value.

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "tags" : {
             "terms" : {
                 "field" : "tags",
                 "missing": "N/A" <1>
             }
         }
    }
}
--------------------------------------------------
// CONSOLE

<1> Documents without a value in the `tags` field will fall into the same bucket as documents that have the value `N/A`.

==== Mixing field types

WARNING: When aggregating on multiple indices the type of the aggregated field may not be the same in all indices.
Some types are compatible with each other (`integer` and `long` or `float` and `double`) but when the types are a mix
of decimal and non-decimal number the terms aggregation will promote the non-decimal numbers to decimal numbers.
This can result in a loss of precision in the bucket values.

[[search-aggregations-bucket-terms-aggregation]]
=== Terms Aggregation

A multi-bucket value source based aggregation where buckets are dynamically built - one per unique value.

//////////////////////////

[source,js]
--------------------------------------------------
PUT /products
{
    "mappings": {
        "product": {
            "properties": {
                "genre": {
                    "type": "keyword"
                },
                "product": {
                    "type": "keyword"
                }
            }
        }
    }
}

POST /products/product/_bulk?refresh
{"index":{"_id":0}}
{"genre": "rock", "product": "Product A"}
{"index":{"_id":1}}
{"genre": "rock"}
{"index":{"_id":2}}
{"genre": "rock"}
{"index":{"_id":3}}
{"genre": "jazz", "product": "Product Z"}
{"index":{"_id":4}}
{"genre": "jazz"}
{"index":{"_id":5}}
{"genre": "electronic"}
{"index":{"_id":6}}
{"genre": "electronic"}
{"index":{"_id":7}}
{"genre": "electronic"}
{"index":{"_id":8}}
{"genre": "electronic"}
{"index":{"_id":9}}
{"genre": "electronic"}
{"index":{"_id":10}}
{"genre": "electronic"}

-------------------------------------------------
// NOTCONSOLE
// TESTSETUP

//////////////////////////

Example:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : { "field" : "genre" }
        }
    }
}
--------------------------------------------------
// CONSOLE
// TEST[s/_search/_search\?filter_path=aggregations/]

Response:

[source,js]
--------------------------------------------------
{
    ...
    "aggregations" : {
        "genres" : {
            "doc_count_error_upper_bound": 0, <1>
            "sum_other_doc_count": 0, <2>
            "buckets" : [ <3>
                {
                    "key" : "electronic",
                    "doc_count" : 6
                },
                {
                    "key" : "rock",
                    "doc_count" : 3
                },
                {
                    "key" : "jazz",
                    "doc_count" : 2
                }
            ]
        }
    }
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\.//]
<1> an upper bound of the error on the document counts for each term, see <<search-aggregations-bucket-terms-aggregation-approximate-counts,below>>
<2> when there are lots of unique terms, Elasticsearch only returns the top terms; this number is the sum of the document counts for all buckets that are not part of the response
<3> the list of the top buckets, the meaning of `top` being defined by the <<search-aggregations-bucket-terms-aggregation-order,order>>

By default, the `terms` aggregation will return the buckets for the top ten terms ordered by the `doc_count`. One can
change this default behaviour by setting the `size` parameter.

[[search-aggregations-bucket-terms-aggregation-size]]
==== Size

The `size` parameter can be set to define how many term buckets should be returned out of the overall terms list. By
default, the node coordinating the search process will request each shard to provide its own top `size` term buckets
and once all shards respond, it will reduce the results to the final list that will then be returned to the client.
This means that if the number of unique terms is greater than `size`, the returned list is slightly off and not accurate
(it could be that the term counts are slightly off and it could even be that a term that should have been in the top
size buckets was not returned).

NOTE: If you want to retrieve **all** terms or all combinations of terms in a nested `terms` aggregation
      you should use the <<search-aggregations-bucket-composite-aggregation,Composite>> aggregation which
      allows to paginate over all possible terms rather than setting a size greater than the cardinality of the field in the
      `terms` aggregation. The `terms` aggregation is meant to return the `top` terms and does not allow pagination.

[[search-aggregations-bucket-terms-aggregation-approximate-counts]]
==== Document counts are approximate

As described above, the document counts (and the results of any sub aggregations) in the terms aggregation are not always
accurate.  This is because each shard provides its own view of what the ordered list of terms should be and these are
combined to give a final view. Consider the following scenario:

A request is made to obtain the top 5 terms in the field product, ordered by descending document count from an index with
3 shards. In this case each shard is asked to give its top 5 terms.

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "products" : {
            "terms" : {
                "field" : "product",
                "size" : 5
            }
        }
    }
}
--------------------------------------------------
// CONSOLE
// TEST[s/_search/_search\?filter_path=aggregations/]

The terms for each of the three shards are shown below with their
respective document counts in brackets:

[width="100%",cols="^2,^2,^2,^2",options="header"]
|=========================================================
|    | Shard A        | Shard B        | Shard C

| 1  | Product A (25) | Product A (30) | Product A (45)
| 2  | Product B (18) | Product B (25) | Product C (44)
| 3  | Product C (6)  | Product F (17) | Product Z (36)
| 4  | Product D (3)  | Product Z (16) | Product G (30)
| 5  | Product E (2)  | Product G (15) | Product E (29)
| 6  | Product F (2)  | Product H (14) | Product H (28)
| 7  | Product G (2)  | Product I (10) | Product Q (2)
| 8  | Product H (2)  | Product Q (6)  | Product D (1)
| 9  | Product I (1)  | Product J (8)  |
| 10 | Product J (1)  | Product C (4)  |

|=========================================================

The shards will return their top 5 terms so the results from the shards will be:

[width="100%",cols="^2,^2,^2,^2",options="header"]
|=========================================================
|    | Shard A        | Shard B        | Shard C

| 1  | Product A (25) | Product A (30) | Product A (45)
| 2  | Product B (18) | Product B (25) | Product C (44)
| 3  | Product C (6)  | Product F (17) | Product Z (36)
| 4  | Product D (3)  | Product Z (16) | Product G (30)
| 5  | Product E (2)  | Product G (15) | Product E (29)

|=========================================================

Taking the top 5 results from each of the shards (as requested) and combining them to make a final top 5 list produces
the following:

[width="40%",cols="^2,^2"]
|=========================================================

| 1  | Product A (100)
| 2  | Product Z (52)
| 3  | Product C (50)
| 4  | Product G (45)
| 5  | Product B (43)

|=========================================================

Because Product A was returned from all shards we know that its document count value is accurate. Product C was only
returned by shards A and C so its document count is shown as 50 but this is not an accurate count. Product C exists on
shard B, but its count of 4 was not high enough to put Product C into the top 5 list for that shard. Product Z was also
returned only by 2 shards but the third shard does not contain the term. There is no way of knowing, at the point of
combining the results to produce the final list of terms, that there is an error in the document count for Product C and
not for Product Z. Product H has a document count of 44 across all 3 shards but was not included in the final list of
terms because it did not make it into the top five terms on any of the shards.

==== Shard Size

The higher the requested `size` is, the more accurate the results will be, but also, the more expensive it will be to
compute the final results (both due to bigger priority queues that are managed on a shard level and due to bigger data
transfers between the nodes and the client).

The `shard_size` parameter can be  used to minimize the extra work that comes with bigger requested `size`. When defined,
it will determine how many terms the coordinating node will request from each shard. Once all the shards responded, the
coordinating node will then reduce them to a final result which will be based on the `size` parameter - this way,
one can increase the accuracy of the returned terms and avoid the overhead of streaming a big list of buckets back to
the client.

NOTE:   `shard_size` cannot be smaller than `size` (as it doesn't make much sense). When it is, Elasticsearch will
        override it and reset it to be equal to `size`.

The default `shard_size` will be `size` if the search request needs to go to a single shard, and `(size * 1.5 + 10)`
otherwise.

==== Calculating Document Count Error

There are two error values which can be shown on the terms aggregation. The first gives a value for the aggregation as
a whole which represents the maximum potential document count for a term which did not make it into the final list of
terms. This is calculated as the sum of the document count from the last term returned from each shard. For the example
given above the value would be 46 (2 + 15 + 29). This means that in the worst case scenario a term which was not returned
could have the 4th highest document count.

[source,js]
--------------------------------------------------
{
    ...
    "aggregations" : {
        "products" : {
            "doc_count_error_upper_bound" : 46,
            "sum_other_doc_count" : 79,
            "buckets" : [
                {
                    "key" : "Product A",
                    "doc_count" : 100
                },
                {
                    "key" : "Product Z",
                    "doc_count" : 52
                }
                ...
            ]
        }
    }
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\.//]
// TESTRESPONSE[s/: (\-)?[0-9]+/: $body.$_path/]

==== Per bucket document count error

The second error value can be enabled by setting the `show_term_doc_count_error` parameter to true:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "products" : {
            "terms" : {
                "field" : "product",
                "size" : 5,
                "show_term_doc_count_error": true
            }
        }
    }
}
--------------------------------------------------
// CONSOLE
// TEST[s/_search/_search\?filter_path=aggregations/]

This shows an error value for each term returned by the aggregation which represents the 'worst case' error in the document count
and can be useful when deciding on a value for the `shard_size` parameter. This is calculated by summing the document counts for
the last term returned by all shards which did not return the term. In the example above the error in the document count for Product C
would be 15 as Shard B was the only shard not to return the term and the document count of the last term it did return was 15.
The actual document count of Product C was 54 so the document count was only actually off by 4 even though the worst case was that
it would be off by 15. Product A, however has an error of 0 for its document count, since every shard returned it we can be confident
that the count returned is accurate.

[source,js]
--------------------------------------------------
{
    ...
    "aggregations" : {
        "products" : {
            "doc_count_error_upper_bound" : 46,
            "sum_other_doc_count" : 79,
            "buckets" : [
                {
                    "key" : "Product A",
                    "doc_count" : 100,
                    "doc_count_error_upper_bound" : 0
                },
                {
                    "key" : "Product Z",
                    "doc_count" : 52,
                    "doc_count_error_upper_bound" : 2
                }
                ...
            ]
        }
    }
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\.//]
// TESTRESPONSE[s/: (\-)?[0-9]+/: $body.$_path/]

These errors can only be calculated in this way when the terms are ordered by descending document count. When the aggregation is
ordered by the terms values themselves (either ascending or descending) there is no error in the document count since if a shard
does not return a particular term which appears in the results from another shard, it must not have that term in its index. When the
aggregation is either sorted by a sub aggregation or in order of ascending document count, the error in the document counts cannot be
determined and is given a value of -1 to indicate this.

[[search-aggregations-bucket-terms-aggregation-order]]
==== Order

The order of the buckets can be customized by setting the `order` parameter. By default, the buckets are ordered by
their `doc_count` descending.  It is possible to change this behaviour as documented below:

WARNING: Sorting by ascending `_count` or by sub aggregation is discouraged as it increases the
<<search-aggregations-bucket-terms-aggregation-approximate-counts,error>> on document counts.
It is fine when a single shard is queried, or when the field that is being aggregated was used
as a routing key at index time: in these cases results will be accurate since shards have disjoint
values. However otherwise, errors are unbounded. One particular case that could still be useful
is sorting by <<search-aggregations-metrics-min-aggregation,`min`>> or
<<search-aggregations-metrics-max-aggregation,`max`>> aggregation: counts will not be accurate
but at least the top buckets will be correctly picked.

Ordering the buckets by their doc `_count` in an ascending manner:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : {
                "field" : "genre",
                "order" : { "_count" : "asc" }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

Ordering the buckets alphabetically by their terms in an ascending manner:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : {
                "field" : "genre",
                "order" : { "_key" : "asc" }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

deprecated[6.0.0, Use `_key` instead of `_term` to order buckets by their term]

Ordering the buckets by single value metrics sub-aggregation (identified by the aggregation name):

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : {
                "field" : "genre",
                "order" : { "max_play_count" : "desc" }
            },
            "aggs" : {
                "max_play_count" : { "max" : { "field" : "play_count" } }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

Ordering the buckets by multi value metrics sub-aggregation (identified by the aggregation name):

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : {
                "field" : "genre",
                "order" : { "playback_stats.max" : "desc" }
            },
            "aggs" : {
                "playback_stats" : { "stats" : { "field" : "play_count" } }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

[NOTE]
.Pipeline aggs cannot be used for sorting
=======================================

<<search-aggregations-pipeline,Pipeline aggregations>> are run during the
reduce phase after all other aggregations have already completed.  For this
reason, they cannot be used for ordering.

=======================================

It is also possible to order the buckets based on a "deeper" aggregation in the hierarchy. This is supported as long
as the aggregations path are of a single-bucket type, where the last aggregation in the path may either be a single-bucket
one or a metrics one. If it's a single-bucket type, the order will be defined by the number of docs in the bucket (i.e. `doc_count`),
in case it's a metrics one, the same rules as above apply (where the path must indicate the metric name to sort by in case of
a multi-value metrics aggregation, and in case of a single-value metrics aggregation the sort will be applied on that value).

The path must be defined in the following form:

// https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_Form
[source,ebnf]
--------------------------------------------------
AGG_SEPARATOR       =  '>' ;
METRIC_SEPARATOR    =  '.' ;
AGG_NAME            =  <the name of the aggregation> ;
METRIC              =  <the name of the metric (in case of multi-value metrics aggregation)> ;
PATH                =  <AGG_NAME> [ <AGG_SEPARATOR>, <AGG_NAME> ]* [ <METRIC_SEPARATOR>, <METRIC> ] ;
--------------------------------------------------

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "countries" : {
            "terms" : {
                "field" : "artist.country",
                "order" : { "rock>playback_stats.avg" : "desc" }
            },
            "aggs" : {
                "rock" : {
                    "filter" : { "term" : { "genre" :  "rock" }},
                    "aggs" : {
                        "playback_stats" : { "stats" : { "field" : "play_count" }}
                    }
                }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

The above will sort the artist's countries buckets based on the average play count among the rock songs.

Multiple criteria can be used to order the buckets by providing an array of order criteria such as the following:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "countries" : {
            "terms" : {
                "field" : "artist.country",
                "order" : [ { "rock>playback_stats.avg" : "desc" }, { "_count" : "desc" } ]
            },
            "aggs" : {
                "rock" : {
                    "filter" : { "term" : { "genre" : "rock" }},
                    "aggs" : {
                        "playback_stats" : { "stats" : { "field" : "play_count" }}
                    }
                }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

The above will sort the artist's countries buckets based on the average play count among the rock songs and then by
their `doc_count` in descending order.

NOTE: In the event that two buckets share the same values for all order criteria the bucket's term value is used as a
tie-breaker in ascending alphabetical order to prevent non-deterministic ordering of buckets.

==== Minimum document count

It is possible to only return terms that match more than a configured number of hits using the `min_doc_count` option:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "tags" : {
            "terms" : {
                "field" : "tags",
                "min_doc_count": 10
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

The above aggregation would only return tags which have been found in 10 hits or more. Default value is `1`.

Terms are collected and ordered on a shard level and merged with the terms collected from other shards in a second step. However, the shard does not have the information about the global document count available. The decision if a term is added to a candidate list depends only on the order computed on the shard using local shard frequencies. The `min_doc_count` criterion is only applied after merging local terms statistics of all shards. In a way the decision to add the term as a candidate is made without being very _certain_ about if the term will actually reach the required `min_doc_count`. This might cause many (globally) high frequent terms to be missing in the final result if low frequent terms populated the candidate lists. To avoid this, the `shard_size` parameter can be increased to allow more candidate terms on the shards. However, this increases memory consumption and network traffic.

`shard_min_doc_count` parameter

The parameter `shard_min_doc_count` regulates the _certainty_ a shard has if the term should actually be added to the candidate list or not with respect to the `min_doc_count`. Terms will only be considered if their local shard frequency within the set is higher than the `shard_min_doc_count`. If your dictionary contains many low frequent terms and you are not interested in those (for example misspellings), then you can set the `shard_min_doc_count` parameter to filter out candidate terms on a shard level that will with a reasonable certainty not reach the required `min_doc_count` even after merging the local counts. `shard_min_doc_count` is set to `0` per default and has no effect unless you explicitly set it.

NOTE:    Setting `min_doc_count`=`0` will also return buckets for terms that didn't match any hit. However, some of
         the returned terms which have a document count of zero might only belong to deleted documents or documents
         from other types, so there is no warranty that a `match_all` query would find a positive document count for
         those terms.

WARNING: When NOT sorting on `doc_count` descending, high values of `min_doc_count` may return a number of buckets
         which is less than `size` because not enough data was gathered from the shards. Missing buckets can be
         back by increasing `shard_size`.
         Setting `shard_min_doc_count` too high will cause terms to be filtered out on a shard level. This value should be set much lower than `min_doc_count/#shards`.

[[search-aggregations-bucket-terms-aggregation-script]]
==== Script

Generating the terms using a script:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : {
                "script" : {
                    "source": "doc['genre'].value",
                    "lang": "painless"
                }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

This will interpret the `script` parameter as an `inline` script with the default script language and no script parameters. To use a stored script use the following syntax:

//////////////////////////

[source,js]
--------------------------------------------------
POST /_scripts/my_script
{
  "script": {
    "lang": "painless",
    "source": "doc[params.field].value"
  }
}
--------------------------------------------------
// CONSOLE

//////////////////////////

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : {
                "script" : {
                    "id": "my_script",
                    "params": {
                        "field": "genre"
                    }
                }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE
// TEST[continued]

==== Value Script

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "genres" : {
            "terms" : {
                "field" : "gender",
                "script" : {
                    "source" : "'Genre: ' +_value",
                    "lang" : "painless"
                }
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

==== Filtering Values

It is possible to filter the values for which buckets will be created. This can be done using the `include` and
`exclude` parameters which are based on regular expression strings or arrays of exact values. Additionally,
`include` clauses can filter using `partition` expressions.

===== Filtering Values with regular expressions

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "tags" : {
            "terms" : {
                "field" : "tags",
                "include" : ".*sport.*",
                "exclude" : "water_.*"
            }
        }
    }
}
--------------------------------------------------
// CONSOLE

In the above example, buckets will be created for all the tags that has the word `sport` in them, except those starting
with `water_` (so the tag `water_sports` will no be aggregated). The `include` regular expression will determine what
values are "allowed" to be aggregated, while the `exclude` determines the values that should not be aggregated. When
both are defined, the `exclude` has precedence, meaning, the `include` is evaluated first and only then the `exclude`.

The syntax is the same as <<regexp-syntax,regexp queries>>.

===== Filtering Values with exact values

For matching based on exact values the `include` and `exclude` parameters can simply take an array of
strings that represent the terms as they are found in the index:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "JapaneseCars" : {
             "terms" : {
                 "field" : "make",
                 "include" : ["mazda", "honda"]
             }
         },
        "ActiveCarManufacturers" : {
             "terms" : {
                 "field" : "make",
                 "exclude" : ["rover", "jensen"]
             }
         }
    }
}
--------------------------------------------------
// CONSOLE

===== Filtering Values with partitions

Sometimes there are too many unique terms to process in a single request/response pair so
it can be useful to break the analysis up into multiple requests.
This can be achieved by grouping the field's values into a number of partitions at query-time and processing
only one partition in each request.
Consider this request which is looking for accounts that have not logged any access recently:

[source,js]
--------------------------------------------------
GET /_search
{
   "size": 0,
   "aggs": {
      "expired_sessions": {
         "terms": {
            "field": "account_id",
            "include": {
               "partition": 0,
               "num_partitions": 20
            },
            "size": 10000,
            "order": {
               "last_access": "asc"
            }
         },
         "aggs": {
            "last_access": {
               "max": {
                  "field": "access_date"
               }
            }
         }
      }
   }
}
--------------------------------------------------
// CONSOLE

This request is finding the last logged access date for a subset of customer accounts because we
might want to expire some customer accounts who haven't been seen for a long while.
The `num_partitions` setting has requested that the unique account_ids are organized evenly into twenty
partitions (0 to 19). and the `partition` setting in this request filters to only consider account_ids falling
into partition 0. Subsequent requests should ask for partitions 1 then 2 etc to complete the expired-account analysis.

Note that the `size` setting for the number of results returned needs to be tuned with the `num_partitions`.
For this particular account-expiration example the process for balancing values for `size` and `num_partitions` would be as follows:

1. Use the `cardinality` aggregation to estimate the total number of unique account_id values
2. Pick a value for `num_partitions` to break the number from 1) up into more manageable chunks
3. Pick a `size` value for the number of responses we want from each partition
4. Run a test request

If we have a circuit-breaker error we are trying to do too much in one request and must increase `num_partitions`.
If the request was successful but the last account ID in the date-sorted test response was still an account we might want to
expire then we may be missing accounts of interest and have set our numbers too low. We must either

* increase the `size` parameter to return more results per partition (could be heavy on memory) or
* increase the `num_partitions` to consider less accounts per request (could increase overall processing time as we need to make more requests)

Ultimately this is a balancing act between managing the Elasticsearch resources required to process a single request and the volume
of requests that the client application must issue to complete a task.

==== Multi-field terms aggregation

The `terms` aggregation does not support collecting terms from multiple fields
in the same document.  The reason is that the `terms` agg doesn't collect the
string term values themselves, but rather uses
<<search-aggregations-bucket-terms-aggregation-execution-hint,global ordinals>>
to produce a list of all of the unique values in the field.  Global ordinals
results in an important performance boost which would not be possible across
multiple fields.

There are two approaches that you can use to perform a `terms` agg across
multiple fields:

<<search-aggregations-bucket-terms-aggregation-script,Script>>::

Use a script to retrieve terms from multiple fields.  This disables the global
ordinals optimization and will be slower than collecting terms from a single
field, but it gives you the flexibility to implement this option at search
time.

<<copy-to,`copy_to` field>>::

If you know ahead of time that you want to collect the terms from two or more
fields, then use `copy_to` in your mapping to create a new dedicated field at
index time which contains the values from both fields.  You can aggregate on
this single field, which will benefit from the global ordinals optimization.

==== Collect mode

Deferring calculation of child aggregations

For fields with many unique terms and a small number of required results it can be more efficient to delay the calculation
of child aggregations until the top parent-level aggs have been pruned. Ordinarily, all branches of the aggregation tree
are expanded in one depth-first pass and only then any pruning occurs.
In some scenarios this can be very wasteful and can hit memory constraints.
An example problem scenario is querying a movie database for the 10 most popular actors and their 5 most common co-stars:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "actors" : {
             "terms" : {
                 "field" : "actors",
                 "size" : 10
             },
            "aggs" : {
                "costars" : {
                     "terms" : {
                         "field" : "actors",
                         "size" : 5
                     }
                 }
            }
         }
    }
}
--------------------------------------------------
// CONSOLE

Even though the number of actors may be comparatively small and we want only 50 result buckets there is a combinatorial explosion of buckets
during calculation - a single actor can produce n² buckets where n is the number of actors. The sane option would be to first determine
the 10 most popular actors and only then examine the top co-stars for these 10 actors. This alternative strategy is what we call the `breadth_first` collection
mode as opposed to the `depth_first` mode.

NOTE: The `breadth_first` is the default mode for fields with a cardinality bigger than the requested size or when the cardinality is unknown (numeric fields or scripts for instance).
It is possible to override the default heuristic and to provide a collect mode directly in the request:

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "actors" : {
             "terms" : {
                 "field" : "actors",
                 "size" : 10,
                 "collect_mode" : "breadth_first" <1>
             },
            "aggs" : {
                "costars" : {
                     "terms" : {
                         "field" : "actors",
                         "size" : 5
                     }
                 }
            }
         }
    }
}
--------------------------------------------------
// CONSOLE

<1> the possible values are `breadth_first` and `depth_first`

When using `breadth_first` mode the set of documents that fall into the uppermost buckets are
cached for subsequent replay so there is a memory overhead in doing this which is linear with the number of matching documents.
Note that the `order` parameter can still be used to refer to data from a child aggregation when using the `breadth_first` setting - the parent
aggregation understands that this child aggregation will need to be called first before any of the other child aggregations.

WARNING: Nested aggregations such as `top_hits` which require access to score information under an aggregation that uses the `breadth_first`
collection mode need to replay the query on the second pass but only for the documents belonging to the top buckets.

[[search-aggregations-bucket-terms-aggregation-execution-hint]]
==== Execution hint

There are different mechanisms by which terms aggregations can be executed:

 - by using field values directly in order to aggregate data per-bucket (`map`)
 - by using global ordinals of the field and allocating one bucket per global ordinal (`global_ordinals`)

Elasticsearch tries to have sensible defaults so this is something that generally doesn't need to be configured.

`global_ordinals` is the default option for `keyword` field, it uses global ordinals to allocates buckets dynamically
so memory usage is linear to the number of values of the documents that are part of the aggregation scope.

`map` should only be considered when very few documents match a query. Otherwise the ordinals-based execution mode
is significantly faster. By default, `map` is only used when running an aggregation on scripts, since they don't have
ordinals.

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "tags" : {
             "terms" : {
                 "field" : "tags",
                 "execution_hint": "map" <1>
             }
         }
    }
}
--------------------------------------------------
// CONSOLE

<1> The possible values are `map`, `global_ordinals`

Please note that Elasticsearch will ignore this execution hint if it is not applicable and that there is no backward compatibility guarantee on these hints.

==== Missing value

The `missing` parameter defines how documents that are missing a value should be treated.
By default they will be ignored but it is also possible to treat them as if they
had a value.

[source,js]
--------------------------------------------------
GET /_search
{
    "aggs" : {
        "tags" : {
             "terms" : {
                 "field" : "tags",
                 "missing": "N/A" <1>
             }
         }
    }
}
--------------------------------------------------
// CONSOLE

<1> Documents without a value in the `tags` field will fall into the same bucket as documents that have the value `N/A`.

==== Mixing field types

WARNING: When aggregating on multiple indices the type of the aggregated field may not be the same in all indices.
Some types are compatible with each other (`integer` and `long` or `float` and `double`) but when the types are a mix
of decimal and non-decimal number the terms aggregation will promote the non-decimal numbers to decimal numbers.
This can result in a loss of precision in the bucket values.


Fixing doc error

Fixing doc error

[[eshadoop-5.2.1]]
== Elasticsearch for Apache Hadoop version 5.2.1
February 14, 2017

ES-Hadoop 5.2.1 is a version compatibility release, tested specifically against Elasticsearch 5.2.1.

[[docs-5.2.1]]
=== Documentation
* Updated links that point to the ES reference documentation to use version 5.2 instead of 2.0.

[[eshadoop-5.2.1]]
== Elasticsearch for Apache Hadoop version 5.2.1
February 14, 2017

ES-Hadoop 5.2.1 is a version compatibility release, tested specifically against Elasticsearch 5.2.1.

[[docs-5.1.2]]
=== Documentation
* Updated links that point to the ES reference documentation to use version 5.2 instead of 2.0.


No issue. Fixed table.

No issue. Fixed table.

// Copyright 2015
// Ubiquitous Knowledge Processing (UKP) Lab and FG Language Technology
// Technische Universität Darmstadt
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

[[sect_database]]
== Database

WebAnno uses an SQL database to store project and user data.

We test MySQL using a MySQL server. WebAnno uses by default an embedded HSQLDB database. However, we
recommend using the embedded database only for testing purposes. For production use, we recommend
using a MySQL server. The reason for this is, that:

* we do more testing on the MySQL server and
* in the past, we had cases where we described in-place upgrade procedures that required performing
  SQL commands to change the data model as part of the upgrade. We promise to try avoiding this in
  the future. However, in case we offer advice on fixing anything directly in the database, this
  advice will refer to a MySQL database.

We try to keep the data model simple, so there should be no significant requirements to the database
being used. Theoretically, it should be possible to use any JDBC-compatible database after adding a
corresponding driver to the classpath and configuring WebAnno to use the driver in the
`settings.properties` file.

If you plan to use UTF-8 encoding for project name and tagset/tag name, make sure either of the following settings for MySQL databse
a) in the `settings.properties` file, make sure that  `database.url` includes

----
useUnicode=true&characterEncoding=UTF-8
----

b) change the `my.conf` MySQL databse configuration file to include the following line

----
character-set-server = utf8
----

=== Using HSQLDB in production

WebAnno displays a warning in the user interface when an embedded database is being used. In case
that you really want to run WebAnno with an embedded database in production, you probably want to
disable this warning. To do so, please add the following entry to the `settings.properties` file:

----
warnings.embeddedDatabase=false
----

// Copyright 2015
// Ubiquitous Knowledge Processing (UKP) Lab and FG Language Technology
// Technische Universität Darmstadt
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

[[sect_database]]
== Database

WebAnno uses an SQL database to store project and user data.

We test MySQL using a MySQL server. WebAnno uses by default an embedded HSQLDB database. However, we
recommend using the embedded database only for testing purposes. For production use, we recomment
using a MySQL server. The reason for this is, that:

* we do more testing on the MySQL server and
* in the past, we had cases where we described in-place upgrade procedures that required performing
  SQL commands to change the data model as part of the upgrade. We promise to try avoiding this in
  the future. However, in case we offer advice on fixing anything directly in the database, this
  advice will refer to a MySQL database.

We try to keep the data model simple, so there should be no significant requirements to the database
being used. Theoretically, it should be possible to use any JDBC-compatible database after adding a
corresponding driver to the classpath and configuring WebAnno to use the driver in the
`settings.properties` file.

If you plan to use UTF-8 encoding for project name and tagset/tag name, make sure either of the following settings for MySQL databse
a) in the `settings.properties` file, make sure that  `database.url` includes

----
useUnicode=true&characterEncoding=UTF-8
----

b) change the `my.conf` MySQL databse configuration file to include the following line

----
character-set-server = utf8
----

=== Using HSQLDB in production

WebAnno displays a warning in the user interface when an embedded database is being used. In case
that you really want to run WebAnno with an embedded database in production, you probably want to
disable this warning. To do so, please add the following entry to the `settings.properties` file:

----
warnings.embeddedDatabase=false
----