Regen

Regen

[[telegram-component]]
= Telegram Component
:page-source: components/camel-telegram/src/main/docs/telegram-component.adoc

*Since Camel 2.18*

// HEADER START
*Both producer and consumer is supported*
// HEADER END

The Telegram component provides access to the https://core.telegram.org/bots/api[Telegram Bot API].
It allows a Camel-based application to send and receive messages by acting as a Bot, participating in
direct conversations with normal users, private and public groups or channels.

A Telegram Bot must be created before using this component, following the instructions at the
https://core.telegram.org/bots#3-how-do-i-create-a-bot[Telegram Bot developers home].
When a new Bot is created, the https://telegram.me/botfather[BotFather] provides an
**authorization token** corresponding to the Bot. The authorization token is a mandatory parameter
for the camel-telegram endpoint.

NOTE: In order to allow the Bot to receive all messages exchanged within a group or channel (not just
the ones starting with a '/' character), ask the BotFather to *disable the privacy mode*, using the
*/setprivacy* command.

Maven users will need to add the following dependency to their `pom.xml`
for this component:

[source,xml]
------------------------------------------------------------
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-telegram</artifactId>
    <version>x.x.x</version>
    <!-- use the same version as your Camel core version -->
</dependency>
------------------------------------------------------------

== URI format

[source,text]
----------------------------------------------------
telegram:type/authorizationToken[?options]
----------------------------------------------------

You can append query options to the URI in the following format,
`?option=value&option=value&...`

== Options

// component options: START
The Telegram component supports 7 options, which are listed below.

[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *authorizationToken* (security) | The default Telegram authorization token to be used when the information is not provided in the endpoints. |  | String
| *client* (advanced) | To use a custom AsyncHttpClient |  | AsyncHttpClient
| *clientConfig* (advanced) | To configure the AsyncHttpClient to use a custom com.ning.http.client.AsyncHttpClientConfig instance. |  | AsyncHttpClientConfig
| *baseUri* (advanced) | Set an alternative base URI, e.g. when you want to test the component against a mock Telegram API. | https://api.telegram.org | String
| *basicPropertyBinding* (advanced) | Whether the component should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | boolean
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean
| *bridgeErrorHandler* (consumer) | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | boolean
|===
// component options: END

// endpoint options: START
The Telegram endpoint is configured using URI syntax:

----
telegram:type
----

with the following path and query parameters:

=== Path Parameters (1 parameters):

[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *type* | *Required* The endpoint type. Currently, only the 'bots' type is supported. |  | String
|===

=== Query Parameters (31 parameters):

[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *bridgeErrorHandler* (consumer) | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | boolean
| *limit* (consumer) | Limit on the number of updates that can be received in a single polling request. | 100 | Integer
| *sendEmptyMessageWhenIdle* (consumer) | If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead. | false | boolean
| *timeout* (consumer) | Timeout in seconds for long polling. Put 0 for short polling or a bigger number for long polling. Long polling produces shorter response time. | 30 | Integer
| *exceptionHandler* (consumer) | To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored. |  | ExceptionHandler
| *exchangePattern* (consumer) | Sets the exchange pattern when the consumer creates an exchange. |  | ExchangePattern
| *pollStrategy* (consumer) | A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel. |  | PollingConsumerPollStrategy
| *chatId* (producer) | The identifier of the chat that will receive the produced messages. Chat ids can be first obtained from incoming messages (eg. when a telegram user starts a conversation with a bot, its client sends automatically a '/start' message containing the chat id). It is an optional parameter, as the chat id can be set dynamically for each outgoing message (using body or headers). |  | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean
| *baseUri* (advanced) | Set an alternative base URI, e.g. when you want to test the component against a mock Telegram API. |  | String
| *basicPropertyBinding* (advanced) | Whether the endpoint should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | boolean
| *bufferSize* (advanced) | The initial in-memory buffer size used when transferring data between Camel and AHC Client. | 4096 | int
| *clientConfig* (advanced) | To configure the AsyncHttpClient to use a custom com.ning.http.client.AsyncHttpClientConfig instance. |  | AsyncHttpClientConfig
| *synchronous* (advanced) | Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported). | false | boolean
| *backoffErrorThreshold* (scheduler) | The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in. |  | int
| *backoffIdleThreshold* (scheduler) | The number of subsequent idle polls that should happen before the backoffMultipler should kick-in. |  | int
| *backoffMultiplier* (scheduler) | To let the scheduled polling consumer backoff if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt is happening again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured. |  | int
| *delay* (scheduler) | Milliseconds before the next poll. You can also specify time values using units, such as 60s (60 seconds), 5m30s (5 minutes and 30 seconds), and 1h (1 hour). | 500 | long
| *greedy* (scheduler) | If greedy is enabled, then the ScheduledPollConsumer will run immediately again, if the previous run polled 1 or more messages. | false | boolean
| *initialDelay* (scheduler) | Milliseconds before the first poll starts. You can also specify time values using units, such as 60s (60 seconds), 5m30s (5 minutes and 30 seconds), and 1h (1 hour). | 1000 | long
| *repeatCount* (scheduler) | Specifies a maximum limit of number of fires. So if you set it to 1, the scheduler will only fire once. If you set it to 5, it will only fire five times. A value of zero or negative means fire forever. | 0 | long
| *runLoggingLevel* (scheduler) | The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that. | TRACE | LoggingLevel
| *scheduledExecutorService* (scheduler) | Allows for configuring a custom/shared thread pool to use for the consumer. By default each consumer has its own single threaded thread pool. |  | ScheduledExecutorService
| *scheduler* (scheduler) | To use a cron scheduler from either camel-spring or camel-quartz component | none | String
| *schedulerProperties* (scheduler) | To configure additional properties when using a custom scheduler or any of the Quartz, Spring based scheduler. |  | Map
| *startScheduler* (scheduler) | Whether the scheduler should be auto started. | true | boolean
| *timeUnit* (scheduler) | Time unit for initialDelay and delay options. | MILLISECONDS | TimeUnit
| *useFixedDelay* (scheduler) | Controls if fixed delay or fixed rate is used. See ScheduledExecutorService in JDK for details. | true | boolean
| *proxyHost* (proxy) | HTTP proxy host which could be used when sending out the message. |  | String
| *proxyPort* (proxy) | HTTP proxy port which could be used when sending out the message. |  | Integer
| *authorizationToken* (security) | *Required* The authorization token for using the bot (ask the BotFather) |  | String
|===
// endpoint options: END
// spring-boot-auto-configure options: START
== Spring Boot Auto-Configuration

When using Spring Boot make sure to use the following Maven dependency to have support for auto configuration:

[source,xml]
----
<dependency>
  <groupId>org.apache.camel.springboot</groupId>
  <artifactId>camel-telegram-starter</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel core version -->
</dependency>
----

The component supports 8 options, which are listed below.

[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *camel.component.telegram.authorization-token* | The default Telegram authorization token to be used when the information is not provided in the endpoints. |  | String
| *camel.component.telegram.base-uri* | Set an alternative base URI, e.g. when you want to test the component against a mock Telegram API. |  | String
| *camel.component.telegram.basic-property-binding* | Whether the component should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | Boolean
| *camel.component.telegram.bridge-error-handler* | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | Boolean
| *camel.component.telegram.client* | To use a custom AsyncHttpClient. The option is a org.asynchttpclient.AsyncHttpClient type. |  | String
| *camel.component.telegram.client-config* | To configure the AsyncHttpClient to use a custom com.ning.http.client.AsyncHttpClientConfig instance. The option is a org.asynchttpclient.AsyncHttpClientConfig type. |  | String
| *camel.component.telegram.enabled* | Whether to enable auto configuration of the telegram component. This is enabled by default. |  | Boolean
| *camel.component.telegram.lazy-start-producer* | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | Boolean
|===
// spring-boot-auto-configure options: END

== Message Headers

[width="100%",cols="20%,80%",options="header",]
|=======================================================================
|Name |Description
|`CamelTelegramChatId` |This header is used by the producer endpoint in order to
resolve the chat id that will receive the message. The recipient chat id can be
placed (in order of priority) in message body, in the `CamelTelegramChatId` header
or in the endpoint configuration (`chatId` option).
This header is also present in all incoming messages.

|`CamelTelegramMediaType` |This header is used to identify the media type when
the outgoing message is composed of pure binary data. Possible values are strings or enum values
belonging to the `org.apache.camel.component.telegram.TelegramMediaType` enumeration.

|`CamelTelegramMediaTitleCaption` |This header is used to provide a caption or title
for outgoing binary messages.

|`CamelTelegramParseMode` |This header is used to format text messages using HTML or Markdown (see `org.apache.camel.component.telegram.TelegramParseMode`).

|=======================================================================

== Usage

The Telegram component supports both consumer and producer endpoints.
It can also be used in *reactive chat-bot mode* (to consume, then produce messages).

== Producer Example

The following is a basic example of how to send a message to a Telegram chat through the
Telegram Bot API.

in Java DSL

[source,java]
---------------------------------------------------------
from("direct:start").to("telegram:bots/123456789:insertYourAuthorizationTokenHere");
---------------------------------------------------------

or in Spring XML

[source,xml]
---------------------------------------------
<route>
    <from uri="direct:start"/>
    <to uri="telegram:bots/123456789:insertYourAuthorizationTokenHere"/>
<route>
---------------------------------------------

The code `123456789:insertYourAuthorizationTokenHere` is the *authorization token* corresponding to the Bot.

When using the producer endpoint without specifying the *chat id* option, the target chat will be identified using information contained in the body or headers of the message.
The following message bodies are allowed for a producer endpoint (messages of type `OutgoingXXXMessage` belong to the package `org.apache.camel.component.telegram.model`)

[width="100%",cols="40%,60%",options="header",]
|===================================================
| Java Type | Description

| `OutgoingTextMessage` | To send a text message to a chat
| `OutgoingPhotoMessage` | To send a photo (JPG, PNG) to a chat
| `OutgoingAudioMessage` | To send a mp3 audio to a chat
| `OutgoingVideoMessage` | To send a mp4 video to a chat
| `OutgoingDocumentMessage` | To send a file to a chat (any media type)
| `OutgoingStickerMessage` | To send a sticker to a chat (WEBP)
| `EditMessageTextMessage` | To edit text and game messages (editMessageText)
| `EditMessageCaptionMessage` | To edit captions of messages (editMessageCaption)
| `EditMessageMediaMessage` | To edit animation, audio, document, photo, or video messages. (editMessageMedia)
| `EditMessageReplyMarkupMessage` | To edit only the reply markup of message. (editMessageReplyMarkup)
| `EditMessageDelete` | To delete a message, including service messages. (deleteMessage)
| `SendLocationMessage` | To send a location (setSendLocation)
| `EditMessageLiveLocationMessage` | To send changes to a live location (editMessageLiveLocation)
| `StopMessageLiveLocationMessage` | To stop updating a live location message sent by the bot or via the bot (for inline bots) before live_period expires (stopMessageLiveLocation)
| `SendVenueMessage` | To send information about a venue (sendVenue)
| `byte[]` | To send any media type supported. It requires the `CamelTelegramMediaType` header to be set to the appropriate media type
| `String` | To send a text message to a chat. It gets converted automatically into a `OutgoingTextMessage`

|===================================================

== Consumer Example

The following is a basic example of how to receive all messages that telegram users are sending to the configured Bot.
In Java DSL

[source,java]
---------------------------------------------------------
from("telegram:bots/123456789:insertYourAuthorizationTokenHere")
.bean(ProcessorBean.class)
---------------------------------------------------------

or in Spring XML

[source,xml]
---------------------------------------------
<route>
    <from uri="telegram:bots/123456789:insertYourAuthorizationTokenHere"/>
    <bean ref="myBean" />
<route>

<bean id="myBean" class="com.example.MyBean"/>
---------------------------------------------

The `MyBean` is a simple bean that will receive the messages

[source,java]
---------------------------------------------------------
public class MyBean {

    public void process(String message) {
        // or Exchange, or org.apache.camel.component.telegram.model.IncomingMessage (or both)

        // do process
    }

}
---------------------------------------------------------

Supported types for incoming messages are

[width="100%",cols="40%,60%",options="header",]
|===================================================
| Java Type | Description

| `IncomingMessage` | The full object representation of an incoming message
| `String` | The content of the message, for text messages only

|===================================================

== Reactive Chat-Bot Example

The reactive chat-bot mode is a simple way of using the Camel component to build a simple
chat bot that replies directly to chat messages received from the Telegram users.

The following is a basic configuration of the chat-bot in Java DSL

[source,java]
---------------------------------------------------------
from("telegram:bots/123456789:insertYourAuthorizationTokenHere")
.bean(ChatBotLogic.class)
.to("telegram:bots/123456789:insertYourAuthorizationTokenHere");
---------------------------------------------------------

or in Spring XML

[source,xml]
---------------------------------------------
<route>
    <from uri="telegram:bots/123456789:insertYourAuthorizationTokenHere"/>
    <bean ref="chatBotLogic" />
    <to uri="telegram:bots/123456789:insertYourAuthorizationTokenHere"/>
<route>

<bean id="chatBotLogic" class="com.example.ChatBotLogic"/>
---------------------------------------------

The `ChatBotLogic` is a simple bean that implements a generic String-to-String method.

[source,java]
---------------------------------------------------------
public class ChatBotLogic {

    public String chatBotProcess(String message) {
        if( "do-not-reply".equals(message) ) {
            return null; // no response in the chat
        }

        return "echo from the bot: " + message; // echoes the message
    }

}
---------------------------------------------------------

Every non-null string returned by the `chatBotProcess` method is automatically routed to the
chat that originated the request (as the `CamelTelegramChatId` header is used to route the message).

== Getting the Chat ID

If you want to push messages to a specific Telegram chat when an event occurs, you need to
retrieve the corresponding chat ID. The chat ID is not currently shown in the telegram client,
but you can obtain it using a simple route.

First, add the bot to the chat where you want to push messages, then run a route like the following one.

[source,java]
---------------------------------------------------------
from("telegram:bots/123456789:insertYourAuthorizationTokenHere")
.to("log:INFO?showHeaders=true");
---------------------------------------------------------

Any message received by the bot will be dumped to your log together with information about the chat (`CamelTelegramChatId`
header).

Once you get the chat ID, you can use the following sample route to push message to it.

[source,java]
---------------------------------------------------------
from("timer:tick")
.setBody().constant("Hello")
to("telegram:bots/123456789:insertYourAuthorizationTokenHere?chatId=123456")
---------------------------------------------------------

Note that the corresponding URI parameter is simply `chatId`.

== Customizing keyboard

You can customize the user keyboard instead of asking him to write an option. `OutgoingTextMessage` has the property `ReplyKeyboardMarkup` which can be used for such thing.

[source,java]
---------------------------------------------------------
from("telegram:bots/123456789:insertYourAuthorizationTokenHere")
    .process(exchange -> {

        OutgoingTextMessage msg = new OutgoingTextMessage();
        msg.setText("Choose one option!");

        InlineKeyboardButton buttonOptionOneI = InlineKeyboardButton.builder()
                .text("Option One - I").build();

        InlineKeyboardButton buttonOptionOneII = InlineKeyboardButton.builder()
                .text("Option One - II").build();

        InlineKeyboardButton buttonOptionTwoI = InlineKeyboardButton.builder()
                .text("Option Two - I").build();

        ReplyKeyboardMarkup replyMarkup = ReplyKeyboardMarkup.builder()
                .keyboard()
                    .addRow(Arrays.asList(buttonOptionOneI, buttonOptionOneII))
                    .addRow(Arrays.asList(buttonOptionTwoI))
                    .close()
                .oneTimeKeyboard(true)
                .build();

        msg.setReplyKeyboardMarkup(replyMarkup);

        exchange.getIn().setBody(msg);
    })
    .to("telegram:bots/123456789:insertYourAuthorizationTokenHere");
---------------------------------------------------------

If you want to disable it the next message must have the property `removeKeyboard` set on `ReplyKeyboardMarkup` object.

[source,java]
---------------------------------------------------------
from("telegram:bots/123456789:insertYourAuthorizationTokenHere")
    .process(exchange -> {

        OutgoingTextMessage msg = new OutgoingTextMessage();
        msg.setText("Your answer was accepted!");

        ReplyKeyboardMarkup replyMarkup = ReplyKeyboardMarkup.builder()
                .removeKeyboard(true)
                .build();

        msg.setReplyKeyboardMarkup(replyMarkup);

        exchange.getIn().setBody(msg);
    })
    .to("telegram:bots/123456789:insertYourAuthorizationTokenHere");
---------------------------------------------------------

== Webhook Mode

The Telegram component supports usage in the *webhook mode* using the *camel-webhook* component.

In order to enable webhook mode, users need first to add a REST implementation to their application.
Maven users, for example, can add *netty-http* to their `pom.xml` file:

[source,xml]
------------------------------------------------------------
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-netty-http</artifactId>
    <version>x.x.x</version>
    <!-- use the same version as your Camel core version -->
</dependency>
------------------------------------------------------------

Once done, you need to prepend the webhook URI to the telegram URI you want to use.

In Java DSL:

[source,java]
---------------------------------------------------------
from("webhook:telegram:bots/123456789:insertYourAuthorizationTokenHere").to("log:info");
---------------------------------------------------------

Some endpoints will be exposed by your application and Telegram will be configured to send messages to them.
You need to ensure that your server is exposed to the internet and to pass the right value of the
*camel.component.webhook.configuration.webhook-external-url* property.

Refer to the *camel-webhook* component documentation for instructions on how to set it.

[[telegram-component]]
= Telegram Component
:page-source: components/camel-telegram/src/main/docs/telegram-component.adoc

*Since Camel 2.18*

// HEADER START
*Both producer and consumer is supported*
// HEADER END

The Telegram component provides access to the https://core.telegram.org/bots/api[Telegram Bot API].
It allows a Camel-based application to send and receive messages by acting as a Bot, participating in
direct conversations with normal users, private and public groups or channels.

A Telegram Bot must be created before using this component, following the instructions at the
https://core.telegram.org/bots#3-how-do-i-create-a-bot[Telegram Bot developers home].
When a new Bot is created, the https://telegram.me/botfather[BotFather] provides an
**authorization token** corresponding to the Bot. The authorization token is a mandatory parameter
for the camel-telegram endpoint.

NOTE: In order to allow the Bot to receive all messages exchanged within a group or channel (not just
the ones starting with a '/' character), ask the BotFather to *disable the privacy mode*, using the
*/setprivacy* command.

Maven users will need to add the following dependency to their `pom.xml`
for this component:

[source,xml]
------------------------------------------------------------
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-telegram</artifactId>
    <version>x.x.x</version>
    <!-- use the same version as your Camel core version -->
</dependency>
------------------------------------------------------------

== URI format

[source,text]
----------------------------------------------------
telegram:type/authorizationToken[?options]
----------------------------------------------------

You can append query options to the URI in the following format,
`?option=value&option=value&...`

== Options

// component options: START
The Telegram component supports 7 options, which are listed below.

[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *authorizationToken* (security) | The default Telegram authorization token to be used when the information is not provided in the endpoints. |  | String
| *client* (advanced) | To use a custom AsyncHttpClient |  | AsyncHttpClient
| *clientConfig* (advanced) | To configure the AsyncHttpClient to use a custom com.ning.http.client.AsyncHttpClientConfig instance. |  | AsyncHttpClientConfig
| *baseUri* (advanced) | Set an alternative base URI, e.g. when you want to test the component against a mock Telegram API. | https://api.telegram.org | String
| *basicPropertyBinding* (advanced) | Whether the component should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | boolean
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean
| *bridgeErrorHandler* (consumer) | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | boolean
|===
// component options: END

// endpoint options: START
The Telegram endpoint is configured using URI syntax:

----
telegram:type
----

with the following path and query parameters:

=== Path Parameters (1 parameters):

[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *type* | *Required* The endpoint type. Currently, only the 'bots' type is supported. |  | String
|===

=== Query Parameters (31 parameters):

[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *bridgeErrorHandler* (consumer) | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | boolean
| *limit* (consumer) | Limit on the number of updates that can be received in a single polling request. | 100 | Integer
| *sendEmptyMessageWhenIdle* (consumer) | If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead. | false | boolean
| *timeout* (consumer) | Timeout in seconds for long polling. Put 0 for short polling or a bigger number for long polling. Long polling produces shorter response time. | 30 | Integer
| *exceptionHandler* (consumer) | To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored. |  | ExceptionHandler
| *exchangePattern* (consumer) | Sets the exchange pattern when the consumer creates an exchange. |  | ExchangePattern
| *pollStrategy* (consumer) | A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel. |  | PollingConsumerPollStrategy
| *chatId* (producer) | The identifier of the chat that will receive the produced messages. Chat ids can be first obtained from incoming messages (eg. when a telegram user starts a conversation with a bot, its client sends automatically a '/start' message containing the chat id). It is an optional parameter, as the chat id can be set dynamically for each outgoing message (using body or headers). |  | String
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean
| *baseUri* (advanced) | Set an alternative base URI, e.g. when you want to test the component against a mock Telegram API. |  | String
| *basicPropertyBinding* (advanced) | Whether the endpoint should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | boolean
| *bufferSize* (advanced) | The initial in-memory buffer size used when transferring data between Camel and AHC Client. | 4096 | int
| *clientConfig* (advanced) | To configure the AsyncHttpClient to use a custom com.ning.http.client.AsyncHttpClientConfig instance. |  | AsyncHttpClientConfig
| *synchronous* (advanced) | Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported). | false | boolean
| *backoffErrorThreshold* (scheduler) | The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in. |  | int
| *backoffIdleThreshold* (scheduler) | The number of subsequent idle polls that should happen before the backoffMultipler should kick-in. |  | int
| *backoffMultiplier* (scheduler) | To let the scheduled polling consumer backoff if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt is happening again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured. |  | int
| *delay* (scheduler) | Milliseconds before the next poll. You can also specify time values using units, such as 60s (60 seconds), 5m30s (5 minutes and 30 seconds), and 1h (1 hour). | 500 | long
| *greedy* (scheduler) | If greedy is enabled, then the ScheduledPollConsumer will run immediately again, if the previous run polled 1 or more messages. | false | boolean
| *initialDelay* (scheduler) | Milliseconds before the first poll starts. You can also specify time values using units, such as 60s (60 seconds), 5m30s (5 minutes and 30 seconds), and 1h (1 hour). | 1000 | long
| *repeatCount* (scheduler) | Specifies a maximum limit of number of fires. So if you set it to 1, the scheduler will only fire once. If you set it to 5, it will only fire five times. A value of zero or negative means fire forever. | 0 | long
| *runLoggingLevel* (scheduler) | The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that. | TRACE | LoggingLevel
| *scheduledExecutorService* (scheduler) | Allows for configuring a custom/shared thread pool to use for the consumer. By default each consumer has its own single threaded thread pool. |  | ScheduledExecutorService
| *scheduler* (scheduler) | To use a cron scheduler from either camel-spring or camel-quartz component | none | String
| *schedulerProperties* (scheduler) | To configure additional properties when using a custom scheduler or any of the Quartz, Spring based scheduler. |  | Map
| *startScheduler* (scheduler) | Whether the scheduler should be auto started. | true | boolean
| *timeUnit* (scheduler) | Time unit for initialDelay and delay options. | MILLISECONDS | TimeUnit
| *useFixedDelay* (scheduler) | Controls if fixed delay or fixed rate is used. See ScheduledExecutorService in JDK for details. | true | boolean
| *proxyHost* (proxy) | HTTP proxy host which could be used when sending out the message. |  | String
| *proxyPort* (proxy) | HTTP proxy port which could be used when sending out the message. |  | Integer
| *authorizationToken* (security) | *Required* The authorization token for using the bot (ask the BotFather) |  | String
|===
// endpoint options: END
// spring-boot-auto-configure options: START
== Spring Boot Auto-Configuration

When using Spring Boot make sure to use the following Maven dependency to have support for auto configuration:

[source,xml]
----
<dependency>
  <groupId>org.apache.camel.springboot</groupId>
  <artifactId>camel-telegram-starter</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel core version -->
</dependency>
----

The component supports 8 options, which are listed below.

[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *camel.component.telegram.authorization-token* | The default Telegram authorization token to be used when the information is not provided in the endpoints. |  | String
| *camel.component.telegram.base-uri* | Set an alternative base URI, e.g. when you want to test the component against a mock Telegram API. |  | String
| *camel.component.telegram.basic-property-binding* | Whether the component should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | Boolean
| *camel.component.telegram.bridge-error-handler* | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | Boolean
| *camel.component.telegram.client* | To use a custom AsyncHttpClient. The option is a org.asynchttpclient.AsyncHttpClient type. |  | String
| *camel.component.telegram.client-config* | To configure the AsyncHttpClient to use a custom com.ning.http.client.AsyncHttpClientConfig instance. The option is a org.asynchttpclient.AsyncHttpClientConfig type. |  | String
| *camel.component.telegram.enabled* | Whether to enable auto configuration of the telegram component. This is enabled by default. |  | Boolean
| *camel.component.telegram.lazy-start-producer* | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | Boolean
|===
// spring-boot-auto-configure options: END

== Message Headers

[width="100%",cols="20%,80%",options="header",]
|=======================================================================
|Name |Description
|`CamelTelegramChatId` |This header is used by the producer endpoint in order to
resolve the chat id that will receive the message. The recipient chat id can be
placed (in order of priority) in message body, in the `CamelTelegramChatId` header
or in the endpoint configuration (`chatId` option).
This header is also present in all incoming messages.

|`CamelTelegramMediaType` |This header is used to identify the media type when
the outgoing message is composed of pure binary data. Possible values are strings or enum values
belonging to the `org.apache.camel.component.telegram.TelegramMediaType` enumeration.

|`CamelTelegramMediaTitleCaption` |This header is used to provide a caption or title
for outgoing binary messages.

|`CamelTelegramParseMode` |This header is used to format text messages using HTML or Markdown (see `org.apache.camel.component.telegram.TelegramParseMode`).

|=======================================================================

== Usage

The Telegram component supports both consumer and producer endpoints.
It can also be used in *reactive chat-bot mode* (to consume, then produce messages).

== Producer Example

The following is a basic example of how to send a message to a Telegram chat through the
Telegram Bot API.

in Java DSL

[source,java]
---------------------------------------------------------
from("direct:start").to("telegram:bots/123456789:insertYourAuthorizationTokenHere");
---------------------------------------------------------

or in Spring XML

[source,xml]
---------------------------------------------
<route>
    <from uri="direct:start"/>
    <to uri="telegram:bots/123456789:insertYourAuthorizationTokenHere"/>
<route>
---------------------------------------------

The code `123456789:insertYourAuthorizationTokenHere` is the *authorization token* corresponding to the Bot.

When using the producer endpoint without specifying the *chat id* option, the target chat will be identified using information contained in the body or headers of the message.
The following message bodies are allowed for a producer endpoint (messages of type `OutgoingXXXMessage` belong to the package `org.apache.camel.component.telegram.model`)

[width="100%",cols="40%,60%",options="header",]
|===================================================
| Java Type | Description

| `OutgoingTextMessage` | To send a text message to a chat
| `OutgoingPhotoMessage` | To send a photo (JPG, PNG) to a chat
| `OutgoingAudioMessage` | To send a mp3 audio to a chat
| `OutgoingVideoMessage` | To send a mp4 video to a chat
| `OutgoingDocumentMessage` | To send a file to a chat (any media type)
| `OutgoingStickerMessage` | To send a sticker to a chat (WEBP)
| `SendLocationMessage` | To send a location (setSendLocation)
| `EditMessageLiveLocationMessage` | To send changes to a live location (editMessageLiveLocation)
| `StopMessageLiveLocationMessage` | To stop updating a live location message sent by the bot or via the bot (for inline bots) before live_period expires (stopMessageLiveLocation)
| `SendVenueMessage` | To send information about a venue (sendVenue)
| `byte[]` | To send any media type supported. It requires the `CamelTelegramMediaType` header to be set to the appropriate media type
| `String` | To send a text message to a chat. It gets converted automatically into a `OutgoingTextMessage`

|===================================================

== Consumer Example

The following is a basic example of how to receive all messages that telegram users are sending to the configured Bot.
In Java DSL

[source,java]
---------------------------------------------------------
from("telegram:bots/123456789:insertYourAuthorizationTokenHere")
.bean(ProcessorBean.class)
---------------------------------------------------------

or in Spring XML

[source,xml]
---------------------------------------------
<route>
    <from uri="telegram:bots/123456789:insertYourAuthorizationTokenHere"/>
    <bean ref="myBean" />
<route>

<bean id="myBean" class="com.example.MyBean"/>
---------------------------------------------

The `MyBean` is a simple bean that will receive the messages

[source,java]
---------------------------------------------------------
public class MyBean {

    public void process(String message) {
        // or Exchange, or org.apache.camel.component.telegram.model.IncomingMessage (or both)

        // do process
    }

}
---------------------------------------------------------

Supported types for incoming messages are

[width="100%",cols="40%,60%",options="header",]
|===================================================
| Java Type | Description

| `IncomingMessage` | The full object representation of an incoming message
| `String` | The content of the message, for text messages only

|===================================================

== Reactive Chat-Bot Example

The reactive chat-bot mode is a simple way of using the Camel component to build a simple
chat bot that replies directly to chat messages received from the Telegram users.

The following is a basic configuration of the chat-bot in Java DSL

[source,java]
---------------------------------------------------------
from("telegram:bots/123456789:insertYourAuthorizationTokenHere")
.bean(ChatBotLogic.class)
.to("telegram:bots/123456789:insertYourAuthorizationTokenHere");
---------------------------------------------------------

or in Spring XML

[source,xml]
---------------------------------------------
<route>
    <from uri="telegram:bots/123456789:insertYourAuthorizationTokenHere"/>
    <bean ref="chatBotLogic" />
    <to uri="telegram:bots/123456789:insertYourAuthorizationTokenHere"/>
<route>

<bean id="chatBotLogic" class="com.example.ChatBotLogic"/>
---------------------------------------------

The `ChatBotLogic` is a simple bean that implements a generic String-to-String method.

[source,java]
---------------------------------------------------------
public class ChatBotLogic {

    public String chatBotProcess(String message) {
        if( "do-not-reply".equals(message) ) {
            return null; // no response in the chat
        }

        return "echo from the bot: " + message; // echoes the message
    }

}
---------------------------------------------------------

Every non-null string returned by the `chatBotProcess` method is automatically routed to the
chat that originated the request (as the `CamelTelegramChatId` header is used to route the message).

== Getting the Chat ID

If you want to push messages to a specific Telegram chat when an event occurs, you need to
retrieve the corresponding chat ID. The chat ID is not currently shown in the telegram client,
but you can obtain it using a simple route.

First, add the bot to the chat where you want to push messages, then run a route like the following one.

[source,java]
---------------------------------------------------------
from("telegram:bots/123456789:insertYourAuthorizationTokenHere")
.to("log:INFO?showHeaders=true");
---------------------------------------------------------

Any message received by the bot will be dumped to your log together with information about the chat (`CamelTelegramChatId`
header).

Once you get the chat ID, you can use the following sample route to push message to it.

[source,java]
---------------------------------------------------------
from("timer:tick")
.setBody().constant("Hello")
to("telegram:bots/123456789:insertYourAuthorizationTokenHere?chatId=123456")
---------------------------------------------------------

Note that the corresponding URI parameter is simply `chatId`.

== Customizing keyboard

You can customize the user keyboard instead of asking him to write an option. `OutgoingTextMessage` has the property `ReplyKeyboardMarkup` which can be used for such thing.

[source,java]
---------------------------------------------------------
from("telegram:bots/123456789:insertYourAuthorizationTokenHere")
    .process(exchange -> {

        OutgoingTextMessage msg = new OutgoingTextMessage();
        msg.setText("Choose one option!");

        InlineKeyboardButton buttonOptionOneI = InlineKeyboardButton.builder()
                .text("Option One - I").build();

        InlineKeyboardButton buttonOptionOneII = InlineKeyboardButton.builder()
                .text("Option One - II").build();

        InlineKeyboardButton buttonOptionTwoI = InlineKeyboardButton.builder()
                .text("Option Two - I").build();

        ReplyKeyboardMarkup replyMarkup = ReplyKeyboardMarkup.builder()
                .keyboard()
                    .addRow(Arrays.asList(buttonOptionOneI, buttonOptionOneII))
                    .addRow(Arrays.asList(buttonOptionTwoI))
                    .close()
                .oneTimeKeyboard(true)
                .build();

        msg.setReplyKeyboardMarkup(replyMarkup);

        exchange.getIn().setBody(msg);
    })
    .to("telegram:bots/123456789:insertYourAuthorizationTokenHere");
---------------------------------------------------------

If you want to disable it the next message must have the property `removeKeyboard` set on `ReplyKeyboardMarkup` object.

[source,java]
---------------------------------------------------------
from("telegram:bots/123456789:insertYourAuthorizationTokenHere")
    .process(exchange -> {

        OutgoingTextMessage msg = new OutgoingTextMessage();
        msg.setText("Your answer was accepted!");

        ReplyKeyboardMarkup replyMarkup = ReplyKeyboardMarkup.builder()
                .removeKeyboard(true)
                .build();

        msg.setReplyKeyboardMarkup(replyMarkup);

        exchange.getIn().setBody(msg);
    })
    .to("telegram:bots/123456789:insertYourAuthorizationTokenHere");
---------------------------------------------------------

== Webhook Mode

The Telegram component supports usage in the *webhook mode* using the *camel-webhook* component.

In order to enable webhook mode, users need first to add a REST implementation to their application.
Maven users, for example, can add *netty-http* to their `pom.xml` file:

[source,xml]
------------------------------------------------------------
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-netty-http</artifactId>
    <version>x.x.x</version>
    <!-- use the same version as your Camel core version -->
</dependency>
------------------------------------------------------------

Once done, you need to prepend the webhook URI to the telegram URI you want to use.

In Java DSL:

[source,java]
---------------------------------------------------------
from("webhook:telegram:bots/123456789:insertYourAuthorizationTokenHere").to("log:info");
---------------------------------------------------------

Some endpoints will be exposed by your application and Telegram will be configured to send messages to them.
You need to ensure that your server is exposed to the internet and to pass the right value of the
*camel.component.webhook.configuration.webhook-external-url* property.

Refer to the *camel-webhook* component documentation for instructions on how to set it.


Fix malformed JSON in Delete API example (#21168)

Obviously, there should be

    "result": "deleted"

instead of

    "result: deleted"

[[docs-delete]]
== Delete API

The delete API allows to delete a typed JSON document from a specific
index based on its id. The following example deletes the JSON document
from an index called twitter, under a type called tweet, with id valued
1:

[source,js]
--------------------------------------------------
$ curl -XDELETE 'http://localhost:9200/twitter/tweet/1'
--------------------------------------------------

The result of the above delete operation is:

[source,js]
--------------------------------------------------
{
    "_shards" : {
        "total" : 10,
        "failed" : 0,
        "successful" : 10
    },
    "found" : true,
    "_index" : "twitter",
    "_type" : "tweet",
    "_id" : "1",
    "_version" : 2,
    "result": "deleted"
}
--------------------------------------------------

[float]
[[delete-versioning]]
=== Versioning

Each document indexed is versioned. When deleting a document, the
`version` can be specified to make sure the relevant document we are
trying to delete is actually being deleted and it has not changed in the
meantime. Every write operation executed on a document, deletes included,
causes its version to be incremented.

[float]
[[delete-routing]]
=== Routing

When indexing using the ability to control the routing, in order to
delete a document, the routing value should also be provided. For
example:

[source,js]
--------------------------------------------------
$ curl -XDELETE 'http://localhost:9200/twitter/tweet/1?routing=kimchy'
--------------------------------------------------

The above will delete a tweet with id 1, but will be routed based on the
user. Note, issuing a delete without the correct routing, will cause the
document to not be deleted.

When the `_routing` mapping is set as `required` and no routing value is
specified, the delete api will throw a `RoutingMissingException` and reject
the request.

[float]
[[delete-parent]]
=== Parent

The `parent` parameter can be set, which will basically be the same as
setting the routing parameter.

Note that deleting a parent document does not automatically delete its
children. One way of deleting all child documents given a parent's id is
to use the <<docs-delete-by-query,Delete By Query API>> to perform a
 index with the automatically generated (and indexed)
field _parent, which is in the format parent_type#parent_id.

When deleting a child document its parent id must be specified, otherwise
the delete request will be rejected and a `RoutingMissingException` will be
thrown instead.

[float]
[[delete-index-creation]]
=== Automatic index creation

The delete operation automatically creates an index if it has not been
created before (check out the <<indices-create-index,create index API>>
for manually creating an index), and also automatically creates a
dynamic type mapping for the specific type if it has not been created
before (check out the <<indices-put-mapping,put mapping>>
API for manually creating type mapping).

[float]
[[delete-distributed]]
=== Distributed

The delete operation gets hashed into a specific shard id. It then gets
redirected into the primary shard within that id group, and replicated
(if needed) to shard replicas within that id group.

[float]
[[delete-wait-for-active-shards]]
=== Wait For Active Shards

When making delete requests, you can set the `wait_for_active_shards`
parameter to require a minimum number of shard copies to be active
before starting to process the delete request. See
<<index-wait-for-active-shards,here>> for further details and a usage
example.

[float]
[[delete-refresh]]
=== Refresh

Control when the changes made by this request are visible to search. See
<<docs-refresh>>.

[float]
[[delete-timeout]]
=== Timeout

The primary shard assigned to perform the delete operation might not be
available when the delete operation is executed. Some reasons for this
might be that the primary shard is currently recovering from a store
or undergoing relocation. By default, the delete operation will wait on
the primary shard to become available for up to 1 minute before failing
and responding with an error. The `timeout` parameter can be used to
explicitly specify how long it waits. Here is an example of setting it
to 5 minutes:

[source,js]
--------------------------------------------------
$ curl -XDELETE 'http://localhost:9200/twitter/tweet/1?timeout=5m'
--------------------------------------------------

[[docs-delete]]
== Delete API

The delete API allows to delete a typed JSON document from a specific
index based on its id. The following example deletes the JSON document
from an index called twitter, under a type called tweet, with id valued
1:

[source,js]
--------------------------------------------------
$ curl -XDELETE 'http://localhost:9200/twitter/tweet/1'
--------------------------------------------------

The result of the above delete operation is:

[source,js]
--------------------------------------------------
{
    "_shards" : {
        "total" : 10,
        "failed" : 0,
        "successful" : 10
    },
    "found" : true,
    "_index" : "twitter",
    "_type" : "tweet",
    "_id" : "1",
    "_version" : 2,
    "result: deleted"
}
--------------------------------------------------

[float]
[[delete-versioning]]
=== Versioning

Each document indexed is versioned. When deleting a document, the
`version` can be specified to make sure the relevant document we are
trying to delete is actually being deleted and it has not changed in the
meantime. Every write operation executed on a document, deletes included,
causes its version to be incremented.

[float]
[[delete-routing]]
=== Routing

When indexing using the ability to control the routing, in order to
delete a document, the routing value should also be provided. For
example:

[source,js]
--------------------------------------------------
$ curl -XDELETE 'http://localhost:9200/twitter/tweet/1?routing=kimchy'
--------------------------------------------------

The above will delete a tweet with id 1, but will be routed based on the
user. Note, issuing a delete without the correct routing, will cause the
document to not be deleted.

When the `_routing` mapping is set as `required` and no routing value is
specified, the delete api will throw a `RoutingMissingException` and reject
the request.

[float]
[[delete-parent]]
=== Parent

The `parent` parameter can be set, which will basically be the same as
setting the routing parameter.

Note that deleting a parent document does not automatically delete its
children. One way of deleting all child documents given a parent's id is
to use the <<docs-delete-by-query,Delete By Query API>> to perform a
 index with the automatically generated (and indexed)
field _parent, which is in the format parent_type#parent_id.

When deleting a child document its parent id must be specified, otherwise
the delete request will be rejected and a `RoutingMissingException` will be
thrown instead.

[float]
[[delete-index-creation]]
=== Automatic index creation

The delete operation automatically creates an index if it has not been
created before (check out the <<indices-create-index,create index API>>
for manually creating an index), and also automatically creates a
dynamic type mapping for the specific type if it has not been created
before (check out the <<indices-put-mapping,put mapping>>
API for manually creating type mapping).

[float]
[[delete-distributed]]
=== Distributed

The delete operation gets hashed into a specific shard id. It then gets
redirected into the primary shard within that id group, and replicated
(if needed) to shard replicas within that id group.

[float]
[[delete-wait-for-active-shards]]
=== Wait For Active Shards

When making delete requests, you can set the `wait_for_active_shards`
parameter to require a minimum number of shard copies to be active
before starting to process the delete request. See
<<index-wait-for-active-shards,here>> for further details and a usage
example.

[float]
[[delete-refresh]]
=== Refresh

Control when the changes made by this request are visible to search. See
<<docs-refresh>>.

[float]
[[delete-timeout]]
=== Timeout

The primary shard assigned to perform the delete operation might not be
available when the delete operation is executed. Some reasons for this
might be that the primary shard is currently recovering from a store
or undergoing relocation. By default, the delete operation will wait on
the primary shard to become available for up to 1 minute before failing
and responding with an error. The `timeout` parameter can be used to
explicitly specify how long it waits. Here is an example of setting it
to 5 minutes:

[source,js]
--------------------------------------------------
$ curl -XDELETE 'http://localhost:9200/twitter/tweet/1?timeout=5m'
--------------------------------------------------


[Docs] Fix typo in index API documentation (#43740)

[[docs-index_]]
== Index API

IMPORTANT: See <<removal-of-types>>.

The index API adds or updates a typed JSON document in a specific index,
making it searchable. The following example inserts the JSON document
into the "twitter" index, under a type called `_doc` with an id of 1:

[source,js]
--------------------------------------------------
PUT twitter/_doc/1
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
--------------------------------------------------
// CONSOLE

The result of the above index operation is:

[source,js]
--------------------------------------------------
{
    "_shards" : {
        "total" : 2,
        "failed" : 0,
        "successful" : 2
    },
    "_index" : "twitter",
    "_type" : "_doc",
    "_id" : "1",
    "_version" : 1,
    "_seq_no" : 0,
    "_primary_term" : 1,
    "result" : "created"
}
--------------------------------------------------
// TESTRESPONSE[s/"successful" : 2/"successful" : 1/]

The `_shards` header provides information about the replication process of the index operation:

`total`:: Indicates how many shard copies (primary and replica shards) the index operation should be executed on.
`successful`:: Indicates the number of shard copies the index operation succeeded on.
`failed`:: An array that contains replication-related errors in the case an index operation failed on a replica shard.

The index operation is successful in the case `successful` is at least 1.

NOTE:   Replica shards may not all be started when an indexing operation successfully returns (by default, only the
        primary is required, but this behavior can be <<index-wait-for-active-shards,changed>>). In that case,
        `total` will be equal to the total shards based on the `number_of_replicas` setting and `successful` will be
        equal to the number of shards started (primary plus replicas). If there were no failures, the `failed` will be 0.

[float]
[[index-creation]]
=== Automatic Index Creation

The index operation automatically creates an index if it does not already
exist, and applies any <<indices-templates,index templates>> that are
configured. The index operation also creates a dynamic type mapping for the
specified type if one does not already exist. By default, new fields and
objects will automatically be added to the mapping definition for the specified
type if needed. Check out the <<mapping,mapping>> section for more information
on mapping definitions, and the <<indices-put-mapping,put mapping>> API for
information about updating type mappings manually.

Automatic index creation is controlled by the `action.auto_create_index`
setting. This setting defaults to `true`, meaning that indices are always
automatically created. Automatic index creation can be permitted only for
indices matching certain patterns by changing the value of this setting to a
comma-separated list of these patterns. It can also be explicitly permitted and
forbidden by prefixing patterns in the list with a `+` or `-`.  Finally it can
be completely disabled by changing this setting to `false`.

[source,js]
--------------------------------------------------
PUT _cluster/settings
{
    "persistent": {
        "action.auto_create_index": "twitter,index10,-index1*,+ind*" <1>
    }
}

PUT _cluster/settings
{
    "persistent": {
        "action.auto_create_index": "false" <2>
    }
}

PUT _cluster/settings
{
    "persistent": {
        "action.auto_create_index": "true" <3>
    }
}
--------------------------------------------------
// CONSOLE

<1> Permit only the auto-creation of indices called `twitter`, `index10`, no
other index matching `index1*`, and any other index matching `ind*`. The
patterns are matched in the order in which they are given.

<2> Completely disable the auto-creation of indices.

<3> Permit the auto-creation of indices with any name. This is the default.

[float]
[[operation-type]]
=== Operation Type

The index operation also accepts an `op_type` that can be used to force
a `create` operation, allowing for "put-if-absent" behavior. When
`create` is used, the index operation will fail if a document by that id
already exists in the index.

Here is an example of using the `op_type` parameter:

[source,js]
--------------------------------------------------
PUT twitter/_doc/1?op_type=create
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
--------------------------------------------------
// CONSOLE

Another option to specify `create` is to use the following uri:

[source,js]
--------------------------------------------------
PUT twitter/_doc/1/_create
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
--------------------------------------------------
// CONSOLE

[float]
=== Automatic ID Generation

The index operation can be executed without specifying the id. In such a
case, an id will be generated automatically. In addition, the `op_type`
will automatically be set to `create`. Here is an example (note the
*POST* used instead of *PUT*):

[source,js]
--------------------------------------------------
POST twitter/_doc/
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
--------------------------------------------------
// CONSOLE

The result of the above index operation is:

[source,js]
--------------------------------------------------
{
    "_shards" : {
        "total" : 2,
        "failed" : 0,
        "successful" : 2
    },
    "_index" : "twitter",
    "_type" : "_doc",
    "_id" : "W0tpsmIBdwcYyG50zbta",
    "_version" : 1,
    "_seq_no" : 0,
    "_primary_term" : 1,
    "result": "created"
}
--------------------------------------------------
// TESTRESPONSE[s/W0tpsmIBdwcYyG50zbta/$body._id/ s/"successful" : 2/"successful" : 1/]

[float]
[[optimistic-concurrency-control-index]]
=== Optimistic concurrency control

Index operations can be made conditional and only be performed if the last
modification to the document was assigned the sequence number and primary
term specified by the `if_seq_no` and `if_primary_term` parameters. If a
mismatch is detected, the operation will result in a `VersionConflictException`
and a status code of 409. See <<optimistic-concurrency-control>> for more details.

[float]
[[index-routing]]
=== Routing

By default, shard placement — or `routing` — is controlled by using a
hash of the document's id value. For more explicit control, the value
fed into the hash function used by the router can be directly specified
on a per-operation basis using the `routing` parameter. For example:

[source,js]
--------------------------------------------------
POST twitter/_doc?routing=kimchy
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
--------------------------------------------------
// CONSOLE

In the example above, the "_doc" document is routed to a shard based on
the `routing` parameter provided: "kimchy".

When setting up explicit mapping, the `_routing` field can be optionally
used to direct the index operation to extract the routing value from the
document itself. This does come at the (very minimal) cost of an
additional document parsing pass. If the `_routing` mapping is defined
and set to be `required`, the index operation will fail if no routing
value is provided or extracted.

[float]
[[index-distributed]]
=== Distributed

The index operation is directed to the primary shard based on its route
(see the Routing section above) and performed on the actual node
containing this shard. After the primary shard completes the operation,
if needed, the update is distributed to applicable replicas.

[float]
[[index-wait-for-active-shards]]
=== Wait For Active Shards

To improve the resiliency of writes to the system, indexing operations
can be configured to wait for a certain number of active shard copies
before proceeding with the operation. If the requisite number of active
shard copies are not available, then the write operation must wait and
retry, until either the requisite shard copies have started or a timeout
occurs. By default, write operations only wait for the primary shards
to be active before proceeding (i.e. `wait_for_active_shards=1`).
This default can be overridden in the index settings dynamically
by setting `index.write.wait_for_active_shards`. To alter this behavior
per operation, the `wait_for_active_shards` request parameter can be used.

Valid values are `all` or any positive integer up to the total number
of configured copies per shard in the index (which is `number_of_replicas+1`).
Specifying a negative value or a number greater than the number of
shard copies will throw an error.

For example, suppose we have a cluster of three nodes, `A`, `B`, and `C` and
we create an index `index` with the number of replicas set to 3 (resulting in
4 shard copies, one more copy than there are nodes). If we
attempt an indexing operation, by default the operation will only ensure
the primary copy of each shard is available before proceeding. This means
that even if `B` and `C` went down, and `A` hosted the primary shard copies,
the indexing operation would still proceed with only one copy of the data.
If `wait_for_active_shards` is set on the request to `3` (and all 3 nodes
are up), then the indexing operation will require 3 active shard copies
before proceeding, a requirement which should be met because there are 3
active nodes in the cluster, each one holding a copy of the shard. However,
if we set `wait_for_active_shards` to `all` (or to `4`, which is the same),
the indexing operation will not proceed as we do not have all 4 copies of
each shard active in the index. The operation will timeout
unless a new node is brought up in the cluster to host the fourth copy of
the shard.

It is important to note that this setting greatly reduces the chances of
the write operation not writing to the requisite number of shard copies,
but it does not completely eliminate the possibility, because this check
occurs before the write operation commences. Once the write operation
is underway, it is still possible for replication to fail on any number of
shard copies but still succeed on the primary. The `_shards` section of the
write operation's response reveals the number of shard copies on which
replication succeeded/failed.

[source,js]
--------------------------------------------------
{
    "_shards" : {
        "total" : 2,
        "failed" : 0,
        "successful" : 2
    }
}
--------------------------------------------------
// NOTCONSOLE

[float]
[[index-refresh]]
=== Refresh

Control when the changes made by this request are visible to search. See
<<docs-refresh,refresh>>.

[float]
[[index-noop]]
=== Noop Updates

When updating a document using the index API a new version of the document is
always created even if the document hasn't changed. If this isn't acceptable
use the `_update` API with `detect_noop` set to true. This option isn't
available on the index API because the index API doesn't fetch the old source
and isn't able to compare it against the new source.

There isn't a hard and fast rule about when noop updates aren't acceptable.
It's a combination of lots of factors like how frequently your data source
sends updates that are actually noops and how many queries per second
Elasticsearch runs on the shard receiving the updates.

[float]
[[timeout]]
=== Timeout

The primary shard assigned to perform the index operation might not be
available when the index operation is executed. Some reasons for this
might be that the primary shard is currently recovering from a gateway
or undergoing relocation. By default, the index operation will wait on
the primary shard to become available for up to 1 minute before failing
and responding with an error. The `timeout` parameter can be used to
explicitly specify how long it waits. Here is an example of setting it
to 5 minutes:

[source,js]
--------------------------------------------------
PUT twitter/_doc/1?timeout=5m
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
--------------------------------------------------
// CONSOLE

[float]
[[index-versioning]]
=== Versioning

Each indexed document is given a version number. By default,
internal versioning is used that starts at 1 and increments
with each update, deletes included. Optionally, the version number can be
set to an external value (for example, if maintained in a
database). To enable this functionality, `version_type` should be set to
`external`. The value provided must be a numeric, long value greater than or equal to 0,
and less than around 9.2e+18.

When using the external version type, the system checks to see if
the version number passed to the index request is greater than the
version of the currently stored document. If true, the document will be
indexed and the new version number used. If the value provided is less
than or equal to the stored document's version number, a version
conflict will occur and the index operation will fail. For example:

[source,js]
--------------------------------------------------
PUT twitter/_doc/1?version=2&version_type=external
{
    "message" : "elasticsearch now has versioning support, double cool!"
}
--------------------------------------------------
// CONSOLE
// TEST[continued]

*NOTE:* Versioning is completely real time, and is not affected by the
near real time aspects of search operations. If no version is provided,
then the operation is executed without any version checks.

The above will succeed since the supplied version of 2 is higher than
the current document version of 1. If the document was already updated
and its version was set to 2 or higher, the indexing command will fail
and result in a conflict (409 http status code).

A nice side effect is that there is no need to maintain strict ordering
of async indexing operations executed as a result of changes to a source
database, as long as version numbers from the source database are used.
Even the simple case of updating the Elasticsearch index using data from
a database is simplified if external versioning is used, as only the
latest version will be used if the index operations arrive out of order for
whatever reason.

[float]
==== Version types

Next to the `external` version type explained above, Elasticsearch
also supports other types for specific use cases. Here is an overview of
the different version types and their semantics.

`internal`:: Only index the document if the given version is identical to the version
of the stored document.
ifdef::asciidoctor[]
deprecated:[6.7.0, "Please use `if_seq_no` & `if_primary_term` instead. See <<optimistic-concurrency-control>> for more details."]
endif::[]
ifndef::asciidoctor[]
deprecated[6.7.0, Please use `if_seq_no` & `if_primary_term` instead. See <<optimistic-concurrency-control>> for more details.]
endif::[]

`external` or `external_gt`:: Only index the document if the given version is strictly higher
than the version of the stored document *or* if there is no existing document. The given
version will be used as the new version and will be stored with the new document. The supplied
version must be a non-negative long number.

`external_gte`:: Only index the document if the given version is *equal* or higher
than the version of the stored document. If there is no existing document
the operation will succeed as well. The given version will be used as the new version
and will be stored with the new document. The supplied version must be a non-negative long number.

*NOTE*: The `external_gte` version type is meant for special use cases and
should be used with care. If used incorrectly, it can result in loss of data.
There is another option, `force`, which is deprecated because it can cause
primary and replica shards to diverge.

[[docs-index_]]
== Index API

IMPORTANT: See <<removal-of-types>>.

The index API adds or updates a typed JSON document in a specific index,
making it searchable. The following example inserts the JSON document
into the "twitter" index, under a type called `_doc` with an id of 1:

[source,js]
--------------------------------------------------
PUT twitter/_doc/1
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
--------------------------------------------------
// CONSOLE

The result of the above index operation is:

[source,js]
--------------------------------------------------
{
    "_shards" : {
        "total" : 2,
        "failed" : 0,
        "successful" : 2
    },
    "_index" : "twitter",
    "_type" : "_doc",
    "_id" : "1",
    "_version" : 1,
    "_seq_no" : 0,
    "_primary_term" : 1,
    "result" : "created"
}
--------------------------------------------------
// TESTRESPONSE[s/"successful" : 2/"successful" : 1/]

The `_shards` header provides information about the replication process of the index operation:

`total`:: Indicates how many shard copies (primary and replica shards) the index operation should be executed on.
`successful`:: Indicates the number of shard copies the index operation succeeded on.
`failed`:: An array that contains replication-related errors in the case an index operation failed on a replica shard.

The index operation is successful in the case `successful` is at least 1.

NOTE:   Replica shards may not all be started when an indexing operation successfully returns (by default, only the
        primary is required, but this behavior can be <<index-wait-for-active-shards,changed>>). In that case,
        `total` will be equal to the total shards based on the `number_of_replicas` setting and `successful` will be
        equal to the number of shards started (primary plus replicas). If there were no failures, the `failed` will be 0.

[float]
[[index-creation]]
=== Automatic Index Creation

The index operation automatically creates an index if it does not already
exist, and applies any <<indices-templates,index templates>> that are
configured. The index operation also creates a dynamic type mapping for the
specified type if one does not already exist. By default, new fields and
objects will automatically be added to the mapping definition for the specified
type if needed. Check out the <<mapping,mapping>> section for more information
on mapping definitions, and the <<indices-put-mapping,put mapping>> API for
information about updating type mappings manually.

Automatic index creation is controlled by the `action.auto_create_index`
setting. This setting defaults to `true`, meaning that indices are always
automatically created. Automatic index creation can be permitted only for
indices matching certain patterns by changing the value of this setting to a
comma-separated list of these patterns. It can also be explicitly permitted and
forbidden by prefixing patterns in the list with a `+` or `-`.  Finally it can
be completely disabled by changing this setting to `false`.

[source,js]
--------------------------------------------------
PUT _cluster/settings
{
    "persistent": {
        "action.auto_create_index": "twitter,index10,-index1*,+ind*" <1>
    }
}

PUT _cluster/settings
{
    "persistent": {
        "action.auto_create_index": "false" <2>
    }
}

PUT _cluster/settings
{
    "persistent": {
        "action.auto_create_index": "true" <3>
    }
}
--------------------------------------------------
// CONSOLE

<1> Permit only the auto-creation of indices called `twitter`, `index10`, no
other index matching `index1*`, and any other index matching `ind*`. The
patterns are matched in the order in which they are given.

<2> Completely disable the auto-creation of indices.

<3> Permit the auto-creation of indices with any name. This is the default.

[float]
[[operation-type]]
=== Operation Type

The index operation also accepts an `op_type` that can be used to force
a `create` operation, allowing for "put-if-absent" behavior. When
`create` is used, the index operation will fail if a document by that id
already exists in the index.

Here is an example of using the `op_type` parameter:

[source,js]
--------------------------------------------------
PUT twitter/_doc/1?op_type=create
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
--------------------------------------------------
// CONSOLE

Another option to specify `create` is to use the following uri:

[source,js]
--------------------------------------------------
PUT twitter/_doc/1/_create
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
--------------------------------------------------
// CONSOLE

[float]
=== Automatic ID Generation

The index operation can be executed without specifying the id. In such a
case, an id will be generated automatically. In addition, the `op_type`
will automatically be set to `create`. Here is an example (note the
*POST* used instead of *PUT*):

[source,js]
--------------------------------------------------
POST twitter/_doc/
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
--------------------------------------------------
// CONSOLE

The result of the above index operation is:

[source,js]
--------------------------------------------------
{
    "_shards" : {
        "total" : 2,
        "failed" : 0,
        "successful" : 2
    },
    "_index" : "twitter",
    "_type" : "_doc",
    "_id" : "W0tpsmIBdwcYyG50zbta",
    "_version" : 1,
    "_seq_no" : 0,
    "_primary_term" : 1,
    "result": "created"
}
--------------------------------------------------
// TESTRESPONSE[s/W0tpsmIBdwcYyG50zbta/$body._id/ s/"successful" : 2/"successful" : 1/]

[float]
[[optimistic-concurrency-control-index]]
=== Optimistic concurrency control

Index operations can be made conditional and only be performed if the last
modification to the document was assigned the sequence number and primary
term specified by the `if_seq_no` and `if_primary_term` parameters. If a
mismatch is detected, the operation will result in a `VersionConflictException`
and a status code of 409. See <<optimistic-concurrency-control>> for more details.

[float]
[[index-routing]]
=== Routing

By default, shard placement — or `routing` — is controlled by using a
hash of the document's id value. For more explicit control, the value
fed into the hash function used by the router can be directly specified
on a per-operation basis using the `routing` parameter. For example:

[source,js]
--------------------------------------------------
POST twitter/_doc?routing=kimchy
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
--------------------------------------------------
// CONSOLE

In the example above, the "_doc" document is routed to a shard based on
the `routing` parameter provided: "kimchy".

When setting up explicit mapping, the `_routing` field can be optionally
used to direct the index operation to extract the routing value from the
document itself. This does come at the (very minimal) cost of an
additional document parsing pass. If the `_routing` mapping is defined
and set to be `required`, the index operation will fail if no routing
value is provided or extracted.

[float]
[[index-distributed]]
=== Distributed

The index operation is directed to the primary shard based on its route
(see the Routing section above) and performed on the actual node
containing this shard. After the primary shard completes the operation,
if needed, the update is distributed to applicable replicas.

[float]
[[index-wait-for-active-shards]]
=== Wait For Active Shards

To improve the resiliency of writes to the system, indexing operations
can be configured to wait for a certain number of active shard copies
before proceeding with the operation. If the requisite number of active
shard copies are not available, then the write operation must wait and
retry, until either the requisite shard copies have started or a timeout
occurs. By default, write operations only wait for the primary shards
to be active before proceeding (i.e. `wait_for_active_shards=1`).
This default can be overridden in the index settings dynamically
by setting `index.write.wait_for_active_shards`. To alter this behavior
per operation, the `wait_for_active_shards` request parameter can be used.

Valid values are `all` or any positive integer up to the total number
of configured copies per shard in the index (which is `number_of_replicas+1`).
Specifying a negative value or a number greater than the number of
shard copies will throw an error.

For example, suppose we have a cluster of three nodes, `A`, `B`, and `C` and
we create an index `index` with the number of replicas set to 3 (resulting in
4 shard copies, one more copy than there are nodes). If we
attempt an indexing operation, by default the operation will only ensure
the primary copy of each shard is available before proceeding. This means
that even if `B` and `C` went down, and `A` hosted the primary shard copies,
the indexing operation would still proceed with only one copy of the data.
If `wait_for_active_shards` is set on the request to `3` (and all 3 nodes
are up), then the indexing operation will require 3 active shard copies
before proceeding, a requirement which should be met because there are 3
active nodes in the cluster, each one holding a copy of the shard. However,
if we set `wait_for_active_shards` to `all` (or to `4`, which is the same),
the indexing operation will not proceed as we do not have all 4 copies of
each shard active in the index. The operation will timeout
unless a new node is brought up in the cluster to host the fourth copy of
the shard.

It is important to note that this setting greatly reduces the chances of
the write operation not writing to the requisite number of shard copies,
but it does not completely eliminate the possibility, because this check
occurs before the write operation commences. Once the write operation
is underway, it is still possible for replication to fail on any number of
shard copies but still succeed on the primary. The `_shards` section of the
write operation's response reveals the number of shard copies on which
replication succeeded/failed.

[source,js]
--------------------------------------------------
{
    "_shards" : {
        "total" : 2,
        "failed" : 0,
        "successful" : 2
    }
}
--------------------------------------------------
// NOTCONSOLE

[float]
[[index-refresh]]
=== Refresh

Control when the changes made by this request are visible to search. See
<<docs-refresh,refresh>>.

[float]
[[index-noop]]
=== Noop Updates

When updating a document using the index API a new version of the document is
always created even if the document hasn't changed. If this isn't acceptable
use the `_update` API with `detect_noop` set to true. This option isn't
available on the index API because the index API doesn't fetch the old source
and isn't able to compare it against the new source.

There isn't a hard and fast rule about when noop updates aren't acceptable.
It's a combination of lots of factors like how frequently your data source
sends updates that are actually noops and how many queries per second
Elasticsearch runs on the shard receiving the updates.

[float]
[[timeout]]
=== Timeout

The primary shard assigned to perform the index operation might not be
available when the index operation is executed. Some reasons for this
might be that the primary shard is currently recovering from a gateway
or undergoing relocation. By default, the index operation will wait on
the primary shard to become available for up to 1 minute before failing
and responding with an error. The `timeout` parameter can be used to
explicitly specify how long it waits. Here is an example of setting it
to 5 minutes:

[source,js]
--------------------------------------------------
PUT twitter/_doc/1?timeout=5m
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
--------------------------------------------------
// CONSOLE

[float]
[[index-versioning]]
=== Versioning

Each indexed document is given a version number. By default,
internal versioning is used that starts at 1 and increments
with each update, deletes included. Optionally, the version number can be
set to an external value (for example, if maintained in a
database). To enable this functionality, `version_type` should be set to
`external`. The value provided must be a numeric, long value greater than or equal to 0,
and less than around 9.2e+18.

When using the external version type, the system checks to see if
the version number passed to the index request is greater than the
version of the currently stored document. If true, the document will be
indexed and the new version number used. If the value provided is less
than or equal to the stored document's version number, a version
conflict will occur and the index operation will fail. For example:

[source,js]
--------------------------------------------------
PUT twitter/_doc/1?version=2&version_type=external
{
    "message" : "elasticsearch now has versioning support, double cool!"
}
--------------------------------------------------
// CONSOLE
// TEST[continued]

*NOTE:* Versioning is completely real time, and is not affected by the
near real time aspects of search operations. If no version is provided,
then the operation is executed without any version checks.

The above will succeed since the the supplied version of 2 is higher than
the current document version of 1. If the document was already updated
and its version was set to 2 or higher, the indexing command will fail
and result in a conflict (409 http status code).

A nice side effect is that there is no need to maintain strict ordering
of async indexing operations executed as a result of changes to a source
database, as long as version numbers from the source database are used.
Even the simple case of updating the Elasticsearch index using data from
a database is simplified if external versioning is used, as only the
latest version will be used if the index operations arrive out of order for
whatever reason.

[float]
==== Version types

Next to the `external` version type explained above, Elasticsearch
also supports other types for specific use cases. Here is an overview of
the different version types and their semantics.

`internal`:: Only index the document if the given version is identical to the version
of the stored document.
ifdef::asciidoctor[]
deprecated:[6.7.0, "Please use `if_seq_no` & `if_primary_term` instead. See <<optimistic-concurrency-control>> for more details."]
endif::[]
ifndef::asciidoctor[]
deprecated[6.7.0, Please use `if_seq_no` & `if_primary_term` instead. See <<optimistic-concurrency-control>> for more details.]
endif::[]

`external` or `external_gt`:: Only index the document if the given version is strictly higher
than the version of the stored document *or* if there is no existing document. The given
version will be used as the new version and will be stored with the new document. The supplied
version must be a non-negative long number.

`external_gte`:: Only index the document if the given version is *equal* or higher
than the version of the stored document. If there is no existing document
the operation will succeed as well. The given version will be used as the new version
and will be stored with the new document. The supplied version must be a non-negative long number.

*NOTE*: The `external_gte` version type is meant for special use cases and
should be used with care. If used incorrectly, it can result in loss of data.
There is another option, `force`, which is deprecated because it can cause
primary and replica shards to diverge.


Update changelog.

Update changelog.

= Changelog

== Version 0.6.0

Date: 2015-06-28

- Update to buddy-core 0.6.0

== Version 0.5.0

Date: 2015-06-15

- `check` function is now null pointer safe.

== Version 0.4.2

Date: 2015-04-03

- Update buddy-core to 0.5.0

== Version 0.4.1

Date: 2015-03-14

- Update buddy-core from 0.4.0 to 0.4.2

== Version 0.4.0

Date: 2015-02-22

- Update buddy-core dependency version to 0.4.0
- Adapt the code to buddy-core 0.4.0

== Version 0.3.0

Date: 2015-01-18

- First version splitted from monolitic buddy package.
- Add complete refactored version of hashers, more flexible and extensible.
- Add support for pbkdf2+sha256 and pbkdf2+sha3_256 password hasher algorithms.
- Maintain the old namespace for backward compatibility.

= Changelog

== Version 0.5.0

Date: 2015-06-15

- `check` function is now null pointer safe.

== Version 0.4.2

Date: 2015-04-03

- Update buddy-core to 0.5.0

== Version 0.4.1

Date: 2015-03-14

- Update buddy-core from 0.4.0 to 0.4.2

== Version 0.4.0

Date: 2015-02-22

- Update buddy-core dependency version to 0.4.0
- Adapt the code to buddy-core 0.4.0

== Version 0.3.0

Date: 2015-01-18

- First version splitted from monolitic buddy package.
- Add complete refactored version of hashers, more flexible and extensible.
- Add support for pbkdf2+sha256 and pbkdf2+sha3_256 password hasher algorithms.
- Maintain the old namespace for backward compatibility.


Update 2016-07-18-Tracking-tests-network-statistics-with-Docker.adoc


Fix AsciiDoc issue

Fix AsciiDoc issue

Using PIV for SSH through PKCS11
--------------------------------

This is a step-by-step for how to get a YubiKey with PIV to work for
public-key authentication with OpenSSH through PKCS11.
Primarily on a OS X or Linux system.

Prerequisites
-------------

* a YubiKey with the PIV application loaded
* the yubico-piv-tool software
* the OpenSC software
* OpenSSH
** If you are using OSX El Capitan (10.11) or earlier, for ssh-agent to work a newer OpenSSH than is delivered with the system; macOS Sierra (10.12) contains a compatible version

[NOTE]
The following example assume that you have not yet changed the management key. If you have changed the management key, add `--key` to the `yubico-piv-tool -a import-certificate` command below.

Steps
-----

1. Generate a key in slot 9a (any slot should suffice):

  $ yubico-piv-tool -s 9a -a generate -o public.pem
+
[NOTE]
RSA 4096-bit keys are not currently supported due to a limitation in the PIV spec: https://github.com/Yubico/yubico-piv-tool/issues/58

2. Create a self-signed certificate for that key.
The only use for the X.509 certificate is to make PIV/PKCS#11 lib happy.
They would want to be able to extract the public-key from the smartcard,
and do that through the X.509 certificate.

  $ yubico-piv-tool -a verify-pin -a selfsign-certificate -s 9a -S "/CN=SSH key/" -i public.pem -o cert.pem

3. Load the certificate:

   $ yubico-piv-tool -a import-certificate -s 9a -i cert.pem

4. Find out where OpenSC has installed the pkcs11 module.

  * For OS X with binary installation this is typically in `/Library/OpenSC/lib/`. Homebrew users can use `export OPENSC_LIBS=$(brew --prefix opensc)/lib`.

  * For a Debian based system this is typically in `/usr/lib/x86_64-linux-gnu/`
+
After this we'll call this location `$OPENSC_LIBS`

5. Export the public key in correct format for ssh and once you got it,
add it to authorized_keys on the target system.

   $ ssh-keygen -D $OPENSC_LIBS/opensc-pkcs11.so -e
+
[NOTE]
The command will export all keys stored on the YubiKey Neo.
Hopefully it will keep the slot order so it should be not hard to guess which
is the public key associated with your targeted private key.

6. Authenticate to the target system using the new key:

   $ ssh -I $OPENSC_LIBS/opensc-pkcs11.so user@remote.example.com

7. This can also be setup to work with ssh-agent: (Optional)

   $ ssh-add -s $OPENSC_LIBS/opensc-pkcs11.so
+
NOTE: On OS X prior to macOS 10.12 “Sierra” this typically requires installation of a third-party OpenSSH from Homebrew or the like and using that ssh-agent.
+
To confirm that the ssh-agent correctly finds that key and getting the public key in correct format:

   $ ssh-add -L

Using PIV for SSH through PKCS11
--------------------------------

This is a step-by-step for how to get a YubiKey with PIV to work for
public-key authentication with OpenSSH through PKCS11.
Primarily on a OS X or Linux system.

Prerequisites
-------------

* a YubiKey with the PIV application loaded
* the yubico-piv-tool software
* the OpenSC software
* OpenSSH
** If you are using OSX El Capitan (10.11) or earlier, for ssh-agent to work a newer OpenSSH than is delivered with the system; macOS Sierra (10.12) contains a compatible version

[NOTE]
The following example assume that you have not yet changed the management key. If you have changed the management key, add `--key` to the `yubico-piv-tool -a import-certificate` command below.

Steps
-----

1. Generate a key in slot 9a (any slot should suffice):

  $ yubico-piv-tool -s 9a -a generate -o public.pem

[NOTE]
RSA 4096-bit keys are not currently supported due to a limitation in the PIV spec: https://github.com/Yubico/yubico-piv-tool/issues/58

2. Create a self-signed certificate for that key.
The only use for the X.509 certificate is to make PIV/PKCS#11 lib happy.
They would want to be able to extract the public-key from the smartcard,
and do that through the X.509 certificate.

  $ yubico-piv-tool -a verify-pin -a selfsign-certificate -s 9a -S "/CN=SSH key/" -i public.pem -o cert.pem

3. Load the certificate:

   $ yubico-piv-tool -a import-certificate -s 9a -i cert.pem

4. Find out where OpenSC has installed the pkcs11 module.

  * For OS X with binary installation this is typically in `/Library/OpenSC/lib/`. Homebrew users can use `export OPENSC_LIBS=$(brew --prefix opensc)/lib`.

  * For a Debian based system this is typically in `/usr/lib/x86_64-linux-gnu/`
+
After this we'll call this location `$OPENSC_LIBS`

5. Export the public key in correct format for ssh and once you got it,
add it to authorized_keys on the target system.

   $ ssh-keygen -D $OPENSC_LIBS/opensc-pkcs11.so -e
+
[NOTE]
The command will export all keys stored on the YubiKey Neo.
Hopefully it will keep the slot order so it should be not hard to guess which
is the public key associated with your targeted private key.

6. Authenticate to the target system using the new key:

   $ ssh -I $OPENSC_LIBS/opensc-pkcs11.so user@remote.example.com

7. This can also be setup to work with ssh-agent: (Optional)

   $ ssh-add -s $OPENSC_LIBS/opensc-pkcs11.so
+
NOTE: On OS X prior to macOS 10.12 “Sierra” this typically requires installation of a third-party OpenSSH from Homebrew or the like and using that ssh-agent.
+
To confirm that the ssh-agent correctly finds that key and getting the public key in correct format:

   $ ssh-add -L