[DOCS] Remove text fields from classification dependent variables (#54849)

[role="xpack"]
[testenv="platinum"]
[[put-dfanalytics]]
=== Create {dfanalytics-jobs} API
[subs="attributes"]
++++
<titleabbrev>Create {dfanalytics-jobs}</titleabbrev>
++++

Instantiates a {dfanalytics-job}.

experimental[]

[[ml-put-dfanalytics-request]]
==== {api-request-title}

`PUT _ml/data_frame/analytics/<data_frame_analytics_id>`

[[ml-put-dfanalytics-prereq]]
==== {api-prereq-title}

If the {es} {security-features} are enabled, you must have the following
built-in roles and privileges:

* `machine_learning_admin`
* `kibana_admin` (UI only)

* source index: `read`, `view_index_metadata`
* destination index: `read`, `create_index`, `manage` and `index`
* cluster: `monitor` (UI only)

For more information, see <<security-privileges>> and <<built-in-roles>>.

+
--
NOTE: It is possible that secondary authorization headers are supplied in the
request. If this is the case, the secondary authorization headers are used
instead of the primary headers.
--

[[ml-put-dfanalytics-desc]]
==== {api-description-title}

This API creates a {dfanalytics-job} that performs an analysis on the source
index and stores the outcome in a destination index.

The destination index will be automatically created if it does not exist. The
`index.number_of_shards` and `index.number_of_replicas` settings of the source
index will be copied over the destination index. When the source index matches
multiple indices, these settings will be set to the maximum values found in the
source indices.

The mappings of the source indices are also attempted to be copied over
to the destination index, however, if the mappings of any of the fields don't
match among the source indices, the attempt will fail with an error message.

If the destination index already exists, then it will be use as is. This makes
it possible to set up the destination index in advance with custom settings
and mappings.

[discrete]
[[ml-hyperparam-optimization]]
===== Hyperparameter optimization

If you don't supply {regression} or {classification} parameters, _hyperparameter
optimization_ occurs, which sets a value for the undefined parameters. The
starting point is calculated for data dependent parameters by examining the loss
on the training data. Subject to the size constraint, this operation provides an
upper bound on the improvement in validation loss.

A fixed number of rounds is used for optimization which depends on the number of
parameters being optimized. The optimization starts with random search, then
Bayesian optimization is performed that is targeting maximum expected
improvement. If you override any parameters by explicitely setting it, the
optimization calculates the value of the remaining parameters accordingly and
uses the value you provided for the overridden parameter. The number of rounds
are reduced respectively. The validation error is estimated in each round by
using 4-fold cross validation.

[[ml-put-dfanalytics-path-params]]
==== {api-path-parms-title}

`<data_frame_analytics_id>`::
(Required, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-data-frame-analytics-define]

[role="child_attributes"]
[[ml-put-dfanalytics-request-body]]
==== {api-request-body-title}

`allow_lazy_start`::
(Optional, boolean)
include::{docdir}/ml/ml-shared.asciidoc[tag=allow-lazy-start]

//Begin analysis
`analysis`::
(Required, object)
The analysis configuration, which contains the information necessary to perform
one of the following types of analysis: {classification}, {oldetection}, or
{regression}.
+
.Properties of `analysis`
[%collapsible%open]
====
//Begin classification
`classification`:::
(Required^*^, object)
The configuration information necessary to perform
{ml-docs}/dfa-classification.html[{classification}].
+
TIP: Advanced parameters are for fine-tuning {classanalysis}. They are set
automatically by <<ml-hyperparam-optimization,hyperparameter optimization>>
to give minimum validation error. It is highly recommended to use the default
values unless you fully understand the function of these parameters.
+
.Properties of `classification`
[%collapsible%open]
=====
`class_assignment_objective`::::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=class-assignment-objective]

`dependent_variable`::::
(Required, string)
+
include::{docdir}/ml/ml-shared.asciidoc[tag=dependent-variable]
+
The data type of the field must be numeric (`integer`, `short`, `long`, `byte`),
categorical (`ip` or `keyword`), or boolean.

`eta`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=eta]

`feature_bag_fraction`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=feature-bag-fraction]

`gamma`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=gamma]

`lambda`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=lambda]

`max_trees`::::
(Optional, integer)
include::{docdir}/ml/ml-shared.asciidoc[tag=max-trees]

`num_top_classes`::::
(Optional, integer)
include::{docdir}/ml/ml-shared.asciidoc[tag=num-top-classes]

`num_top_feature_importance_values`::::
(Optional, integer)
Advanced configuration option. Specifies the maximum number of
{ml-docs}/dfa-classification.html#dfa-classification-feature-importance[feature
importance] values per document to return. By default, it is zero and no feature importance
calculation occurs.

`prediction_field_name`::::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=prediction-field-name]

`randomize_seed`::::
(Optional, long)
include::{docdir}/ml/ml-shared.asciidoc[tag=randomize-seed]

`training_percent`::::
(Optional, integer)
include::{docdir}/ml/ml-shared.asciidoc[tag=training-percent]
//End classification
=====
//Begin outlier_detection
`outlier_detection`:::
(Required^*^, object)
The configuration information necessary to perform
{ml-docs}/dfa-outlier-detection.html[{oldetection}]:
+
.Properties of `outlier_detection`
[%collapsible%open]
=====
`compute_feature_influence`::::
(Optional, boolean)
include::{docdir}/ml/ml-shared.asciidoc[tag=compute-feature-influence]

`feature_influence_threshold`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=feature-influence-threshold]

`method`::::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=method]

`n_neighbors`::::
(Optional, integer)
include::{docdir}/ml/ml-shared.asciidoc[tag=n-neighbors]

`outlier_fraction`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=outlier-fraction]

`standardization_enabled`::::
(Optional, boolean)
include::{docdir}/ml/ml-shared.asciidoc[tag=standardization-enabled]
//End outlier_detection
=====
//Begin regression
`regression`:::
(Required^*^, object)
The configuration information necessary to perform
{ml-docs}/dfa-regression.html[{regression}].
+
TIP: Advanced parameters are for fine-tuning {reganalysis}. They are set
automatically by <<ml-hyperparam-optimization,hyperparameter optimization>>
to give minimum validation error. It is highly recommended to use the default
values unless you fully understand the function of these parameters.
+
.Properties of `regression`
[%collapsible%open]
=====
`dependent_variable`::::
(Required, string)
+
include::{docdir}/ml/ml-shared.asciidoc[tag=dependent-variable]
+
The data type of the field must be numeric.

`eta`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=eta]

`feature_bag_fraction`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=feature-bag-fraction]

`gamma`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=gamma]

`lambda`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=lambda]

`max_trees`::::
(Optional, integer)
include::{docdir}/ml/ml-shared.asciidoc[tag=max-trees]

`num_top_feature_importance_values`::::
(Optional, integer)
Advanced configuration option. Specifies the maximum number of
{ml-docs}/dfa-regression.html#dfa-regression-feature-importance[feature importance]
values per document to return. By default, it is zero and no feature importance
calculation occurs.

`prediction_field_name`::::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=prediction-field-name]

`randomize_seed`::::
(Optional, long)
include::{docdir}/ml/ml-shared.asciidoc[tag=randomize-seed]

`training_percent`::::
(Optional, integer)
include::{docdir}/ml/ml-shared.asciidoc[tag=training-percent]
=====
//End regression
====
//End analysis

//Begin analyzed_fields
`analyzed_fields`::
(Optional, object)
include::{docdir}/ml/ml-shared.asciidoc[tag=analyzed-fields]
+
.Properties of `analyzed_fields`
[%collapsible%open]
====
`excludes`:::
(Optional, array)
include::{docdir}/ml/ml-shared.asciidoc[tag=analyzed-fields-excludes]

`includes`:::
(Optional, array)
include::{docdir}/ml/ml-shared.asciidoc[tag=analyzed-fields-includes]
//End analyzed_fields
====

`description`::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=description-dfa]

`dest`::
(Required, object)
include::{docdir}/ml/ml-shared.asciidoc[tag=dest]

`model_memory_limit`::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=model-memory-limit-dfa]

`source`::
(object)
include::{docdir}/ml/ml-shared.asciidoc[tag=source-put-dfa]

[[ml-put-dfanalytics-example]]
==== {api-examples-title}

[[ml-put-dfanalytics-example-preprocess]]
===== Preprocessing actions example

The following example shows how to limit the scope of the analysis to certain
fields, specify excluded fields in the destination index, and use a query to
filter your data before analysis.

[source,console]
--------------------------------------------------
PUT _ml/data_frame/analytics/model-flight-delays-pre
{
  "source": {
    "index": [
      "kibana_sample_data_flights" <1>
    ],
    "query": { <2>
      "range": {
        "DistanceKilometers": {
          "gt": 0
        }
      }
    },
    "_source": { <3>
      "includes": [],
      "excludes": [
        "FlightDelay",
        "FlightDelayType"
      ]
    }
  },
  "dest": { <4>
    "index": "df-flight-delays",
    "results_field": "ml-results"
  },
  "analysis": {
  "regression": {
    "dependent_variable": "FlightDelayMin",
    "training_percent": 90
    }
  },
  "analyzed_fields": { <5>
    "includes": [],
    "excludes": [
      "FlightNum"
    ]
  },
  "model_memory_limit": "100mb"
}
--------------------------------------------------
// TEST[skip:setup kibana sample data]

<1> The source index to analyze.
<2> This query filters out entire documents that will not be present in the
destination index.
<3> The `_source` object defines fields in the dataset that will be included or
excluded in the destination index. In this case, `includes` does not specify any
fields, so the default behavior takes place: all the fields of the source index
will included except the ones that are explicitly specified in `excludes`.
<4> Defines the destination index that contains the results of the analysis and
the fields of the source index specified in the `_source` object. Also defines
the name of the `results_field`.
<5> Specifies fields to be included in or excluded from the analysis. This does
not affect whether the fields will be present in the destination index, only
affects whether they are used in the analysis.

In this example, we can see that all the fields of the source index are included
in the destination index except `FlightDelay` and `FlightDelayType` because
these are defined as excluded fields by the `excludes` parameter of the
`_source` object. The `FlightNum` field is included in the destination index,
however it is not included in the analysis because it is explicitly specified as
excluded field by the `excludes` parameter of the `analyzed_fields` object.

[[ml-put-dfanalytics-example-od]]
===== {oldetection-cap} example

The following example creates the `loganalytics` {dfanalytics-job}, the analysis
type is `outlier_detection`:

[source,console]
--------------------------------------------------
PUT _ml/data_frame/analytics/loganalytics
{
  "description": "Outlier detection on log data",
  "source": {
    "index": "logdata"
  },
  "dest": {
    "index": "logdata_out"
  },
  "analysis": {
    "outlier_detection": {
      "compute_feature_influence": true,
      "outlier_fraction": 0.05,
      "standardization_enabled": true
    }
  }
}
--------------------------------------------------
// TEST[setup:setup_logdata]

The API returns the following result:

[source,console-result]
----
{
    "id": "loganalytics",
    "description": "Outlier detection on log data",
    "source": {
        "index": ["logdata"],
        "query": {
            "match_all": {}
        }
    },
    "dest": {
        "index": "logdata_out",
        "results_field": "ml"
    },
    "analysis": {
        "outlier_detection": {
            "compute_feature_influence": true,
            "outlier_fraction": 0.05,
            "standardization_enabled": true
        }
    },
    "model_memory_limit": "1gb",
    "create_time" : 1562265491319,
    "version" : "8.0.0",
    "allow_lazy_start" : false
}
----
// TESTRESPONSE[s/1562265491319/$body.$_path/]
// TESTRESPONSE[s/"version" : "8.0.0"/"version" : $body.version/]

[[ml-put-dfanalytics-example-r]]
===== {regression-cap} examples

The following example creates the `house_price_regression_analysis`
{dfanalytics-job}, the analysis type is `regression`:

[source,console]
--------------------------------------------------
PUT _ml/data_frame/analytics/house_price_regression_analysis
{
  "source": {
    "index": "houses_sold_last_10_yrs"
  },
  "dest": {
    "index": "house_price_predictions"
  },
  "analysis":
    {
      "regression": {
        "dependent_variable": "price"
      }
    }
}
--------------------------------------------------
// TEST[skip:TBD]

The API returns the following result:

[source,console-result]
----
{
  "id" : "house_price_regression_analysis",
  "source" : {
    "index" : [
      "houses_sold_last_10_yrs"
    ],
    "query" : {
      "match_all" : { }
    }
  },
  "dest" : {
    "index" : "house_price_predictions",
    "results_field" : "ml"
  },
  "analysis" : {
    "regression" : {
      "dependent_variable" : "price",
      "training_percent" : 100
    }
  },
  "model_memory_limit" : "1gb",
  "create_time" : 1567168659127,
  "version" : "8.0.0",
  "allow_lazy_start" : false
}
----
// TESTRESPONSE[s/1567168659127/$body.$_path/]
// TESTRESPONSE[s/"version": "8.0.0"/"version": $body.version/]

The following example creates a job and specifies a training percent:

[source,console]
--------------------------------------------------
PUT _ml/data_frame/analytics/student_performance_mathematics_0.3
{
 "source": {
   "index": "student_performance_mathematics"
 },
 "dest": {
   "index":"student_performance_mathematics_reg"
 },
 "analysis":
   {
     "regression": {
       "dependent_variable": "G3",
       "training_percent": 70,  <1>
       "randomize_seed": 19673948271  <2>
     }
   }
}
--------------------------------------------------
// TEST[skip:TBD]

<1> The `training_percent` defines the percentage of the data set that will be
used for training the model.
<2> The `randomize_seed` is the seed used to randomly pick which data is used
for training.

[[ml-put-dfanalytics-example-c]]
===== {classification-cap} example

The following example creates the `loan_classification` {dfanalytics-job}, the
analysis type is `classification`:

[source,console]
--------------------------------------------------
PUT _ml/data_frame/analytics/loan_classification
{
  "source" : {
    "index": "loan-applicants"
  },
  "dest" : {
    "index": "loan-applicants-classified"
  },
  "analysis" : {
    "classification": {
      "dependent_variable": "label",
      "training_percent": 75,
      "num_top_classes": 2
    }
  }
}
--------------------------------------------------
// TEST[skip:TBD]

[role="xpack"]
[testenv="platinum"]
[[put-dfanalytics]]
=== Create {dfanalytics-jobs} API
[subs="attributes"]
++++
<titleabbrev>Create {dfanalytics-jobs}</titleabbrev>
++++

Instantiates a {dfanalytics-job}.

experimental[]

[[ml-put-dfanalytics-request]]
==== {api-request-title}

`PUT _ml/data_frame/analytics/<data_frame_analytics_id>`

[[ml-put-dfanalytics-prereq]]
==== {api-prereq-title}

If the {es} {security-features} are enabled, you must have the following
built-in roles and privileges:

* `machine_learning_admin`
* `kibana_admin` (UI only)

* source index: `read`, `view_index_metadata`
* destination index: `read`, `create_index`, `manage` and `index`
* cluster: `monitor` (UI only)

For more information, see <<security-privileges>> and <<built-in-roles>>.

+
--
NOTE: It is possible that secondary authorization headers are supplied in the
request. If this is the case, the secondary authorization headers are used
instead of the primary headers.
--

[[ml-put-dfanalytics-desc]]
==== {api-description-title}

This API creates a {dfanalytics-job} that performs an analysis on the source
index and stores the outcome in a destination index.

The destination index will be automatically created if it does not exist. The
`index.number_of_shards` and `index.number_of_replicas` settings of the source
index will be copied over the destination index. When the source index matches
multiple indices, these settings will be set to the maximum values found in the
source indices.

The mappings of the source indices are also attempted to be copied over
to the destination index, however, if the mappings of any of the fields don't
match among the source indices, the attempt will fail with an error message.

If the destination index already exists, then it will be use as is. This makes
it possible to set up the destination index in advance with custom settings
and mappings.

[discrete]
[[ml-hyperparam-optimization]]
===== Hyperparameter optimization

If you don't supply {regression} or {classification} parameters, _hyperparameter
optimization_ occurs, which sets a value for the undefined parameters. The
starting point is calculated for data dependent parameters by examining the loss
on the training data. Subject to the size constraint, this operation provides an
upper bound on the improvement in validation loss.

A fixed number of rounds is used for optimization which depends on the number of
parameters being optimized. The optimization starts with random search, then
Bayesian optimization is performed that is targeting maximum expected
improvement. If you override any parameters by explicitely setting it, the
optimization calculates the value of the remaining parameters accordingly and
uses the value you provided for the overridden parameter. The number of rounds
are reduced respectively. The validation error is estimated in each round by
using 4-fold cross validation.

[[ml-put-dfanalytics-path-params]]
==== {api-path-parms-title}

`<data_frame_analytics_id>`::
(Required, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=job-id-data-frame-analytics-define]

[role="child_attributes"]
[[ml-put-dfanalytics-request-body]]
==== {api-request-body-title}

`allow_lazy_start`::
(Optional, boolean)
include::{docdir}/ml/ml-shared.asciidoc[tag=allow-lazy-start]

//Begin analysis
`analysis`::
(Required, object)
The analysis configuration, which contains the information necessary to perform
one of the following types of analysis: {classification}, {oldetection}, or
{regression}.
+
.Properties of `analysis`
[%collapsible%open]
====
//Begin classification
`classification`:::
(Required^*^, object)
The configuration information necessary to perform
{ml-docs}/dfa-classification.html[{classification}].
+
TIP: Advanced parameters are for fine-tuning {classanalysis}. They are set
automatically by <<ml-hyperparam-optimization,hyperparameter optimization>>
to give minimum validation error. It is highly recommended to use the default
values unless you fully understand the function of these parameters.
+
.Properties of `classification`
[%collapsible%open]
=====
`class_assignment_objective`::::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=class-assignment-objective]

`dependent_variable`::::
(Required, string)
+
include::{docdir}/ml/ml-shared.asciidoc[tag=dependent-variable]
+
The data type of the field must be numeric (`integer`, `short`, `long`, `byte`),
categorical (`ip`, `keyword`, `text`), or boolean.

`eta`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=eta]

`feature_bag_fraction`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=feature-bag-fraction]

`gamma`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=gamma]

`lambda`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=lambda]

`max_trees`::::
(Optional, integer)
include::{docdir}/ml/ml-shared.asciidoc[tag=max-trees]

`num_top_classes`::::
(Optional, integer)
include::{docdir}/ml/ml-shared.asciidoc[tag=num-top-classes]

`num_top_feature_importance_values`::::
(Optional, integer)
Advanced configuration option. Specifies the maximum number of
{ml-docs}/dfa-classification.html#dfa-classification-feature-importance[feature
importance] values per document to return. By default, it is zero and no feature importance
calculation occurs.

`prediction_field_name`::::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=prediction-field-name]

`randomize_seed`::::
(Optional, long)
include::{docdir}/ml/ml-shared.asciidoc[tag=randomize-seed]

`training_percent`::::
(Optional, integer)
include::{docdir}/ml/ml-shared.asciidoc[tag=training-percent]
//End classification
=====
//Begin outlier_detection
`outlier_detection`:::
(Required^*^, object)
The configuration information necessary to perform
{ml-docs}/dfa-outlier-detection.html[{oldetection}]:
+
.Properties of `outlier_detection`
[%collapsible%open]
=====
`compute_feature_influence`::::
(Optional, boolean)
include::{docdir}/ml/ml-shared.asciidoc[tag=compute-feature-influence]

`feature_influence_threshold`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=feature-influence-threshold]

`method`::::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=method]

`n_neighbors`::::
(Optional, integer)
include::{docdir}/ml/ml-shared.asciidoc[tag=n-neighbors]

`outlier_fraction`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=outlier-fraction]

`standardization_enabled`::::
(Optional, boolean)
include::{docdir}/ml/ml-shared.asciidoc[tag=standardization-enabled]
//End outlier_detection
=====
//Begin regression
`regression`:::
(Required^*^, object)
The configuration information necessary to perform
{ml-docs}/dfa-regression.html[{regression}].
+
TIP: Advanced parameters are for fine-tuning {reganalysis}. They are set
automatically by <<ml-hyperparam-optimization,hyperparameter optimization>>
to give minimum validation error. It is highly recommended to use the default
values unless you fully understand the function of these parameters.
+
.Properties of `regression`
[%collapsible%open]
=====
`dependent_variable`::::
(Required, string)
+
include::{docdir}/ml/ml-shared.asciidoc[tag=dependent-variable]
+
The data type of the field must be numeric.

`eta`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=eta]

`feature_bag_fraction`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=feature-bag-fraction]

`gamma`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=gamma]

`lambda`::::
(Optional, double)
include::{docdir}/ml/ml-shared.asciidoc[tag=lambda]

`max_trees`::::
(Optional, integer)
include::{docdir}/ml/ml-shared.asciidoc[tag=max-trees]

`num_top_feature_importance_values`::::
(Optional, integer)
Advanced configuration option. Specifies the maximum number of
{ml-docs}/dfa-regression.html#dfa-regression-feature-importance[feature importance]
values per document to return. By default, it is zero and no feature importance
calculation occurs.

`prediction_field_name`::::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=prediction-field-name]

`randomize_seed`::::
(Optional, long)
include::{docdir}/ml/ml-shared.asciidoc[tag=randomize-seed]

`training_percent`::::
(Optional, integer)
include::{docdir}/ml/ml-shared.asciidoc[tag=training-percent]
=====
//End regression
====
//End analysis

//Begin analyzed_fields
`analyzed_fields`::
(Optional, object)
include::{docdir}/ml/ml-shared.asciidoc[tag=analyzed-fields]
+
.Properties of `analyzed_fields`
[%collapsible%open]
====
`excludes`:::
(Optional, array)
include::{docdir}/ml/ml-shared.asciidoc[tag=analyzed-fields-excludes]

`includes`:::
(Optional, array)
include::{docdir}/ml/ml-shared.asciidoc[tag=analyzed-fields-includes]
//End analyzed_fields
====

`description`::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=description-dfa]

`dest`::
(Required, object)
include::{docdir}/ml/ml-shared.asciidoc[tag=dest]

`model_memory_limit`::
(Optional, string)
include::{docdir}/ml/ml-shared.asciidoc[tag=model-memory-limit-dfa]

`source`::
(object)
include::{docdir}/ml/ml-shared.asciidoc[tag=source-put-dfa]

[[ml-put-dfanalytics-example]]
==== {api-examples-title}

[[ml-put-dfanalytics-example-preprocess]]
===== Preprocessing actions example

The following example shows how to limit the scope of the analysis to certain
fields, specify excluded fields in the destination index, and use a query to
filter your data before analysis.

[source,console]
--------------------------------------------------
PUT _ml/data_frame/analytics/model-flight-delays-pre
{
  "source": {
    "index": [
      "kibana_sample_data_flights" <1>
    ],
    "query": { <2>
      "range": {
        "DistanceKilometers": {
          "gt": 0
        }
      }
    },
    "_source": { <3>
      "includes": [],
      "excludes": [
        "FlightDelay",
        "FlightDelayType"
      ]
    }
  },
  "dest": { <4>
    "index": "df-flight-delays",
    "results_field": "ml-results"
  },
  "analysis": {
  "regression": {
    "dependent_variable": "FlightDelayMin",
    "training_percent": 90
    }
  },
  "analyzed_fields": { <5>
    "includes": [],
    "excludes": [
      "FlightNum"
    ]
  },
  "model_memory_limit": "100mb"
}
--------------------------------------------------
// TEST[skip:setup kibana sample data]

<1> The source index to analyze.
<2> This query filters out entire documents that will not be present in the
destination index.
<3> The `_source` object defines fields in the dataset that will be included or
excluded in the destination index. In this case, `includes` does not specify any
fields, so the default behavior takes place: all the fields of the source index
will included except the ones that are explicitly specified in `excludes`.
<4> Defines the destination index that contains the results of the analysis and
the fields of the source index specified in the `_source` object. Also defines
the name of the `results_field`.
<5> Specifies fields to be included in or excluded from the analysis. This does
not affect whether the fields will be present in the destination index, only
affects whether they are used in the analysis.

In this example, we can see that all the fields of the source index are included
in the destination index except `FlightDelay` and `FlightDelayType` because
these are defined as excluded fields by the `excludes` parameter of the
`_source` object. The `FlightNum` field is included in the destination index,
however it is not included in the analysis because it is explicitly specified as
excluded field by the `excludes` parameter of the `analyzed_fields` object.

[[ml-put-dfanalytics-example-od]]
===== {oldetection-cap} example

The following example creates the `loganalytics` {dfanalytics-job}, the analysis
type is `outlier_detection`:

[source,console]
--------------------------------------------------
PUT _ml/data_frame/analytics/loganalytics
{
  "description": "Outlier detection on log data",
  "source": {
    "index": "logdata"
  },
  "dest": {
    "index": "logdata_out"
  },
  "analysis": {
    "outlier_detection": {
      "compute_feature_influence": true,
      "outlier_fraction": 0.05,
      "standardization_enabled": true
    }
  }
}
--------------------------------------------------
// TEST[setup:setup_logdata]

The API returns the following result:

[source,console-result]
----
{
    "id": "loganalytics",
    "description": "Outlier detection on log data",
    "source": {
        "index": ["logdata"],
        "query": {
            "match_all": {}
        }
    },
    "dest": {
        "index": "logdata_out",
        "results_field": "ml"
    },
    "analysis": {
        "outlier_detection": {
            "compute_feature_influence": true,
            "outlier_fraction": 0.05,
            "standardization_enabled": true
        }
    },
    "model_memory_limit": "1gb",
    "create_time" : 1562265491319,
    "version" : "8.0.0",
    "allow_lazy_start" : false
}
----
// TESTRESPONSE[s/1562265491319/$body.$_path/]
// TESTRESPONSE[s/"version" : "8.0.0"/"version" : $body.version/]

[[ml-put-dfanalytics-example-r]]
===== {regression-cap} examples

The following example creates the `house_price_regression_analysis`
{dfanalytics-job}, the analysis type is `regression`:

[source,console]
--------------------------------------------------
PUT _ml/data_frame/analytics/house_price_regression_analysis
{
  "source": {
    "index": "houses_sold_last_10_yrs"
  },
  "dest": {
    "index": "house_price_predictions"
  },
  "analysis":
    {
      "regression": {
        "dependent_variable": "price"
      }
    }
}
--------------------------------------------------
// TEST[skip:TBD]

The API returns the following result:

[source,console-result]
----
{
  "id" : "house_price_regression_analysis",
  "source" : {
    "index" : [
      "houses_sold_last_10_yrs"
    ],
    "query" : {
      "match_all" : { }
    }
  },
  "dest" : {
    "index" : "house_price_predictions",
    "results_field" : "ml"
  },
  "analysis" : {
    "regression" : {
      "dependent_variable" : "price",
      "training_percent" : 100
    }
  },
  "model_memory_limit" : "1gb",
  "create_time" : 1567168659127,
  "version" : "8.0.0",
  "allow_lazy_start" : false
}
----
// TESTRESPONSE[s/1567168659127/$body.$_path/]
// TESTRESPONSE[s/"version": "8.0.0"/"version": $body.version/]

The following example creates a job and specifies a training percent:

[source,console]
--------------------------------------------------
PUT _ml/data_frame/analytics/student_performance_mathematics_0.3
{
 "source": {
   "index": "student_performance_mathematics"
 },
 "dest": {
   "index":"student_performance_mathematics_reg"
 },
 "analysis":
   {
     "regression": {
       "dependent_variable": "G3",
       "training_percent": 70,  <1>
       "randomize_seed": 19673948271  <2>
     }
   }
}
--------------------------------------------------
// TEST[skip:TBD]

<1> The `training_percent` defines the percentage of the data set that will be
used for training the model.
<2> The `randomize_seed` is the seed used to randomly pick which data is used
for training.

[[ml-put-dfanalytics-example-c]]
===== {classification-cap} example

The following example creates the `loan_classification` {dfanalytics-job}, the
analysis type is `classification`:

[source,console]
--------------------------------------------------
PUT _ml/data_frame/analytics/loan_classification
{
  "source" : {
    "index": "loan-applicants"
  },
  "dest" : {
    "index": "loan-applicants-classified"
  },
  "analysis" : {
    "classification": {
      "dependent_variable": "label",
      "training_percent": 75,
      "num_top_classes": 2
    }
  }
}
--------------------------------------------------
// TEST[skip:TBD]


fix config link

fix config link

* link:../[docs home]

There are many types of configuration objects in Rewrite. The most basic `True` and `False` are the root of all other objects. Blah blah more stuff here.

* True
* False
* And
* Or
* Not
* Condition
* Operation

* link:../docs[docs home]

There are many types of configuration objects in Rewrite. The most basic `True` and `False` are the root of all other objects. Blah blah more stuff here.

* True
* False
* And
* Or
* Not
* Condition
* Operation


Update 2016-10-23-Google-maps-zoom-theory-and-example-with-Cqt-QWT.adoc

= Google maps zoom theory and example with {C++/qt ; QWT}

===== Theory

Recently I had to set up a zoom on qwt plot, the goal was to get the same zoom as google maps when zooming on some coordinates.

qwt plot example :

image:qwt_plot.png[qwt plot, 500, 500, align="center"]

Open maps and test, put your cursor on a city and zoom, all the map will be zoomed but the city will always be under your cursor.

The solution is easy the main idea is to reset your scales with a factor (ex : 0.9 or 1.1) and to apply a part of the factor by using the position of the cursor in order to get the zoom or unzoom.

We will assume that the x axis is the horizontal and y the vertical.
So if we want to zoom on the point which is at (1, 2.5)  [(x, y)] we will

- get the position of our cursor on the widget (x_cursor, y_cursor)
- define the percentage (x_max, y_max) which are the percentage of the x_cursor position and the x_cursor_max position (size max of the widget) and the same for y_percent with y_cursor and y_cursor_max.
- we will compute the new x scale with x_percent of the factor on a side and 100 - x_percent of the factor for the other side and do the same for the y scale with y_percent / the factor.

After processing all of this we will have the x axis scale [0.2, 4] and the y axis scale [0.4, 8.5].

==== Code with c++/qt and qwt library :

This code is based on the *PlotMagnifier::rescale(double factor);* function.

[source,cpp]
----
void My_PlotMagnifier::rescale_on_cursor(double factor,      // the rescale factor (0.9 or 1.1)
                                         int x_cursor,       // the x, y position of the cursor
                                         int y_cursor,
                                         QSize parent_size)  // the max size of the widget
{
    QwtPlot* plt = plot();

    // max size of the widget
    float x_cursor_max = parent_size.width();
    float y_cursor_max = parent_size.height();

    // percentage position of the cursor in the widget
    float x_percent = (x_cursor * 100) / x_cursor_max;
    float y_percent = (y_cursor * 100) / y_cursor_max;

    // will be the new lowerBound and upperBound of the scales
    float delta_1 = 0;
    float delta_2 = 0;

    _plotScaleEngine->zoomed(true);
    bool doReplot = false;
    const bool autoReplot = plt->autoReplot();
    plt->setAutoReplot( false );

    for ( int axisId = 0; axisId < QwtPlot::axisCnt; axisId++ )
    {
        const QwtScaleDiv &scaleDiv = plt->axisScaleDiv( axisId );
        if ( isAxisEnabled( axisId ) )
        {
            double center = scaleDiv.lowerBound() + scaleDiv.range() / 2;	// Here we set the center of the scale
            const double width_2 = scaleDiv.range() / 2 * factor;			// the width wanted between lowerBound (or upper) and the center
            float interval = (scaleDiv.range() / 2) - width_2;				// the interval between the old width and the wanted width

            if (axisId == 0) 		// y - left
            {
                delta_1 = (center - width_2) + (((100 - y_percent) * interval) / 100) - interval;
                delta_2 = (center + width_2) - ((y_percent * interval) / 100) + interval;
            }
            else if (axisId == 2) 	// x - bottom
            {
                delta_1 = (center - width_2) + (((x_percent * interval) / 100)) - interval;
                delta_2 = (center + width_2) - (((100 - x_percent) * interval) / 100) + interval;
            }
            else
            {
                delta_1 = 0;
                delta_2 = 0;
            }
            plt->setAxisScale(axisId, delta_1, delta_2);
            doReplot = true;
        }
    }
    plt->setAutoReplot( autoReplot );
    if ( doReplot )
        plt->replot();
}

void Gr_PlotMagnifier::widgetWheelEvent(QWheelEvent *WheelEvent)
{
    _plotScaleEngine->getPlot()->select();
    QSize parent_size = this->parentWidget()->size();

    if (WheelEvent->delta() > 0) 	// unzoom
        rescale_on_cursor(0.9, WheelEvent->x(), WheelEvent->y(), parent_size);
    else 							// zoom
        rescale_on_cursor(1.1, WheelEvent->x(), WheelEvent->y(), parent_size);
}

----


Add link to API docs

Add link to API docs

// Module included in the following assemblies:
//
// * monitoring/monitoring-your-own-services.adoc

[id="setting-up-metrics-collection_{context}"]
= Setting up metrics collection

To use the metrics exposed by your service, you need to configure OpenShift Monitoring to scrape metrics from the `/metrics` endpoint. You can do this using a ServiceMonitor, a custom resource definition (CRD) that specifies how a service should be monitored, or a PodMonitor, a CRD that specifies how a pod should be monitored. The former requires a Service object, while the latter does not, allowing Prometheus to directly scrape metrics from the metrics endpoint exposed by a Pod.

This procedure shows how to create a ServiceMonitor for the service.

.Prerequisites

* Log in as a cluster administrator or a user with the `monitor-crd-edit` role.

.Procedure

. Create a YAML file for the ServiceMonitor configuration. In this example, the file is called `example-app-service-monitor.yaml`.

. Fill the file with the configuration for creating the ServiceMonitor:
+
[source,yaml]
----
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    k8s-app: prometheus-example-monitor
  name: prometheus-example-monitor
  namespace: ns1
spec:
  endpoints:
  - interval: 30s
    port: web
    scheme: http
  selector:
    matchLabels:
      app: prometheus-example-app
----
+
This configuration makes OpenShift Monitoring scrape the metrics exposed by the sample service deployed in "Deploying a sample service", which includes the single `version` metric.

. Apply the configuration file to the cluster:
+
----
$ oc apply -f example-app-service-monitor.yaml
----
+
It will take some time to deploy the ServiceMonitor.

. You can check that the ServiceMonitor is running:
+
----
$ oc -n ns1 get servicemonitor
NAME                         AGE
prometheus-example-monitor   81m
----

.Additional resources

See the link:https://github.com/openshift/prometheus-operator/blob/release-4.3/Documentation/api.md[Prometheus Operator API documentation] for more information on ServiceMonitors and PodMonitors.


Fix an OL formatting error in contributor guide

[id="contributing-to-docs-tools-and-setup"]
= Install and set up the tools and software
:icons:
:toc: macro
:toc-title:
:toclevels: 1
:linkattrs:
:description: How to set up and install the tools to contribute

toc::[]

== Create a GitHub account
Before you can contribute to OpenShift documentation, you must
https://www.github.com/join[sign up for a GitHub account].

== Set up authentication
When you have your account set up, follow the instructions to
https://help.github.com/articles/generating-ssh-keys/[generate and set up SSH
keys on GitHub] for proper authentication between your workstation and GitHub.

Confirm authentication is working correctly with the following command:

----
$ ssh -T git@github.com
----

== Fork and clone the OpenShift documentation repository
You must fork and set up the OpenShift documentation repository on your
workstation so that you can create PRs and contribute. These steps must only
be performed during initial setup.

. Fork the https://github.com/openshift/openshift-docs repository into your
GitHub account from the GitHub UI. You can do this by clicking on *Fork* in the
upper right-hand corner.

. In the terminal on your workstation, change into the directory where you want
to clone the forked repository.

.  Clone the forked repository onto your workstation with the following
command, replacing _<user_name>_ with your actual GitHub username.
+
----
$ git clone git@github.com:<user_name>/openshift-docs.git
----

. Change into the directory for the local repository you just cloned.
+
----
$ cd openshift-docs
----

. Add an upstream pointer back to the OpenShift's remote repository, in this
case _openshift-docs_.
+
----
$ git remote add upstream git@github.com:openshift/openshift-docs.git
----

This ensures that you are tracking the remote repository to keep your local
repository in sync with it.

== Install AsciiBinder and dependencies
When you have the documentation repository cloned and set up, you are ready to
install the software and tools you will use to create the content. All OpenShift
documentation is created in AsciiDoc, and is processed with https://github.com/redhataccess/ascii_binder[AsciiBinder],
which is an http://asciidoctor.org/[AsciiDoctor]-based docs management system.

=== What you require
The following are minimum requirements:

* A bash shell environment (Linux and OS X include a bash shell environment out
of the box, but if you are on Windows you can use http://cygwin.com/[Cygwin])
* https://www.ruby-lang.org/en/[Ruby]
* http://www.git-scm.com/[Git]
* A web browser (Firefox, Chrome, or Safari)
* An editor that can strip trailing whitespace, such as
link:https://atom.io/[Atom].

=== Install the required software dependencies on a Linux system
The following instructions describe how to install all the required tools to do
live content editing on a Fedora Linux system.

1. Install the _RubyGems_ package with `yum install rubygems`
+
[NOTE]
====
On certain systems, `yum` installs an older version of RubyGems that can cause issues. As an alternative, you can install RubyGems by using RVM. The following example is referenced from the link:https://rvm.io/rvm/install[RVM site]:

[source,terminal]
----
$ curl -sSL https://get.rvm.io | bash -s stable --ruby
----
====

2. Install _Ruby_ development packages with `yum install ruby-devel`
3. Install _gcc_ with `yum install gcc-c++`
4. Install _redhat-rpm-config_ with `yum install redhat-rpm-config`
5. Install _make_ with `yum install make`
6. Install _asciidoctor-diagram_ with `gem install asciidoctor-diagram`
7. Install the _ascii_binder_ gem with `gem install ascii_binder`

NOTE: If you already have AsciiBinder installed, you might be due for an update.
These directions assume that you are using AsciiBinder 0.2.0 or newer. To check
and update if necessary, simply run `gem update ascii_binder`. Note that you might require root permissions.

=== Building the collection
With the initial setup complete, you are ready to build the collection.

1. From the `openshift-docs` directory, run an initial build:
+
----
$ cd openshift-docs
$ asciibinder build
----
2. Open the generated HTML file in your web browser. This will be located in the
`openshift-docs/_preview/<distro>/<branch>` directory, with the same path and
filename as the original `.adoc` file you edited, only it will be with the
`.html` extension.

== Clean up
The `.gitignore` file is set up to prevent anything under the `_preview` and
`_package` directories from being committed. However, you can reset the
environment manually by running:

----
$ asciibinder clean
----

== Next steps
With the repository and tools set up on your workstation, you can now either
edit existing content or create assemblies and modules.

* link:doc_guidelines.adoc[Review the documentation guidelines] to understand
some basic guidelines to keep things consistent across our content.
* link:create_or_edit_content.adoc[Create a local working branch] on your
workstation to edit existing content or create content.

=== How to deploy to your own OpenShift cluster for testing

You can deploy to your own OpenShift cluster for development. This process will use your github repo to launch the website,
and therefore your github repo must have all of the upstream branches. `main` is used for site changes,
so assuming all your work is in `main`, you can remove all remote branches and then push the upstream branches.

Removing remote branches and updating with upstream branches (this assumes remote repos called `origin` and `upstream`)
[WARNING]
====
This is a destructive process, make sure that this is purely a development repo, as all local and remote branches will be deleted
by performing the below commands.
====
----
$ git fetch --all
$ for branch in $(git branch -r | grep -v "main" | grep "^  origin"); do git push origin --delete $(echo $branch | cut -d '/' -f 2); done
$ git branch -D $(git branch | grep -v 'main' | xargs)
$ for branch in $(git branch -r | grep -v "main" | grep "^  upstream"); do git branch --track $(echo $branch | cut -d '/' -f 2) $(echo $branch | tr -d '[:space:]'); done
$ for branch in $(git branch | grep -v "main"); do git push origin $(echo $branch | tr -d '[:space:]'); done
----

Deploying the docs site to an OpenShift cluster
----
$ oc process -f asciibinder-template.yml -p NAME=community-docs \
    -p SOURCE_REPOSITORY_URL=$(git remote get-url origin) \
    -p SOURCE_REPOSITORY_REF=$(git rev-parse --abbrev-ref HEAD) \
    -p DOC_TYPE=community \
    | oc create -f -
$ oc process -f asciibinder-template.yml -p NAME=commercial-docs \
    -p SOURCE_REPOSITORY_URL=$(git remote get-url origin) \
    -p SOURCE_REPOSITORY_REF=$(git rev-parse --abbrev-ref HEAD) \
    -p DOC_TYPE=commercial \
    | oc create -f -
----

[NOTE]
====
If the build fails with "Fetch source failed" status, you can
delete all the created objects and re-run above with an HTTP uri
as the `SOURCE_REPOSITORY_URL`, or you can
link:https://docs.okd.io/latest/dev_guide/builds/build_inputs.html#source-secrets-combinations[create a source secret]
and add it to the stg1 build, `oc set build-secret --source bc/stg1-docs <secret name>`.
====

You can delete all created objects by running

----
$ oc delete all -l app=community-docs
$ oc delete all -l app=commercial-docs
----

[id="contributing-to-docs-tools-and-setup"]
= Install and set up the tools and software
:icons:
:toc: macro
:toc-title:
:toclevels: 1
:linkattrs:
:description: How to set up and install the tools to contribute

toc::[]

== Create a GitHub account
Before you can contribute to OpenShift documentation, you must
https://www.github.com/join[sign up for a GitHub account].

== Set up authentication
When you have your account set up, follow the instructions to
https://help.github.com/articles/generating-ssh-keys/[generate and set up SSH
keys on GitHub] for proper authentication between your workstation and GitHub.

Confirm authentication is working correctly with the following command:

----
$ ssh -T git@github.com
----

== Fork and clone the OpenShift documentation repository
You must fork and set up the OpenShift documentation repository on your
workstation so that you can create PRs and contribute. These steps must only
be performed during initial setup.

. Fork the https://github.com/openshift/openshift-docs repository into your
GitHub account from the GitHub UI. You can do this by clicking on *Fork* in the
upper right-hand corner.

. In the terminal on your workstation, change into the directory where you want
to clone the forked repository.

.  Clone the forked repository onto your workstation with the following
command, replacing _<user_name>_ with your actual GitHub username.
+
----
$ git clone git@github.com:<user_name>/openshift-docs.git
----

. Change into the directory for the local repository you just cloned.
+
----
$ cd openshift-docs
----

. Add an upstream pointer back to the OpenShift's remote repository, in this
case _openshift-docs_.
+
----
$ git remote add upstream git@github.com:openshift/openshift-docs.git
----

This ensures that you are tracking the remote repository to keep your local
repository in sync with it.

== Install AsciiBinder and dependencies
When you have the documentation repository cloned and set up, you are ready to
install the software and tools you will use to create the content. All OpenShift
documentation is created in AsciiDoc, and is processed with https://github.com/redhataccess/ascii_binder[AsciiBinder],
which is an http://asciidoctor.org/[AsciiDoctor]-based docs management system.

=== What you require
The following are minimum requirements:

* A bash shell environment (Linux and OS X include a bash shell environment out
of the box, but if you are on Windows you can use http://cygwin.com/[Cygwin])
* https://www.ruby-lang.org/en/[Ruby]
* http://www.git-scm.com/[Git]
* A web browser (Firefox, Chrome, or Safari)
* An editor that can strip trailing whitespace, such as
link:https://atom.io/[Atom].

=== Install the required software dependencies on a Linux system
The following instructions describe how to install all the required tools to do
live content editing on a Fedora Linux system.

1. Install the _RubyGems_ package with `yum install rubygems`

[NOTE]
====
On certain systems, `yum` installs an older version of RubyGems that can cause issues. As an alternative, you can install RubyGems by using RVM. The following example is referenced from the link:https://rvm.io/rvm/install[RVM site]:

[source,terminal]
----
$ curl -sSL https://get.rvm.io | bash -s stable --ruby
----
====

2. Install _Ruby_ development packages with `yum install ruby-devel`
3. Install _gcc_ with `yum install gcc-c++`
4. Install _redhat-rpm-config_ with `yum install redhat-rpm-config`
5. Install _make_ with `yum install make`
6. Install _asciidoctor-diagram_ with `gem install asciidoctor-diagram`
7. Install the _ascii_binder_ gem with `gem install ascii_binder`

NOTE: If you already have AsciiBinder installed, you might be due for an update.
These directions assume that you are using AsciiBinder 0.2.0 or newer. To check
and update if necessary, simply run `gem update ascii_binder`. Note that you might require root permissions.

=== Building the collection
With the initial setup complete, you are ready to build the collection.

1. From the `openshift-docs` directory, run an initial build:
+
----
$ cd openshift-docs
$ asciibinder build
----
2. Open the generated HTML file in your web browser. This will be located in the
`openshift-docs/_preview/<distro>/<branch>` directory, with the same path and
filename as the original `.adoc` file you edited, only it will be with the
`.html` extension.

== Clean up
The `.gitignore` file is set up to prevent anything under the `_preview` and
`_package` directories from being committed. However, you can reset the
environment manually by running:

----
$ asciibinder clean
----

== Next steps
With the repository and tools set up on your workstation, you can now either
edit existing content or create assemblies and modules.

* link:doc_guidelines.adoc[Review the documentation guidelines] to understand
some basic guidelines to keep things consistent across our content.
* link:create_or_edit_content.adoc[Create a local working branch] on your
workstation to edit existing content or create content.

=== How to deploy to your own OpenShift cluster for testing

You can deploy to your own OpenShift cluster for development. This process will use your github repo to launch the website,
and therefore your github repo must have all of the upstream branches. `main` is used for site changes,
so assuming all your work is in `main`, you can remove all remote branches and then push the upstream branches.

Removing remote branches and updating with upstream branches (this assumes remote repos called `origin` and `upstream`)
[WARNING]
====
This is a destructive process, make sure that this is purely a development repo, as all local and remote branches will be deleted
by performing the below commands.
====
----
$ git fetch --all
$ for branch in $(git branch -r | grep -v "main" | grep "^  origin"); do git push origin --delete $(echo $branch | cut -d '/' -f 2); done
$ git branch -D $(git branch | grep -v 'main' | xargs)
$ for branch in $(git branch -r | grep -v "main" | grep "^  upstream"); do git branch --track $(echo $branch | cut -d '/' -f 2) $(echo $branch | tr -d '[:space:]'); done
$ for branch in $(git branch | grep -v "main"); do git push origin $(echo $branch | tr -d '[:space:]'); done
----

Deploying the docs site to an OpenShift cluster
----
$ oc process -f asciibinder-template.yml -p NAME=community-docs \
    -p SOURCE_REPOSITORY_URL=$(git remote get-url origin) \
    -p SOURCE_REPOSITORY_REF=$(git rev-parse --abbrev-ref HEAD) \
    -p DOC_TYPE=community \
    | oc create -f -
$ oc process -f asciibinder-template.yml -p NAME=commercial-docs \
    -p SOURCE_REPOSITORY_URL=$(git remote get-url origin) \
    -p SOURCE_REPOSITORY_REF=$(git rev-parse --abbrev-ref HEAD) \
    -p DOC_TYPE=commercial \
    | oc create -f -
----

[NOTE]
====
If the build fails with "Fetch source failed" status, you can
delete all the created objects and re-run above with an HTTP uri
as the `SOURCE_REPOSITORY_URL`, or you can
link:https://docs.okd.io/latest/dev_guide/builds/build_inputs.html#source-secrets-combinations[create a source secret]
and add it to the stg1 build, `oc set build-secret --source bc/stg1-docs <secret name>`.
====

You can delete all created objects by running

----
$ oc delete all -l app=community-docs
$ oc delete all -l app=commercial-docs
----


[SYNCOPE-152] Little doc addition

[SYNCOPE-152] Little doc addition

//
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements.  See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership.  The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License.  You may obtain a copy of the License at
//
//   http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied.  See the License for the
// specific language governing permissions and limitations
// under the License.
//
=== Extensions

The _vanilla_ Apache Syncope deployment can be optional enriched with useful features via an Extension, instead of bloating
every single deployment with unneeded libraries and configurations.

With reference to <<architecture,architecture>>, an extension might add a <<rest>> endpoint, manage the
<<persistence,persistence>> of additional entities, extend the <<security,security>> mechanisms, tweak the
<<provisioning-layer,provisioning layer>>, add features to the <<admin-console-component>> or
the <<enduser-component>>, or even bring all such things together.

Extensions are available from different sources:

. as Maven artifacts published from the Apache Syncope codebase, part of the official releases - this is the case of the
ones detailed below;
. as Maven artifacts published by third parties;
. as part of a given deployment source code, as explained <<customization-extensions, in the following>>.

==== Apache Camel Provisioning Manager

This extension delegates the <<provisioning,provisioning>> process execution to a set of
http://camel.apache.org[Apache Camel^] routes.

The pre-loaded routes can be dynamically changed at runtime via REST or admin console, and modifications are immediately
made available for processing.

For example, on creating a new user, you may wish to send an email to an administrator; or if a user is
reactivated, you may wish to reactivate the user's home page on a web server. +
All these things and more are possible using the myriad of
http://camel.apache.org/components.html[components^] that are available to be used in Apache Camel routes.

[NOTE]
.Extension Sources
====
The source code of this extension is available from the Apache Syncope
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/tree/syncope-{docVersion}/ext/camel[source tree^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/tree/master/ext/camel[source tree^]
endif::[]
.
====

==== Swagger

This extension enables http://swagger.io/swagger-ui/[Swagger UI^] as web interface for dealing with Apache Syncope
<<rest,RESTful>> services.

Once installed, Swagger UI is available at

....
protocol://host:port/syncope/swagger/
....

where `protocol`, `host` and `port` reflect your Java EE container installation.

[NOTE]
.Extension Sources
====
The source code of this extension is available from the Apache Syncope
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/tree/syncope-{docVersion}/ext/swagger-ui[source tree^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/tree/master/ext/swagger-ui[source tree^]
endif::[]
.
====

==== SAML 2.0 Service Provider

This extension can be leveraged to provide
https://en.wikipedia.org/wiki/Security_Assertion_Markup_Language[SAML 2.0^]-based
https://en.wikipedia.org/wiki/Single_sign-on[Single Sign-On^] access to the <<admin-console-component>>,
the <<enduser-component>> or any other Java EE application dealing with the <<core>>.

Once installed, one or more https://en.wikipedia.org/wiki/Identity_provider[Identity Providers^] can be imported from
their https://en.wikipedia.org/wiki/SAML_2.0#SAML_2.0_Metadata[metadata^].
For each Identity Provider, it is to configure which one of the attributes - returned as part of the assertion
containing the attribute statements - is going to be used by Syncope to match the internal users.

[NOTE]
.Extension Sources
====
The source code of this extension is available from the Apache Syncope
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/tree/syncope-{docVersion}/ext/saml2sp[source tree^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/tree/master/ext/saml2sp[source tree^]
endif::[]
.
====

[TIP]
====
This extension adds features to all components and layers that are available, and can be taken as reference when creating
<<customization-extensions,new extensions>>.
====

==== Elasticsearch

This extension provides an alternate internal search engine for <<users-groups-and-any-objects>>, requiring an external
https://www.elastic.co/[Elasticsearch^] cluster.

[TIP]
As search operations are central for different aspects of the <<provisioning,provisioning process>>, the global
performances are expected to improve when using this extension.

[NOTE]
.Extension Sources
====
The source code of this extension is available from the Apache Syncope
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/tree/syncope-{docVersion}/ext/elasticsearch[source tree^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/tree/master/ext/elasticsearch[source tree^]
endif::[]
.
====

==== SCIM

http://www.simplecloud.info/[SCIM^] (System for Cross-domain Identity Management) 2.0 is the open API for managing
identities, published under the IETF:

. https://tools.ietf.org/html/rfc7642[Definitions, Overview, Concepts, and Requirements^]
. https://tools.ietf.org/html/rfc7643[Core Schema^]
. https://tools.ietf.org/html/rfc7644[Protocol^]

This extension enables an additional `/scim` REST endpoint, implementing the communication according to the SCIM 2.0
standard, in order to provision User, Enterprise User and Group SCIM entities to Apache Syncope.

[NOTE]
.Extension Sources
====
The source code of this extension is available from the Apache Syncope
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/tree/syncope-{docVersion}/ext/scimv2[source tree^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/tree/master/ext/scimv2[source tree^]
endif::[]
.
====

//
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements.  See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership.  The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License.  You may obtain a copy of the License at
//
//   http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied.  See the License for the
// specific language governing permissions and limitations
// under the License.
//
=== Extensions

The _vanilla_ Apache Syncope deployment can be optional enriched with useful features via an Extension, instead of bloating
every single deployment with unneeded libraries and configurations.

With reference to <<architecture,architecture>>, an extension might add a <<rest>> endpoint, manage the
<<persistence,persistence>> of additional entities, extend the <<security,security>> mechanisms, tweak the
<<provisioning-layer,provisioning layer>>, add features to the <<admin-console-component>> or
the <<enduser-component>>, or even bring all such things together.

Extensions are available from different sources:

. as Maven artifacts published from the Apache Syncope codebase, part of the official releases - this is the case of the
ones detailed below;
. as Maven artifacts published by third parties;
. as part of a given deployment source code, as explained <<customization-extensions, in the following>>.

==== Apache Camel Provisioning Manager

This extension delegates the <<provisioning,provisioning>> process execution to a set of
http://camel.apache.org[Apache Camel^] routes.

The pre-loaded routes can be dynamically changed at runtime via REST or admin console, and modifications are immediately
made available for processing.

For example, on creating a new user, you may wish to send an email to an administrator; or if a user is
reactivated, you may wish to reactivate the user's home page on a web server. +
All these things and more are possible using the myriad of
http://camel.apache.org/components.html[components^] that are available to be used in Apache Camel routes.

[NOTE]
.Extension Sources
====
The source code of this extension is available from the Apache Syncope
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/tree/syncope-{docVersion}/ext/camel[source tree^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/tree/master/ext/camel[source tree^]
endif::[]
.
====

==== Swagger

This extension enables http://swagger.io/swagger-ui/[Swagger UI^] as web interface for dealing with Apache Syncope
<<rest,RESTful>> services.

Once installed, Swagger UI is available at

....
protocol://host:port/syncope/swagger/
....

where `protocol`, `host` and `port` reflect your Java EE container installation.

[NOTE]
.Extension Sources
====
The source code of this extension is available from the Apache Syncope
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/tree/syncope-{docVersion}/ext/swagger-ui[source tree^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/tree/master/ext/swagger-ui[source tree^]
endif::[]
.
====

==== SAML 2.0 Service Provider

This extension can be leveraged to provide
https://en.wikipedia.org/wiki/Security_Assertion_Markup_Language[SAML 2.0^]-based
https://en.wikipedia.org/wiki/Single_sign-on[Single Sign-On^] access to the <<admin-console-component>>,
the <<enduser-component>> or any other Java EE application dealing with the <<core>>.

Once installed, one or more https://en.wikipedia.org/wiki/Identity_provider[Identity Providers^] can be imported from
their https://en.wikipedia.org/wiki/SAML_2.0#SAML_2.0_Metadata[metadata^].
For each Identity Provider, it is to configure which one of the attributes - returned as part of the assertion
containing the attribute statements - is going to be used by Syncope to match the internal users.

[NOTE]
.Extension Sources
====
The source code of this extension is available from the Apache Syncope
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/tree/syncope-{docVersion}/ext/saml2sp[source tree^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/tree/master/ext/saml2sp[source tree^]
endif::[]
.
====

[TIP]
====
This extension adds features to all components and layers that are available, and can be taken as reference when creating
<<customization-extensions,new extensions>>.
====

==== Elasticsearch

This extension provides an alternate internal search engine for <<users-groups-and-any-objects>>, requiring an external
https://www.elastic.co/[Elasticsearch^] cluster.

[TIP]
As search operations are central for different aspects of the <<provisioning,provisioning process>>, the global
performances are expected to improve when using this extension.

[NOTE]
.Extension Sources
====
The source code of this extension is available from the Apache Syncope
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/tree/syncope-{docVersion}/ext/elasticsearch[source tree^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/tree/master/ext/elasticsearch[source tree^]
endif::[]
.
====

==== SCIM

http://www.simplecloud.info/[SCIM^] (System for Cross-domain Identity Management) 2.0 is the open API for managing
identities, published under the IETF:

. https://tools.ietf.org/html/rfc7642[Definitions, Overview, Concepts, and Requirements^]
. https://tools.ietf.org/html/rfc7643[Core Schema^]
. https://tools.ietf.org/html/rfc7644[Protocol^]

This extension enables an additional `/scim` REST endpoint, implementing the communication according to the SCIM 2.0
standard.

[NOTE]
.Extension Sources
====
The source code of this extension is available from the Apache Syncope
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/tree/syncope-{docVersion}/ext/scimv2[source tree^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/tree/master/ext/scimv2[source tree^]
endif::[]
.
====


feat: allow access to SSL attributes in EL TemplateEngine

gravitee-io/#issues#5322

= Expression Language
:page-sidebar: apim_3_x_sidebar
:page-permalink: apim/3.x/apim_publisherguide_expression_language.html
:page-folder: apim/user-guide/publisher
:page-layout: apim3x

== Overview

The APIM Expression Language (EL for short) is one of the key features
that can be used by API publishers to configure various aspects and services of an API.

The EL is a powerful language used for querying and
manipulating an object graph. It is based on the http://docs.spring.io/spring/docs/current/spring-framework-reference/html/expressions.html[SpEL^] (Spring Expression Language).
This means that you can do everything described in the link.

In addition, APIM extends the standard SpEL capabilities by providing extra objects and properties
inside the expression language context.

== Usage
The basic expression language syntax is as follows:

`{#request.id}`

See the sections below for example expression notations.

== API
=== Properties

As an API publisher, you can define properties for your API. These properties are
automatically _injected_ into the expression language context to be used later.

==== Example

* Get the value of the property `my-property` defined in API properties:
`{#properties['my-property']}`

=== Dictionaries

Dictionaries work in a similar way to properties, but you need to specify the dictionary name as well as the property name.

==== Example

* Get the value of the property `my-property` defined in dictionary `my-dictionary`:
`{#dictionaries['my-dictionary']['my-property']}`

=== Endpoints
When you define endpoints for your API, you need to give them a _name_ which
must be a unique identifier across all endpoints of the API. This identifier can be used to get an endpoint reference (i.e. uri).

For example: when you create an API, a _default_ endpoint is created,
corresponding to the value you set for the backend property. This endpoint can
be retrieved with EL by using the following syntax:

`{#endpoints['default']}`

== Request

The properties you can access for API requests are listed below.

|===
.^|Property      |Description            ^.^|Type            |Example

.^|id            |Identifier             ^.^|string          |12345678-90ab-cdef-1234-567890ab
.^|transactionId |Transaction identifier ^.^|string          |cd123456-7890-abcd-ef12-34567890
.^|uri           |URI                    ^.^|string          |/v2/store/MyStore?order=100
.^|path          |Path                   ^.^|string          |/v2/store/MyStore
.^|paths         |Path parts             ^.^|array of string |[,v2,store,MyStore]
.^|pathInfo      |Path info              ^.^|string          |/store/MyStore
.^|pathInfos     |Path info parts        ^.^|array of string |[,store,MyStore]
.^|contextPath   |Context path           ^.^|string          |/v2/
.^|params        |Query parameters       ^.^|key / value     |order -> 100
.^|pathParams    |Path parameters        ^.^|key / value     |storeId -> MyStore (__see Warning for details__)
.^|headers       |Headers                ^.^|key / value     |X-Custom -> myvalue
.^|method        |HTTP method            ^.^|string          |GET
.^|scheme        |HTTP scheme            ^.^|string          |http
.^|version       |HTTP version           ^.^|string          |HTTP_1_1
.^|timestamp     |Timestamp              ^.^|long            |1602781000267
.^|remoteAddress |Remote address         ^.^|string          |0:0:0:0:0:0:0:1
.^|localAddress  |Local address          ^.^|string          |0:0:0:0:0:0:0:1
.^|content       |Body content           ^.^|string          |-
.^|ssl           |SSLSession information ^.^|<<SSL Object>>  |-
|===

WARNING: `{#request.content}` is only available for policies bound to an `on-request-content` phase.

=== SSL Object
The properties you can access in SSL Session object are listed below.

|===
.^|Property      |Description               ^.^|Type                    |Example

.^|clientHost    |Host name of the client   ^.^|string                  |client.domain.com
.^|clientPort    |Port number of the client ^.^|long                    |443
.^|client        |Client information        ^.^|<<Principal Object>>    |-
.^|server        |Server information        ^.^|<<Principal Object>>    |-
|===

=== Principal Object
The properties you can access in Principal object are listed below.

|===
.^|Property                 |Description                            ^.^|Type    |Example

4+|*Common DN attributes*
.^|businessCategory         |Business category                      ^.^|string  |-
.^|c                        |Country code                           ^.^|string  |FR
.^|cn                       |Common name                            ^.^|string  |-
.^|countryOfCitizenship     |RFC 3039 CountryOfCitizenship          ^.^|string  |-
.^|countryOfResidence       |RFC 3039 CountryOfResidence            ^.^|string  |-
.^|dateOfBirth              |RFC 3039 RFC 3039 DateOfBirth          ^.^|string  |19830719000000Z
.^|dc                       |Domain component                       ^.^|string  |-
.^|description              |Description                            ^.^|string  |-
.^|dmdName                  |RFC 2256 directory management domain   ^.^|string  |-
.^|dnQualifier              |Domain name qualifier                  ^.^|string  |-
.^|e                        |Email address in Verisign certificates ^.^|string  |-
.^|emailAddress             |Email address (RSA PKCS#9 extension)   ^.^|string  |-
.^|gender                   |RFC 3039 Gender                        ^.^|string  |"M", "F", "m" or "f"
.^|generation               |Naming attributes of type X520name     ^.^|string  |-
.^|givenname                |Naming attributes of type X520name     ^.^|string  |-
.^|initials                 |Naming attributes of type X520name     ^.^|string  |-
.^|l                        |Locality name                          ^.^|string  |-
.^|name                     |Name                                   ^.^|string  |-
.^|nameAtBirth              |ISIS-MTT NameAtBirth                   ^.^|string  |-
.^|o                        |Organization                           ^.^|string  |-
.^|organizationIdentifier   |Organization identifier                ^.^|string  |-
.^|ou                       |Organization unit name                 ^.^|string  |-
.^|placeOfBirth             |RFC 3039 PlaceOfBirth                  ^.^|string  |-
.^|postalAddress            |RFC 3039 PostalAddress                 ^.^|string  |-
.^|postalCode               |Postal code                            ^.^|string  |-
.^|pseudonym                |RFC 3039 Pseudonym                     ^.^|string  |-
.^|role                     |Role                                   ^.^|string  |-
.^|serialnumber             |Device serial number name              ^.^|string  |-
.^|st                       |State or province name                 ^.^|string  |-
.^|street                   |Street                                 ^.^|string  |-
.^|surname                  |Naming attributes of type X520name     ^.^|string  |-
.^|t                        |Title                                  ^.^|string  |-
.^|telephoneNumber          |Telephone number                       ^.^|string  |-
.^|uid                      |LDAP User id                           ^.^|string  |-
.^|uniqueIdentifier         |Naming attributes of type X520name     ^.^|string  |-
.^|unstructuredAddress      |Unstructured address (from PKCS#9)     ^.^|string  |-
4+|*Other attributes*
.^|attributes               |Retrieves all attribute values         ^.^|key / value  |"ou" -> ["Test team", "Dev team"]
.^|defined                  |Returns true if the principal object is defined and contains values. False otherwise.         ^.^|boolean  |-
.^|dn                       |Full domain name                       ^.^|string  |-

|===

Even if some of these attributes can be arrays, EL will return the first item in the array.
If you want to retrieve all values of an attribute, you can use the `attributes` field

If the principal is not defined, all values are empty.

=== Examples

* Get the value of the `Content-Type` header for an incoming HTTP request:
`{#request.headers['content-type']}`

* Get the second part of the request path:
`{#request.paths[1]}`

* Get the client HOST from the SSL session:
`{#request.ssl.clientHost}`

* Get the client DN from the SSL session:
`{#request.ssl.client.dn}`

* Get the server organization from the SSL session:
`{#request.ssl.server.o}`

* Get all the organization units of the server from the SSL session:
- `{#request.ssl.server.attributes['ou'][0]}`
- `{#request.ssl.server.attributes['OU'][1]}`
- `{#request.ssl.server.attributes['Ou'][2]}`

* Get a custom attribute of the client from the SSL session:
`{#request.ssl.client.attributes['1.2.3.4'][0]}`

* Determine if the SSL attributes of the client are set:
`{#request.ssl.client.defined}`

== Request context

=== Properties

|===
|Property |Description |Type |Always present

.^|attributes
|Request context attributes
^.^|key-value
^.^|X
|===

=== Attributes
When APIM Gateway handles an incoming HTTP request, some attributes are
automatically created. These attributes are:

|===
|Property |Description |Type |Nullable

.^|context-path
|Context-path
^.^|string
^.^|-

.^|resolved-path
|Resolved-path is the path defined in policies
^.^|string
^.^|-

.^|application
|The authenticated application doing incoming HTTP request
^.^|string
^.^|X (for keyless plan)

.^|api
|Called API
^.^|string
^.^|-

.^|user-id
|The user identifier of incoming HTTP request:

* The subscription id for api-key based plan

* Remote IP for keyless based plan

^.^|string
^.^|-

.^|plan
|Plan used to manage incoming HTTP request
^.^|string
^.^|-

.^|api-key
|the api-key used (in case of an api-key based plan)
^.^|string
^.^|X (for no api-key plan)
|===

Additionally, some policies (like the link:/apim/3.x/apim_policies_oauth2.html#attributes[OAuth2 policy])
register other attributes in the context. See the documentation for the policies you are using for more information.

=== Example

* Get the value of the `user-id` attribute for an incoming HTTP request:

`{#context.attributes['user-id']}`

* Get the value of the `plan` attribute for an incoming HTTP request:

`{#context.attributes['plan']}`

* Check that the path starts with a given value:

`{#request.path.startsWith('/my/api')}`

== Response

=== Properties
|===
.^|Property      |Description            ^.^|Type            |Example

.^|content       |Body content           ^.^|string          |-
.^|headers       |Headers                ^.^|key / value     |X-Custom -> myvalue
.^|status        |Status of the HTTP response      ^.^|int    |200
|===

=== Example

* Get the status of an HTTP response:
`{#response.status}`

== Policies
You can use the EL to update some aspects of policy configuration.
The policy specifies if it supports EL or not.

== Mixin

In previous examples, we showed various ways to manipulate objects available
in the EL context. You can also mix property
usage to provide an increasingly dynamic configuration.

For example, we can retrieve the value of an HTTP header where the name
is based on an API property named `my-property`:

`{#request.headers[#properties['my-property']]}`

= Expression Language
:page-sidebar: apim_3_x_sidebar
:page-permalink: apim/3.x/apim_publisherguide_expression_language.html
:page-folder: apim/user-guide/publisher
:page-layout: apim3x

== Overview

The APIM Expression Language (EL for short) is one of the key features
that can be used by API publishers to configure various aspects and services of an API.

The EL is a powerful language used for querying and
manipulating an object graph. It is based on the http://docs.spring.io/spring/docs/current/spring-framework-reference/html/expressions.html[SpEL^] (Spring Expression Language).
This means that you can do everything described in the link.

In addition, APIM extends the standard SpEL capabilities by providing extra objects and properties
inside the expression language context.

== Usage
The basic expression language syntax is as follows:

`{#request.id}`

See the sections below for example expression notations.

== API
=== Properties

As an API publisher, you can define properties for your API. These properties are
automatically _injected_ into the expression language context to be used later.

==== Example

* Get the value of the property `my-property` defined in API properties:
`{#properties['my-property']}`

=== Dictionaries

Dictionaries work in a similar way to properties, but you need to specify the dictionary name as well as the property name.

==== Example

* Get the value of the property `my-property` defined in dictionary `my-dictionary`:
`{#dictionaries['my-dictionary']['my-property']}`

=== Endpoints
When you define endpoints for your API, you need to give them a _name_ which
must be a unique identifier across all endpoints of the API. This identifier can be used to get an endpoint reference (i.e. uri).

For example: when you create an API, a _default_ endpoint is created,
corresponding to the value you set for the backend property. This endpoint can
be retrieved with EL by using the following syntax:

`{#endpoints['default']}`

== Request

The properties you can access for API requests are listed below.

|===
.^|Property      |Description            ^.^|Type            |Example

.^|id            |Identifier             ^.^|string          |12345678-90ab-cdef-1234-567890ab
.^|transactionId |Transaction identifier ^.^|string          |cd123456-7890-abcd-ef12-34567890
.^|uri           |URI                    ^.^|string          |/v2/store/MyStore?order=100
.^|path          |Path                   ^.^|string          |/v2/store/MyStore
.^|paths         |Path parts             ^.^|array of string |[,v2,store,MyStore]
.^|pathInfo      |Path info              ^.^|string          |/store/MyStore
.^|pathInfos     |Path info parts        ^.^|array of string |[,store,MyStore]
.^|contextPath   |Context path           ^.^|string          |/v2/
.^|params        |Query parameters       ^.^|key / value     |order -> 100
.^|pathParams    |Path parameters        ^.^|key / value     |storeId -> MyStore (__see Warning for details__)
.^|headers       |Headers                ^.^|key / value     |X-Custom -> myvalue
.^|method        |HTTP method            ^.^|string          |GET
.^|scheme        |HTTP scheme            ^.^|string          |http
.^|version       |HTTP version           ^.^|string          |HTTP_1_1
.^|timestamp     |Timestamp              ^.^|long            |1602781000267
.^|remoteAddress |Remote address         ^.^|string          |0:0:0:0:0:0:0:1
.^|localAddress  |Local address          ^.^|string          |0:0:0:0:0:0:0:1
.^|content       |Body content           ^.^|string          |-
|===

WARNING: `{#request.content}` is only available for policies bound to an `on-request-content` phase.

=== Example

* Get the value of the `Content-Type` header for an incoming HTTP request:
`{#request.headers['content-type']}`

* Get the second part of the request path:
`{#request.paths[1]}`

== Request context

=== Properties

|===
|Property |Description |Type |Always present

.^|attributes
|Request context attributes
^.^|key-value
^.^|X
|===

=== Attributes
When APIM Gateway handles an incoming HTTP request, some attributes are
automatically created. These attributes are:

|===
|Property |Description |Type |Nullable

.^|context-path
|Context-path
^.^|string
^.^|-

.^|resolved-path
|Resolved-path is the path defined in policies
^.^|string
^.^|-

.^|application
|The authenticated application doing incoming HTTP request
^.^|string
^.^|X (for keyless plan)

.^|api
|Called API
^.^|string
^.^|-

.^|user-id
|The user identifier of incoming HTTP request:

* The subscription id for api-key based plan

* Remote IP for keyless based plan

^.^|string
^.^|-

.^|plan
|Plan used to manage incoming HTTP request
^.^|string
^.^|-

.^|api-key
|the api-key used (in case of an api-key based plan)
^.^|string
^.^|X (for no api-key plan)
|===

Additionally, some policies (like the link:/apim/3.x/apim_policies_oauth2.html#attributes[OAuth2 policy])
register other attributes in the context. See the documentation for the policies you are using for more information.

=== Example

* Get the value of the `user-id` attribute for an incoming HTTP request:

`{#context.attributes['user-id']}`

* Get the value of the `plan` attribute for an incoming HTTP request:

`{#context.attributes['plan']}`

* Check that the path starts with a given value:

`{#request.path.startsWith('/my/api')}`

== Response

=== Properties
|===
.^|Property      |Description            ^.^|Type            |Example

.^|content       |Body content           ^.^|string          |-
.^|headers       |Headers                ^.^|key / value     |X-Custom -> myvalue
.^|status        |Status of the HTTP response      ^.^|int    |200
|===

=== Example

* Get the status of an HTTP response:
`{#response.status}`

== Policies
You can use the EL to update some aspects of policy configuration.
The policy specifies if it supports EL or not.

== Mixin

In previous examples, we showed various ways to manipulate objects available
in the EL context. You can also mix property
usage to provide an increasingly dynamic configuration.

For example, we can retrieve the value of an HTTP header where the name
is based on an API property named `my-property`:

`{#request.headers[#properties['my-property']]}`


GitHub-26681: Noted new flag for JDK 11+