Add trick for exporting C build flags to sub-Makefiles

Thanks to dozzie on IRC for the suggestion.

[[ports]]
== NIFs and port drivers

Erlang.mk can not only build Erlang projects, but also the C code
that some projects come with, like NIFs and port drivers.

There are two ways to build the C code: using a custom Makefile,
or making Erlang.mk do it directly. The C code will be built
as needed when you run `make`.

// @todo something for easier bootstrapping

=== C source code location and Erlang environment

The C source code should be located in the '$(C_SRC_DIR)' directory.
It defaults to 'c_src/'. Should you need to modify it, all you
need to do is to set the variable in your Makefile before including
Erlang.mk:

[source,make]
C_SRC_DIR = $(CURDIR)/my_nif_source

When this directory exists, Erlang.mk will automatically create a
file named '$(C_SRC_ENV)'. This file defaults to '$(C_SRC_DIR)/env.mk'.
This can also be changed:

[source,make]
C_SRC_ENV = $(C_SRC_DIR)/erlang_env.mk

It contains a few variable definitions for the environment used for the build:

`ERTS_INCLUDE_DIR`::
	Path to the ERTS include files ('erl_driver.h', 'erl_nif.h' and more).
`ERL_INTERFACE_INCLUDE_DIR`::
	Path to the Erl_Interface include files ('ei.h' and related).
`ERL_INTERFACE_LIB_DIR`::
	Path to the Erl_Interface static libraries.

=== Using a custom Makefile

Erlang.mk will automatically run `make` if it detects a Makefile
in '$(C_SRC_DIR)/Makefile'.

The Makefile should have at least two targets: a default target
(which can be anything, for example `all`) which is invoked when
building the C code, and a `clean` target invoked when cleaning
it.

You can include the 'env.mk' file to benefit from the Erlang
environment detection:

[source,make]
include env.mk

=== Using Erlang.mk directly

You don't need to write a Makefile to build C source code, however.
Erlang.mk comes with rules to build both shared libraries and
executables, using the source files it finds in '$(C_SRC_DIR)'.

By default, Erlang.mk will create a shared library. To change
this and create an executable instead, put this in your Makefile
before including Erlang.mk:

[source,make]
C_SRC_TYPE = executable

The generated file name varies depending on the type of project
you have (shared library or executable) and on the platform you
build the project on.

For shared libraries, the generated file name will be
'$(C_SRC_OUTPUT)$(C_SRC_SHARED_EXTENSION)', with the default
being '$(CURDIR)/priv/$(PROJECT)' followed by the extension:
`.dll` on Windows, `.so` everywhere else.

For executables, the generated file name is
'$(C_SRC_OUTPUT)$(C_SRC_EXECUTABLE_EXTENSION)', with the same
default except for the extension: `.exe` on Windows, and otherwise
nothing.

Erlang.mk sets appropriate compile and linker flags by default.
These flags vary depending on the platform, and can of course
be overriden.

`CC`::
	The compiler to be used.
`CFLAGS`::
	C compiler flags.
`CXXFLAGS`::
	C++ compiler flags.
`LDFLAGS`::
	Linker flags.
`LDLIBS`::
	Libraries to link against.

The source files are automatically gathered from the contents
of '$(C_SRC_DIR)'. Erlang.mk looks for '.c', '.C', '.cc' and '.cpp'
source files. You can define the variable `SOURCES` to manually
list the files to compile.

=== Propagating compile and linker flags to sub-Makefiles

In some cases it might be necessary to propagate the flags
you just defined to the sub-Makefiles of your local project.
You generally can't just export those as this could impact
the building of dependencies.

Makefiles allow you to export variables for specific targets.
When doing this, the variables will be exported only when
this target runs, and not for other targets. It is therefore
possible to export them when building the C code without
impacting other build steps.

By adding this to your Makefile all five variables will be
made available to sub-Makefiles when building C code:

[source,make]
----
app-c_src: export CC +=
app-c_src: export CFLAGS +=
app-c_src: export CPPFLAGS +=
app-c_src: export LDFLAGS +=
app-c_src: export LDLIBS +=
----

Appending an empty string to the existing value is necessary
because Makefiles expect an assignment for target-specific
exports. Alternatively you can set a new value:

[source,make]
----
app-c_src: export CFLAGS = -O3
----


[DOCS] Reformatted security troubleshooting pages (elastic/x-pack-elasticsearch#2799)

Original commit: elastic/x-pack-elasticsearch@b10c822ba25bafa20920aa010191b8bc08ecc14b

[[security-troubleshooting]]
== {security} Troubleshooting

Use the information in this section to troubleshoot common problems and find
answers for frequently asked questions.

* <<security-trb-settings>>
* <<security-trb-roles>>
* <<security-trb-extraargs>>
* <<trouble-shoot-active-directory>>
* <<trb-security-maccurl>>
* <<trb-security-sslhandshake>>
* <<trb-security-ssl>>
* <<trb-security-internalserver>>

To get help, see <<xpack-help>>.

[[security-trb-settings]]
=== Some settings are not returned via the nodes settings API

*Symptoms:*

* When you use the {ref}/cluster-nodes-info.html[nodes info API] to retrieve
settings for a node, some information is missing.

*Resolution:*

This is intentional. Some of the settings are considered to be highly
sensitive: all `ssl` settings, ldap `bind_dn`, and `bind_password`.
For this reason, we filter these settings and do not expose them via
the nodes info API rest endpoint. You can also define additional
sensitive settings that should be hidden using the
`xpack.security.hide_settings` setting. For example, this snippet
hides the `url` settings of the `ldap1` realm and all settings of the
`ad1` realm.

[source, yaml]
------------------------------------------
xpack.security.hide_settings: xpack.security.authc.realms.ldap1.url,
xpack.security.authc.realms.ad1.*
------------------------------------------

[[security-trb-roles]]
=== Authorization exceptions

*Symptoms:*

* I configured the appropriate roles and the users, but I still get an
authorization exception.
* I can authenticate to LDAP, but I still get an authorization exception.

*Resolution:*

. Verify that the role names associated with the users match the roles defined
in the `roles.yml` file. You can use the `users` tool to list all the users. Any
unknown roles are marked with `*`.
+
--
[source, shell]
------------------------------------------
bin/xpack/users list
rdeniro        : admin
alpacino       : power_user
jacknich       : monitoring,unknown_role* <1>
------------------------------------------
<1> `unknown_role` was not found in `roles.yml`

For more information about this command, see
{ref}/users-command.html[Users Command].
--

. If you are authenticating to LDAP, a number of configuration options can cause
this error.
+
--
|======================
|_group identification_ |

Groups are located by either an LDAP search or by the "memberOf" attribute on
the user.  Also, If subtree search is turned off, it will search only one
level deep.  See the <<ldap-settings, LDAP Settings>> for all the options.
There are many options here and sticking to the defaults will not work for all
scenarios.

| _group to role mapping_|

Either the `role_mapping.yml` file or the location for this file could be
misconfigured. See <<security-files, Security Files>> for more.

|_role definition_|

The role definition might be missing or invalid.

|======================

To help track down these possibilities, add the following lines to the end of
the `log4j2.properties` configuration file in the `CONFIG_DIR`:

[source,properties]
----------------
logger.authc.name = org.elasticsearch.xpack.security.authc
logger.authc.level = DEBUG
----------------

A successful authentication should produce debug statements that list groups and
role mappings.
--

[[security-trb-extraargs]]
=== Users command fails due to extra arguments

*Symptoms:*

* The `users` command fails with the following message:
`ERROR: extra arguments [...] were provided`.

*Resolution:*

This error occurs when the `users` tool is parsing the input and finds
unexpected arguments. This can happen when there are special characters used in
some of the arguments. For example, on Windows systems the `,` character is
considered a parameter separator; in other words `-r role1,role2` is translated
to `-r role1 role2` and the `users` tool only recognizes `role1` as an expected
parameter. The solution here is to quote the parameter: `-r "role1,role2"`.

For more information about this command, see
{ref}/users-command.html[Users Command].

[[trouble-shoot-active-directory]]
=== Users are frequently locked out of Active Directory

*Symptoms:*

* Certain users are being frequently locked out of Active Directory.

*Resolution:*

Check your realm configuration; realms are checked serially, one after another.
If your Active Directory realm is being checked before other realms and there
are usernames that appear in both Active Directory and another realm, a valid
login for one realm might be causing failed login attempts in another realm.

For example, if `UserA` exists in both Active Directory and a file realm, and
the Active Directory realm is checked first and file is checked second, an
attempt to authenticate as `UserA` in the file realm would first attempt to
authenticate against Active Directory and fail, before successfully
authenticating against the `file` realm. Because authentication is verified on
each request, the Active Directory realm would be checked - and fail - on each
request for `UserA` in the `file` realm. In this case, while the authentication
request completed successfully, the account on Active Directory would have
received several failed login attempts, and that account might become
temporarily locked out. Plan the order of your realms accordingly.

Also note that it is not typically necessary to define multiple Active Directory
realms to handle domain controller failures. When using Microsoft DNS, the DNS
entry for the domain should always point to an available domain controller.

[[trb-security-maccurl]]
=== Certificate verification fails for curl on Mac

*Symptoms:*

* `curl` on the Mac returns a certificate verification error even when the
`--cacert` option is used.

*Resolution:*

Apple's integration of `curl` with their keychain technology disables the
`--cacert` option.
See http://curl.haxx.se/mail/archive-2013-10/0036.html for more information.

You can use another tool, such as `wget`, to test certificates. Alternately, you
can add the certificate for the signing certificate authority MacOS system
keychain, using a procedure similar to the one detailed at the
http://support.apple.com/kb/PH14003[Apple knowledge base]. Be sure to add the
signing CA's certificate and not the server's certificate.

[[trb-security-sslhandshake]]
=== SSLHandshakeException causes connections to fail

*Symptoms:*

* A `SSLHandshakeException` causes a connection to a node to fail and indicates
that there is a configuration issue. Some of the common exceptions are shown
below with tips on how to resolve these issues.

*Resolution:*

`java.security.cert.CertificateException: No name matching node01.example.com found`::
+
--
Indicates that a client connection was made to `node01.example.com` but the
certificate returned did not contain the name `node01.example.com`. In most
cases, the issue can be resolved by ensuring the name is specified during
certificate creation. For more information, see <<ssl-tls>>. Another scenario is
when the environment does not wish to use DNS names in certificates at all. In
this scenario, all settings in `elasticsearch.yml` should only use IP addresses
including the `network.publish_host` setting.
--

`java.security.cert.CertificateException: No subject alternative names present`::
+
--
Indicates that a client connection was made to an IP address but the returned
certificate did not contain any `SubjectAlternativeName` entries. IP addresses
are only used for hostname verification if they are specified as a
`SubjectAlternativeName` during certificate creation. If the intent was to use
IP addresses for hostname verification, then the certificate will need to be
regenerated with the appropriate IP address. See <<ssl-tls>>.
--

`javax.net.ssl.SSLHandshakeException: null cert chain` and `javax.net.ssl.SSLException: Received fatal alert: bad_certificate`::
+
--
The `SSLHandshakeException` indicates that a self-signed certificate was
returned by the client that is not trusted as it cannot be found in the
`truststore` or `keystore`. This `SSLException` is seen on the client side of
the connection.
--

`sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target` and `javax.net.ssl.SSLException: Received fatal alert: certificate_unknown`::
+
--
This `SunCertPathBuilderException` indicates that a certificate was returned
during the handshake that is not trusted. This message is seen on the client
side of the connection. The `SSLException` is seen on the server side of the
connection. The CA certificate that signed the returned certificate was not
found in the `keystore` or `truststore` and needs to be added to trust this
certificate.
--

[[trb-security-ssl]]
=== Common SSL/TLS exceptions

*Symptoms:*

* You might see some exceptions related to SSL/TLS in your logs. Some of the
common exceptions are shown below with tips on how to resolve these issues. +

*Resolution:*

`WARN: received plaintext http traffic on a https channel, closing connection`::
+
--
Indicates that there was an incoming plaintext http request. This typically
occurs when an external applications attempts to make an unencrypted call to the
REST interface. Please ensure that all applications are using `https` when
calling the REST interface with SSL enabled.
--

`org.elasticsearch.common.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:`::
+
--
Indicates that there was incoming plaintext traffic on an SSL connection. This
typically occurs when a node is not configured to use encrypted communication
and tries to connect to nodes that are using encrypted communication. Please
verify that all nodes are using the same setting for
`xpack.security.transport.ssl.enabled`.

For more information about this setting, see
{ref}/security-settings.html[Security Settings in {es}].
--

`java.io.StreamCorruptedException: invalid internal transport message format, got`::
+
--
Indicates an issue with data received on the transport interface in an unknown
format. This can happen when a node with encrypted communication enabled
connects to a node that has encrypted communication disabled. Please verify that
all nodes are using the same setting for `xpack.security.transport.ssl.enabled`.

For more information about this setting, see
{ref}/security-settings.html[Security Settings in {es}].
--

`java.lang.IllegalArgumentException: empty text`::
+
--
This exception is typically seen when a `https` request is made to a node that
is not using `https`. If `https` is desired, please ensure the following setting
is in `elasticsearch.yml`:

[source,yaml]
----------------
xpack.security.http.ssl.enabled: true
----------------

For more information about this setting, see
{ref}/security-settings.html[Security Settings in {es}].
--

`ERROR: unsupported ciphers [...] were requested but cannot be used in this JVM`::
+
--
This error occurs when a SSL/TLS cipher suite is specified that cannot supported
by the JVM that {es} is running in. Security tries to use the specified cipher
suites that are supported by this JVM. This error can occur when using the
Security defaults as some distributions of OpenJDK do not enable the PKCS11
provider by default. In this case, we recommend consulting your JVM
documentation for details on how to enable the PKCS11 provider.

Another common source of this error is requesting cipher suites that use
encrypting with a key length greater than 128 bits when running on an Oracle JDK.
In this case, you must install the
<<ciphers, JCE Unlimited Strength Jurisdiction Policy Files>>.
--

[[trb-security-internalserver]]
=== Internal Server Error in Kibana

*Symptoms:*

* In 5.1.1, an `UnhandledPromiseRejectionWarning` occurs and {kib} displays an
Internal Server Error.
//TBD: Is the same true for later releases?

*Resolution:*

If the Security plugin is enabled in {es} but disabled in {kib}, you must
still set `elasticsearch.username` and `elasticsearch.password` in `kibana.yml`.
Otherwise, {kib} cannot connect to {es}.

[[security-troubleshooting]]
== {security} Troubleshooting

[float]
=== `settings`

Some settings are not returned via the nodes settings API::
+
--
This is intentional. Some of the settings are considered to be highly
sensitive: all `ssl` settings, ldap `bind_dn`, `bind_password`).
For this reason, we filter these  settings and do not expose them via
the nodes info API rest endpoint. You can also define additional
sensitive settings that should be hidden  using the
`xpack.security.hide_settings` setting. For example, this snippet
hides the `url` settings of the `ldap1` realm and all settings of the
`ad1` realm.

[source, yaml]
------------------------------------------
xpack.security.hide_settings: xpack.security.authc.realms.ldap1.url, xpack.security.authc.realms.ad1.*
------------------------------------------

--

[float]
=== `users`

I configured the appropriate roles and the users, but I still get an authorization exception::
+
--
Verify that the role names associated with the users match the roles defined in the `roles.yml` file. You
can use the `users` tool to list all the users. Any unknown roles are marked with `*`.

[source, shell]
------------------------------------------
bin/xpack/users list
rdeniro        : admin
alpacino       : power_user
jacknich       : monitoring,unknown_role* <1>
------------------------------------------
<1> `unknown_role` was not found in `roles.yml`
--

ERROR: extra arguments [...] were provided::
+
--
This error occurs when the `users` tool is parsing the input and finds unexepected arguments. This can happen when there
are special characters used in some of the arguments. For example, on Windows systems the `,` character is considered
a parameter separator; in other words `-r role1,role2` is translated to `-r role1 role2` and the `users` tool only recognizes
`role1` as an expected parameter. The solution here is to quote the parameter: `-r "role1,role2"`.
--

[[trouble-shoot-active-directory]]
[float]
=== Active Directory

Certain users are being frequently locked out of Active Directory::
+
--
Check your realm configuration; realms are checked serially, one after another. If your Active Directory realm is being checked before other realms and there are usernames
that appear in both Active Directory and another realm, a valid login for one realm may be causing failed login attempts in another realm.

For example, if `UserA` exists in both Active Directory and a file realm, and the Active Directory realm is checked first and
file is checked second, an attempt to authenticate as `UserA` in the file realm would first attempt to authenticate
against Active Directory and fail, before successfully authenticating against the `file` realm. Because authentication is
verified on each request, the Active Directory realm would be checked - and fail - on each request for `UserA` in the `file`
realm. In this case, while the authentication request completed successfully, the account on Active Directory would have received
several failed login attempts, and that account may become temporarily locked out. Plan the order of your realms accordingly.

Also note that it is not typically necessary to define multiple Active Directory realms to handle domain controller failures. When using Microsoft DNS, the DNS entry for the domain should always point to an available domain controller.
--

[float]
=== LDAP

I can authenticate to LDAP, but I still get an authorization exception::
+
--
A number of configuration options can cause this error.

|======================
|_group identification_ |

Groups are located by either an LDAP search or by the "memberOf" attribute on
the user.  Also, If subtree search is turned off, it will search only one
level deep.  See the <<ldap-settings, LDAP Settings>> for all the options.
There are many options here and sticking to the defaults will not work for all
scenarios.

| _group to role mapping_|

Either the `role_mapping.yml` file or the location for this file could be
misconfigured. See <<security-files, Security Files>> for more.

|_role definition_|

The role definition may be missing or invalid.

|======================

To help track down these possibilities, add the following lines to the end of the `log4j2.properties` configuration file in the
`CONFIG_DIR`:

[source,properties]
----------------
logger.authc.name = org.elasticsearch.xpack.security.authc
logger.authc.level = DEBUG
----------------

A successful authentication should produce debug statements that list groups and role mappings.
--

[float]
=== Encryption & Certificates

`curl` on the Mac returns a certificate verification error even when the `--cacert` option is used::
+
--
Apple's integration of `curl` with their keychain technology disables the `--cacert` option.
See http://curl.haxx.se/mail/archive-2013-10/0036.html for more information.

You can use another tool, such as `wget`, to test certificates. Alternately, you can add the certificate for the
signing certificate authority MacOS system keychain, using a procedure similar to the one detailed at the
http://support.apple.com/kb/PH14003[Apple knowledge base]. Be sure to add the signing CA's certificate and not the server's certificate.
--

[float]
==== SSLHandshakeException causing connections to fail

A `SSLHandshakeException` will cause a connection to a node to fail and indicates that there is a configuration issue. Some of the
common exceptions are shown below with tips on how to resolve these issues.

`java.security.cert.CertificateException: No name matching node01.example.com found`::
+
--
Indicates that a client connection was made to `node01.example.com` but the certificate returned did not contain the name `node01.example.com`.
In most cases, the issue can be resolved by ensuring the name is specified during certificate creation. For more information, see <<ssl-tls>>.
Another scenario is when the environment does not wish to use DNS names in certificates at all. In this scenario, all settings
in `elasticsearch.yml` should only use IP addresses including the `network.publish_host` setting.
--

`java.security.cert.CertificateException: No subject alternative names present`::
+
--
Indicates that a client connection was made to an IP address but the returned certificate did not contain any `SubjectAlternativeName` entries.
IP addresses are only used for hostname verification if they are specified as a `SubjectAlternativeName` during certificate creation. If the intent was to use IP addresses for hostname verification, then the certificate
will need to be regenerated with the appropriate IP address. See <<ssl-tls>>.
--

`javax.net.ssl.SSLHandshakeException: null cert chain` and `javax.net.ssl.SSLException: Received fatal alert: bad_certificate`::
+
--
The `SSLHandshakeException` above indicates that a self-signed certificate was returned by the client that is not trusted
as it cannot be found in the `truststore` or `keystore`. The `SSLException` above is seen on the client side of the connection.
--

`sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target` and `javax.net.ssl.SSLException: Received fatal alert: certificate_unknown`::
+
--
The `SunCertPathBuilderException` above indicates that a certificate was returned during the handshake that is not trusted.
This message is seen on the client side of the connection. The `SSLException` above is seen on the server side of the
connection. The CA certificate that signed the returned certificate was not found in the `keystore` or `truststore` and
needs to be added to trust this certificate.
--

[float]
==== Other SSL/TLS related exceptions

The are other exceptions related to SSL that may be seen in the logs. Below you will find some common exceptions and their
meaning.

WARN: received plaintext http traffic on a https channel, closing connection::
+
--
Indicates that there was an incoming plaintext http request. This typically occurs when an external applications attempts
to make an unencrypted call to the REST interface. Please ensure that all applications are using `https` when calling the
REST interface with SSL enabled.
--

`org.elasticsearch.common.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:`::
+
--
Indicates that there was incoming plaintext traffic on an SSL connection. This typically occurs when a node is not
configured to use encrypted communication and tries to connect to nodes that are using encrypted communication. Please
verify that all nodes are using the same setting for `xpack.security.transport.ssl.enabled`.
--

`java.io.StreamCorruptedException: invalid internal transport message format, got`::
+
--
Indicates an issue with data received on the transport interface in an unknown format. This can happen when a node with
encrypted communication enabled connects to a node that has encrypted communication disabled. Please verify that all
nodes are using the same setting for `xpack.security.transport.ssl.enabled`.
--

`java.lang.IllegalArgumentException: empty text`::
+
--
The exception is typically seen when a `https` request is made to a node that is not using `https`. If `https` is desired,
please ensure the following setting is in `elasticsearch.yml`:

[source,yaml]
----------------
xpack.security.http.ssl.enabled: true
----------------
--

ERROR: unsupported ciphers [...] were requested but cannot be used in this JVM::
+
--
This error occurs when a SSL/TLS cipher suite is specified that cannot supported by the JVM that Elasticsearch is running
in. Security will try to use the specified cipher suites that are supported by this JVM. This error can occur when using
the Security defaults as some distributions of OpenJDK do not enable the PKCS11 provider by default. In this case, we
recommend consulting your JVM documentation for details on how to enable the PKCS11 provider.

Another common source of this error is requesting cipher suites that use encrypting with a key length greater than 128 bits
when running on an Oracle JDK. In this case, you will need to install the <<ciphers, JCE Unlimited Strength Jurisdiction Policy Files>>.
--

[float]
==== Internal Server Error in Kibana

If the Security plugin is enabled in Elasticsearch but disabled in Kibana, you must
still set `elasticsearch.username` and `elasticsearch.password` in `kibana.yml`.
Otherwise, Kibana cannot connect to Elasticsearch. In 5.1.1, this results in an
`UnhandledPromiseRejectionWarning` and Kibana displays an Internal Server Error.


Minor tweak to JavaScript API introduction

I found it confusing that this described the "JavaScript API" and then all the examples were in ClojureScript.  I also found referring to "ClojureScript types" a little confusing.  Hopefully this conveys the intent.

= JavaScript API
Mike Fikes
2017-12-06
:type: reference
:toc: macro
:icons: font

ifdef::env-github,env-browser[:outfilesuffix: .adoc]

toc::[]

The implementation of ClojureScript collections define several JavaScript functions
that can be called from ClojureScript using JavaScript interop or directly from
JavaScript.  This page describes those functions which are officially stable and
publicly-consumable.

[[indexOf]]
=== indexOf

==== Syntax

[source,clojure]
----
(.indexOf coll search-value)
(.indexOf coll search-value from-index)
----

==== Parameters

`coll` - a collection satisfying the `sequential?` predicate.

`search-value` - the value to search for.

`from-index` - an optional starting index.

==== Description

Gets the index of a value in a sequential collection, or `-1` if
not found. By default, search proceeds from the beginning, unless an
optional starting index is supplied.

==== Examples

[source,clojure]
----
(.indexOf [1 2 3 5 7] 5) ;; 3
(.indexOf [1 2 3 5 7] 4) ;; -1
(.indexOf [1 2 3 5 2] 2 3) ;; 4
----

[[lastIndexOf]]
=== lastIndexOf

==== Syntax

[source,clojure]
----
(.lastIndexOf coll search-value)
(.lastIndexOf coll search-value from-index)
----

==== Parameters

`coll` - a collection satisfying the `sequential?` predicate.

`search-value` - the value to search for.

`from-index` - an optional starting index.

==== Description

Gets the last index of a value in a sequential collection, or `-1` if
not found. By default, search proceeds from the end, unless an
optional starting index is supplied.

==== Examples

[source,clojure]
----
(.lastIndexOf [1 2 3 5 2 7] 2) ;; 4
(.lastIndexOf [1 2 3 5 2 7] 4) ;; -1
(.lastIndexOf [1 2 3 5 2 7] 2 3) ;; 1
----

= JavaScript API
Mike Fikes
2017-12-06
:type: reference
:toc: macro
:icons: font

ifdef::env-github,env-browser[:outfilesuffix: .adoc]

toc::[]

ClojureScript defines types that have stable, publicly-consumable
JavaScript APIs as specified below.

[[indexOf]]
=== indexOf

==== Syntax

[source,clojure]
----
(.indexOf coll search-value)
(.indexOf coll search-value from-index)
----

==== Parameters

`coll` - a collection satisfying the `sequential?` predicate.

`search-value` - the value to search for.

`from-index` - an optional starting index.

==== Description

Gets the index of a value in a sequential collection, or `-1` if
not found. By default, search proceeds from the beginning, unless an
optional starting index is supplied.

==== Examples

[source,clojure]
----
(.indexOf [1 2 3 5 7] 5) ;; 3
(.indexOf [1 2 3 5 7] 4) ;; -1
(.indexOf [1 2 3 5 2] 2 3) ;; 4
----

[[lastIndexOf]]
=== lastIndexOf

==== Syntax

[source,clojure]
----
(.lastIndexOf coll search-value)
(.lastIndexOf coll search-value from-index)
----

==== Parameters

`coll` - a collection satisfying the `sequential?` predicate.

`search-value` - the value to search for.

`from-index` - an optional starting index.

==== Description

Gets the last index of a value in a sequential collection, or `-1` if
not found. By default, search proceeds from the end, unless an
optional starting index is supplied.

==== Examples

[source,clojure]
----
(.lastIndexOf [1 2 3 5 2 7] 2) ;; 4
(.lastIndexOf [1 2 3 5 2 7] 4) ;; -1
(.lastIndexOf [1 2 3 5 2 7] 2 3) ;; 1
----


Added deletecollection to the verb list

Added deletecollection to the verb list

= Authorization
{product-author}
{product-version}
:data-uri:
:icons:
:experimental:
:toc: macro
:toc-title:
:prewrap!:

toc::[]

== Overview
Authorization policies determine whether a user is allowed to perform a given
link:#action[action] within a project. This allows platform administrators to
use the link:#cluster-policy-and-local-policy[cluster policy] to control who has
various access levels to the OpenShift platform itself and all projects. It also
allows developers to use link:#cluster-policy-and-local-policy[local policy] to
control who has access to their
link:../core_concepts/projects_and_users.html#projects[projects]. Note that
authorization is a separate step from link:authentication.html[authentication],
which is more about determining the identity of who is taking the action.

Authorization is managed using:

[cols="1,7"]
|===

|[[rules-def]]*Rules* |Sets of permitted link:#action[verbs] on a set of
link:../core_concepts/index.html[objects]. For example, whether something can
`create` pods.

|[[roles-def]]*Roles* |Collections of rules.
link:authentication.html#users-and-groups[Users and groups] can be associated
with, or _bound_ to, multiple link:#roles[roles] at the same time.

|[[bindings]]*Bindings* |Associations between users and/or groups with a
link:#roles[role].

|===

Cluster administrators can visualize rules, roles, and bindings
ifdef::openshift-enterprise,openshift-origin[]
link:../../admin_guide/manage_authorization_policy.html#viewing-roles-and-bindings[using
the CLI].
endif::[]
ifdef::openshift-dedicated[]
using the CLI.
endif::[]
For example, consider the following excerpt from viewing a policy, showing rule
sets for the *admin* and *basic-user* link:#roles[default roles]:

====

[options="nowrap"]
----
admin			Verbs					Resources															Resource Names	Extension
			[create delete get list update watch]	[projects resourcegroup:exposedkube resourcegroup:exposedopenshift resourcegroup:granter secrets]				[]
			[get list watch]			[resourcegroup:allkube resourcegroup:allkube-status resourcegroup:allopenshift-status resourcegroup:policy]			[]
basic-user		Verbs					Resources															Resource Names	Extension
			[get]					[users]																[~]
			[list]					[projectrequests]														[]
			[list]					[projects]															[]
			[create]				[subjectaccessreviews]														[]		IsPersonalSubjectAccessReview
----
====

The following excerpt from viewing policy bindings shows the above roles bound
to various users and groups:

====

[options="nowrap"]
----
RoleBinding[admins]:
				Role:	admin
				Users:	[alice system:admin]
				Groups:	[]
RoleBinding[basic-user]:
				Role:	basic-user
				Users:	[joe]
				Groups:	[devel]
----
====

The relationships between the the policy roles, policy bindings, users, and
developers are illustrated below.

image::authorization.png[OpenShift Authorization Policy]

[[evaluating-authorization]]

== Evaluating Authorization

Several factors are combined to make the decision when OpenShift evaluates
authorization:

[cols="1,7"]
|===

|[[identity]]*Identity* |In the context of authorization, both the user name and
list of groups the user belongs to.

|[[action]]*Action* a|The action being performed. In most cases, this consists of:

[horizontal]
Project:: The link:../core_concepts/projects_and_users.html#projects[project]
being accessed.
Verb:: Can be `get`, `list`, `create`, `update`, `delete`, `deletecollection` or `watch`.
Resource Name:: The API endpoint being accessed.

|*Bindings* |The full list of link:#bindings[bindings].

|===

OpenShift evaluates authorizations using the following steps:

. The identity and the project-scoped action is used to find all bindings that
apply to the user or their groups.
. Bindings are used to locate all the roles that apply.
. Roles are used to find all the rules that apply.
. The action is checked against each rule to find a match.
. If no matching rule is found, the action is then denied by default.

[[cluster-policy-and-local-policy]]

== Cluster Policy and Local Policy
There are two levels of authorization policy:

[cols="1,4"]
|===

|*Cluster policy* |link:#roles[Roles] and bindings that are applicable across
all projects. Roles that exist in the cluster policy are considered _cluster
roles_. Cluster bindings can only reference cluster roles.

|*Local policy* |link:#roles[Roles] and bindings that are scoped to a given
project. Roles that exist only in a local policy are considered _local roles_.
Local bindings can reference both cluster and local roles.

|===

This two-level hierarchy allows re-usability over multiple projects through the
cluster policy while allowing customization inside of individual projects
through local policies.

During evaluation, both the cluster bindings and the local bindings are used.
For example:

. Cluster-wide "allow" rules are checked.
. Locally-bound "allow" rules are checked.
. Deny by default.

[[roles]]

== Roles
Roles are collections of policy link:#rules-def[rules], which are sets of
permitted verbs that can be performed on a set of resources. OpenShift includes
a set of default roles that can be added to users and groups in the
link:#cluster-policy-and-local-policy[cluster policy] or in a
link:#cluster-policy-and-local-policy[local policy].

[cols="1,4",options="header"]
|===

|Default Role |Description

|*admin* |A project manager. If used in a
link:#cluster-policy-and-local-policy[local binding], an *admin* user will have
rights to view any resource in the project and modify any resource in the
project except for role creation and quota. If the *cluster-admin* wants to
allow an *admin* to modify roles, the *cluster-admin* must create a
project-scoped `*Policy*` object using JSON.

|*basic-user* |A user that can get basic information about projects and users.

|*cluster-admin* |A super-user that can perform any action in any project. When
granted to a user within a local policy, they have full control over quota and
roles and every action on every resource in the project.

|*cluster-status* |A user that can get basic cluster status information.

|*edit* |A user that can modify most objects in a project, but does not have the
power to view or modify roles or bindings.

|*self-provisioner* |A user that can create their own projects.

|*view* |A user who cannot make any modifications, but can see most objects in a
project. They cannot view or modify roles or bindings.

|===

TIP: Remember that link:authentication.html#users-and-groups[users
and groups] can be associated with, or _bound_ to, multiple roles at the same
time.

Cluster administrators can visualize these roles, including a matrix of the
verbs and resources each are associated using the CLI to
ifdef::openshift-enterprise,openshift-origin[]
link:../../admin_guide/manage_authorization_policy.html#viewing-cluster-roles[view
the cluster roles].
endif::[]
ifdef::openshift-dedicated[]
view the cluster roles.
endif::[]
Additional *system:* roles are listed as well, which
are used for various OpenShift system and component operations.

By default in a local policy, only the binding for the *admin* role is
immediately listed when using the CLI to
ifdef::openshift-enterprise,openshift-origin[]
link:../../admin_guide/manage_authorization_policy.html#viewing-local-bindings[view
local bindings].
endif::[]
ifdef::openshift-dedicated[]
view local bindings.
endif::[]
However, if other default roles are added to users and groups within a local
policy, they become listed in the CLI output, as well.

If you find that these roles do not suit you, a *cluster-admin* user can create
a `*policyBinding*` object named `_<projectname>_:default` with the CLI using a
JSON file. This allows the project *admin* to bind users to roles that are
defined only in the `_<projectname>_` local policy.

ifdef::openshift-enterprise,openshift-origin[]
[[updating-cluster-roles]]

=== Updating Cluster Roles

After any link:../../install_config/upgrading/index.html[OpenShift cluster
upgrade], the recommended default roles may have been updated. See
link:../../install_config/upgrading/manual_upgrades.html#updating-policy-definitions[Updating
Policy Definitions] for instructions on getting to the new recommendations
using:

----
$ oadm policy reconcile-cluster-roles
----
endif::[]

[[security-context-constraints]]

== Security Context Constraints
In addition to link:#overview[authorization policies] that control what a user
can do, OpenShift provides _security context constraints_ (SCC) that control the
actions that a link:../core_concepts/pods_and_services.html#pods[pod] can
perform and what it has the ability to access. Administrators can
link:../../admin_guide/manage_scc.html[manage SCCs] using the CLI.
SCCs are also very useful for
link:../../install_config/persistent_storage/pod_security_context.html[managing
access to persistent storage].

SCCs are objects that define a set of conditions that a pod must run with in
order to be accepted into the system. They allow an administrator to control the
following:

ifdef::openshift-enterprise,openshift-origin[]
. Running of
link:../../install_config/install/prerequisites.html#security-warning[privileged
containers].
endif::[]
ifdef::openshift-dedicated[]
. Running of privileged containers.
endif::[]
. Capabilities a container can request to be added.
. Use of host directories as volumes.
. The SELinux context of the container.
. The user ID.
. The use of host namespaces and networking.
. Allocating an `*FSGroup*` that owns the pod's volumes
. Configuring allowable supplemental groups

Six SCCs are added to the cluster by default, and are viewable by cluster
administrators using the CLI:

====
----
$ oc get scc
NAME               PRIV      CAPS      HOSTDIR   SELINUX     RUNASUSER          FSGROUP    SUPGROUP   PRIORITY
anyuid             false     []        false     MustRunAs   RunAsAny           RunAsAny   RunAsAny   10
hostaccess         false     []        true      MustRunAs   MustRunAsRange     RunAsAny   RunAsAny   <none>
hostmount-anyuid   false     []        true      MustRunAs   RunAsAny           RunAsAny   RunAsAny   <none>
nonroot            false     []        false     MustRunAs   MustRunAsNonRoot   RunAsAny   RunAsAny   <none>
privileged         true      []        true      RunAsAny    RunAsAny           RunAsAny   RunAsAny   <none>
restricted         false     []        false     MustRunAs   MustRunAsRange     RunAsAny   RunAsAny   <none>
----
====

The definition for each SCC is also viewable by cluster administrators using the
CLI. For example, for the privileged SCC:

====
----
# oc export scc/privileged

allowHostDirVolumePlugin: true
allowHostIPC: true
allowHostNetwork: true
allowHostPID: true
allowHostPorts: true
allowPrivilegedContainer: true
allowedCapabilities: null
apiVersion: v1
fsGroup: <1>
  type: RunAsAny
groups: <2>
- system:cluster-admins
- system:nodes
kind: SecurityContextConstraints
metadata:
  annotations:
    kubernetes.io/description: 'privileged allows access to all privileged and host
      features and the ability to run as any user, any group, any fsGroup, and with
      any SELinux context.  WARNING: this is the most relaxed SCC and should be used
      only for cluster administration. Grant with caution.'
  creationTimestamp: null
  name: privileged
priority: null
runAsUser: <3>
  type: RunAsAny
seLinuxContext: <4>
  type: RunAsAny
supplementalGroups: <5>
  type: RunAsAny
users: <6>
- system:serviceaccount:default:registry
- system:serviceaccount:default:router
- system:serviceaccount:openshift-infra:build-controller
----

<1> The `*FSGroup*` strategy which dictates the allowable values for the
Security Context
<2> The groups that have access to this SCC
<3> The run as user strategy type which dictates the allowable values for the
Security Context
<4> The SELinux context strategy type which dictates the allowable values for
the Security Context
<5> The supplemental groups strategy which dictates the allowable supplemental
groups for the Security Context
<6> The users who have access to this SCC

====

The `*users*` and `*groups*` fields on the SCC control which SCCs can be used.
By default, cluster administrators, nodes, and the build controller are granted
access to the privileged SCC. All authenticated users are granted access to the
restricted SCC.

The privileged SCC:

- allows privileged pods.
- allows host directories to be mounted as volumes.
- allows a pod to run as any user.
- allows a pod to run with any MCS label.
- allows a pod to use the host's IPC namespace.
- allows a pod to use the host's PID namespace.
- allows a pod to use any FSGroup.
- allows a pod to use any supplemental group.

The restricted SCC:

- ensures pods cannot run as privileged.
- ensures pods cannot use host directory volumes.
- requires that a pod run as a user in a pre-allocated range of UIDs.
- requires that a pod run with a pre-allocated MCS label.
- allows a pod to use any FSGroup.
- allows a pod to use any supplemental group.

[NOTE]
====
For more information about each SCC, see the *kubernetes.io/description*
annotation available on the SCC.
====

SCCs are comprised of settings and strategies that control the security features
a pod has access to. These settings fall into three categories:

[cols="1,4"]
|===

|*Controlled by a boolean*
|Fields of this type default to the most restrictive value. For example,
`*AllowPrivilegedContainer*` is always set to *false* if unspecified.

|*Controlled by an allowable set*
|Fields of this type are checked against the set to ensure their value is
allowed.

|*Controlled by a strategy*
a|Items that have a strategy to generate a value provide:

- A mechanism to generate the value, and
- A mechanism to ensure that a specified value falls into the set of allowable
values.

|===

=== SCC Strategies

==== RunAsUser

. *MustRunAs* - Requires a `*runAsUser*` to be configured. Uses the configured
`*runAsUser*` as the default. Validates against the configured `*runAsUser*`.
. *MustRunAsRange* - Requires minimum and maximum values to be defined if not
using pre-allocated values. Uses the minimum as the default. Validates against
the entire allowable range.
. *MustRunAsNonRoot* - Requires that the pod be submitted with a non-zero
`*runAsUser*` or have the `USER` directive defined in the image. No default
provided.
. *RunAsAny* - No default provided. Allows any `*runAsUser*` to be specified.

==== SELinuxContext

. *MustRunAs* - Requires `*seLinuxOptions*` to be configured if not using
pre-allocated values. Uses `*seLinuxOptions*` as the default. Validates against
`*seLinuxOptions*`.
. *RunAsAny* - No default provided. Allows any `*seLinuxOptions*` to be
specified.

==== SupplementalGroups

. *MustRunAs* - Requires at least one range to be specified if not using
pre-allocated values. Uses the minimum value of the first range as the default.
Validates against all ranges.
. *RunAsAny* - No default provided. Allows any `*supplementalGroups*` to be
specified.

==== FSGroup

. *MustRunAs* - Requires at least one range to be specified if not using
pre-allocated values. Uses the minimum value of the first range as the default.
Validates against the first ID in the first range.
. *RunAsAny* - No default provided. Allows any `*fsGroup*` ID to be specified.

[[admission]]

=== Admission
_Admission control_ with SCCs allows for control over the creation of resources
based on the capabilities granted to a user.

In terms of the SCCs, this means that an admission controller can inspect the
user information made available in the context to retrieve an appropriate set of
SCCs. Doing so ensures the pod is authorized to make requests about its
operating environment or to generate a set of constraints to apply to the pod.

The set of SCCs that admission uses to authorize a pod are determined by the
user identity and groups that the user belongs to. Additionally, if the pod
specifies a service account, the set of allowable SCCs includes any constraints
accessible to the service account.

Admission uses the following approach to create the final security context for
the pod:

. Retrieve all SCCs available for use.
. Generate field values for security context settings that were not specified
on the request.
. Validate the final settings against the available constraints.

If a matching set of constraints is found, then the pod is accepted. If the
request cannot be matched to an SCC, the pod is rejected.

A pod must validate every field against the SCC. The following are examples for
just two of the fields that must be validated:

[NOTE]
====
These examples are in the context of a strategy using the preallocated values.
====

*A FSGroup SCC Strategy of MustRunAs*

If the pod defines a `*fsGroup*` ID, then that ID must equal the default
`*FSGroup*` ID. Otherwise, the pod is not validated by that SCC and the next SCC
is evaluated. If the `*FSGroup*` strategy is *RunAsAny* and the pod omits a
`*fsGroup*` ID, then the pod matches the SCC based on `*FSGroup*` (though other
strategies may not validate and thus cause the pod to fail).

*A SupplementalGroups SCC Strategy of MustRunAs*

If the pod specification defines one or more `*SupplementalGroups*` IDs, then
the pod's IDs must equal one of the IDs in the namespace's
*openshift.io/sa.scc.supplemental-groups* annotation. Otherwise, the pod is not
validated by that SCC and the next SCC is evaluated. If the
`*SupplementalGroups*` setting is *RunAsAny* and the pod specification omits a
`*SupplementalGroups*` ID, then the pod matches the SCC based on
`*SupplementalGroups*` (though other strategies may not validate and thus cause
the pod to fail).

==== SCC Prioritization

SCCs have a priority field that affects the ordering when attempting to
validate a request by the admission controller.  A higher priority
SCC is moved to the front of the set when sorting.  When the complete set
of available SCCs are determined they are ordered by:

. Highest priority first, nil is considered a 0 priority
. If priorities are equal, the SCCs will be sorted from most restrictive to least restrictive
. If both priorities and restrictions are equal the SCCs will be sorted by name

By default, the anyuid SCC granted to cluster administrators is given priority
in their SCC set.  This allows cluster administrators to run pods as any
user by without specifying a `RunAsUser` on the pod's `SecurityContext`.  The
administrator may still specify a `RunAsUser` if they wish.

==== Understanding Pre-allocated Values and Security Context Constraints

The admission controller is aware of certain conditions in the security context
constraints that trigger it to look up pre-allocated values from a namespace and
populate the security context constraint before processing the pod. Each SCC
strategy is evaluated independently of other strategies, with the pre-allocated
values (where allowed) for each policy aggregated with pod specification values
to make the final values for the various IDs defined in the running pod.

The following SCCs cause the admission controller to look for pre-allocated
values when no ranges are defined in the pod specification:

. A `*RunAsUser*` strategy of *MustRunAsRange* with no minimum or maximum set.
Admission looks for the *openshift.io/sa.scc.uid-range* annotation to populate
range fields.
. An `*SELinuxContext*` strategy of *MustRunAs* with no level set. Admission
looks for the *openshift.io/sa.scc.mcs* annotation to populate the level.
. A `*FSGroup*` strategy of *MustRunAs*. Admission looks for the
*openshift.io/sa.scc.supplemental-groups* annotation.
. A `*SupplementalGroups*` strategy of *MustRunAs*. Admission looks for the
*openshift.io/sa.scc.supplemental-groups* annotation.

During the generation phase, the security context provider will default any
values that are not specifically set in the pod. Defaulting is based on the
strategy being used:

. `*RunAsAny*` and `*MustRunAsNonRoot*` strategies do not provide default
values. Thus, if the pod needs a field defined (for example, a group ID), this
field must be defined inside the pod specification.
. `*MustRunAs*` (single value) strategies provide a default value which is
always used. As an example, for group IDs: even if the pod specification defines
its own ID value, the namespace's default field will also appear in the pod's
groups.
. `*MustRunAsRange*` and `*MustRunAs*` (range-based) strategies provide the
minimum value of the range. As with a single value `*MustRunAs*` strategy, the
namespace's default value will appear in the running pod. If a range-based
strategy is configurable with multiple ranges, it will provide the minimum value
of the first configured range.

[NOTE]
====
`*FSGroup*` and `*SupplementalGroups*` strategies fall back to the
*openshift.io/sa.scc.uid-range* annotation if the
*openshift.io/sa.scc.supplemental-groups* annotation does not exist on the
namespace. If neither exist, the SCC will fail to create.
====

[NOTE]
====
By default, the annotation-based `*FSGroup*` strategy configures itself with a
single range based on the minimum value for the annotation. For example, if your
annotation reads *1/3*, the `*FSGroup*` strategy will configure itself with a
minimum and maximum of *1*. If you want to allow more groups to be accepted for
the `*FSGroup*` field, you can configure a custom SCC that does not use the
annotation.
====

[NOTE]
====
The *openshift.io/sa.scc.supplemental-groups* annotation accepts a comma
delimited list of blocks in the format of `<start>/<length` or `<start>-<end>`.
The *openshift.io/sa.scc.uid-range* annotation accepts only a single block.
====

= Authorization
{product-author}
{product-version}
:data-uri:
:icons:
:experimental:
:toc: macro
:toc-title:
:prewrap!:

toc::[]

== Overview
Authorization policies determine whether a user is allowed to perform a given
link:#action[action] within a project. This allows platform administrators to
use the link:#cluster-policy-and-local-policy[cluster policy] to control who has
various access levels to the OpenShift platform itself and all projects. It also
allows developers to use link:#cluster-policy-and-local-policy[local policy] to
control who has access to their
link:../core_concepts/projects_and_users.html#projects[projects]. Note that
authorization is a separate step from link:authentication.html[authentication],
which is more about determining the identity of who is taking the action.

Authorization is managed using:

[cols="1,7"]
|===

|[[rules-def]]*Rules* |Sets of permitted link:#action[verbs] on a set of
link:../core_concepts/index.html[objects]. For example, whether something can
`create` pods.

|[[roles-def]]*Roles* |Collections of rules.
link:authentication.html#users-and-groups[Users and groups] can be associated
with, or _bound_ to, multiple link:#roles[roles] at the same time.

|[[bindings]]*Bindings* |Associations between users and/or groups with a
link:#roles[role].

|===

Cluster administrators can visualize rules, roles, and bindings
ifdef::openshift-enterprise,openshift-origin[]
link:../../admin_guide/manage_authorization_policy.html#viewing-roles-and-bindings[using
the CLI].
endif::[]
ifdef::openshift-dedicated[]
using the CLI.
endif::[]
For example, consider the following excerpt from viewing a policy, showing rule
sets for the *admin* and *basic-user* link:#roles[default roles]:

====

[options="nowrap"]
----
admin			Verbs					Resources															Resource Names	Extension
			[create delete get list update watch]	[projects resourcegroup:exposedkube resourcegroup:exposedopenshift resourcegroup:granter secrets]				[]
			[get list watch]			[resourcegroup:allkube resourcegroup:allkube-status resourcegroup:allopenshift-status resourcegroup:policy]			[]
basic-user		Verbs					Resources															Resource Names	Extension
			[get]					[users]																[~]
			[list]					[projectrequests]														[]
			[list]					[projects]															[]
			[create]				[subjectaccessreviews]														[]		IsPersonalSubjectAccessReview
----
====

The following excerpt from viewing policy bindings shows the above roles bound
to various users and groups:

====

[options="nowrap"]
----
RoleBinding[admins]:
				Role:	admin
				Users:	[alice system:admin]
				Groups:	[]
RoleBinding[basic-user]:
				Role:	basic-user
				Users:	[joe]
				Groups:	[devel]
----
====

The relationships between the the policy roles, policy bindings, users, and
developers are illustrated below.

image::authorization.png[OpenShift Authorization Policy]

[[evaluating-authorization]]

== Evaluating Authorization

Several factors are combined to make the decision when OpenShift evaluates
authorization:

[cols="1,7"]
|===

|[[identity]]*Identity* |In the context of authorization, both the user name and
list of groups the user belongs to.

|[[action]]*Action* a|The action being performed. In most cases, this consists of:

[horizontal]
Project:: The link:../core_concepts/projects_and_users.html#projects[project]
being accessed.
Verb:: Can be `get`, `list`, `create`, `update`, `delete`, or `watch`.
Resource Name:: The API endpoint being accessed.

|*Bindings* |The full list of link:#bindings[bindings].

|===

OpenShift evaluates authorizations using the following steps:

. The identity and the project-scoped action is used to find all bindings that
apply to the user or their groups.
. Bindings are used to locate all the roles that apply.
. Roles are used to find all the rules that apply.
. The action is checked against each rule to find a match.
. If no matching rule is found, the action is then denied by default.

[[cluster-policy-and-local-policy]]

== Cluster Policy and Local Policy
There are two levels of authorization policy:

[cols="1,4"]
|===

|*Cluster policy* |link:#roles[Roles] and bindings that are applicable across
all projects. Roles that exist in the cluster policy are considered _cluster
roles_. Cluster bindings can only reference cluster roles.

|*Local policy* |link:#roles[Roles] and bindings that are scoped to a given
project. Roles that exist only in a local policy are considered _local roles_.
Local bindings can reference both cluster and local roles.

|===

This two-level hierarchy allows re-usability over multiple projects through the
cluster policy while allowing customization inside of individual projects
through local policies.

During evaluation, both the cluster bindings and the local bindings are used.
For example:

. Cluster-wide "allow" rules are checked.
. Locally-bound "allow" rules are checked.
. Deny by default.

[[roles]]

== Roles
Roles are collections of policy link:#rules-def[rules], which are sets of
permitted verbs that can be performed on a set of resources. OpenShift includes
a set of default roles that can be added to users and groups in the
link:#cluster-policy-and-local-policy[cluster policy] or in a
link:#cluster-policy-and-local-policy[local policy].

[cols="1,4",options="header"]
|===

|Default Role |Description

|*admin* |A project manager. If used in a
link:#cluster-policy-and-local-policy[local binding], an *admin* user will have
rights to view any resource in the project and modify any resource in the
project except for role creation and quota. If the *cluster-admin* wants to
allow an *admin* to modify roles, the *cluster-admin* must create a
project-scoped `*Policy*` object using JSON.

|*basic-user* |A user that can get basic information about projects and users.

|*cluster-admin* |A super-user that can perform any action in any project. When
granted to a user within a local policy, they have full control over quota and
roles and every action on every resource in the project.

|*cluster-status* |A user that can get basic cluster status information.

|*edit* |A user that can modify most objects in a project, but does not have the
power to view or modify roles or bindings.

|*self-provisioner* |A user that can create their own projects.

|*view* |A user who cannot make any modifications, but can see most objects in a
project. They cannot view or modify roles or bindings.

|===

TIP: Remember that link:authentication.html#users-and-groups[users
and groups] can be associated with, or _bound_ to, multiple roles at the same
time.

Cluster administrators can visualize these roles, including a matrix of the
verbs and resources each are associated using the CLI to
ifdef::openshift-enterprise,openshift-origin[]
link:../../admin_guide/manage_authorization_policy.html#viewing-cluster-roles[view
the cluster roles].
endif::[]
ifdef::openshift-dedicated[]
view the cluster roles.
endif::[]
Additional *system:* roles are listed as well, which
are used for various OpenShift system and component operations.

By default in a local policy, only the binding for the *admin* role is
immediately listed when using the CLI to
ifdef::openshift-enterprise,openshift-origin[]
link:../../admin_guide/manage_authorization_policy.html#viewing-local-bindings[view
local bindings].
endif::[]
ifdef::openshift-dedicated[]
view local bindings.
endif::[]
However, if other default roles are added to users and groups within a local
policy, they become listed in the CLI output, as well.

If you find that these roles do not suit you, a *cluster-admin* user can create
a `*policyBinding*` object named `_<projectname>_:default` with the CLI using a
JSON file. This allows the project *admin* to bind users to roles that are
defined only in the `_<projectname>_` local policy.

ifdef::openshift-enterprise,openshift-origin[]
[[updating-cluster-roles]]

=== Updating Cluster Roles

After any link:../../install_config/upgrading/index.html[OpenShift cluster
upgrade], the recommended default roles may have been updated. See
link:../../install_config/upgrading/manual_upgrades.html#updating-policy-definitions[Updating
Policy Definitions] for instructions on getting to the new recommendations
using:

----
$ oadm policy reconcile-cluster-roles
----
endif::[]

[[security-context-constraints]]

== Security Context Constraints
In addition to link:#overview[authorization policies] that control what a user
can do, OpenShift provides _security context constraints_ (SCC) that control the
actions that a link:../core_concepts/pods_and_services.html#pods[pod] can
perform and what it has the ability to access. Administrators can
link:../../admin_guide/manage_scc.html[manage SCCs] using the CLI.
SCCs are also very useful for
link:../../install_config/persistent_storage/pod_security_context.html[managing
access to persistent storage].

SCCs are objects that define a set of conditions that a pod must run with in
order to be accepted into the system. They allow an administrator to control the
following:

ifdef::openshift-enterprise,openshift-origin[]
. Running of
link:../../install_config/install/prerequisites.html#security-warning[privileged
containers].
endif::[]
ifdef::openshift-dedicated[]
. Running of privileged containers.
endif::[]
. Capabilities a container can request to be added.
. Use of host directories as volumes.
. The SELinux context of the container.
. The user ID.
. The use of host namespaces and networking.
. Allocating an `*FSGroup*` that owns the pod's volumes
. Configuring allowable supplemental groups

Six SCCs are added to the cluster by default, and are viewable by cluster
administrators using the CLI:

====
----
$ oc get scc
NAME               PRIV      CAPS      HOSTDIR   SELINUX     RUNASUSER          FSGROUP    SUPGROUP   PRIORITY
anyuid             false     []        false     MustRunAs   RunAsAny           RunAsAny   RunAsAny   10
hostaccess         false     []        true      MustRunAs   MustRunAsRange     RunAsAny   RunAsAny   <none>
hostmount-anyuid   false     []        true      MustRunAs   RunAsAny           RunAsAny   RunAsAny   <none>
nonroot            false     []        false     MustRunAs   MustRunAsNonRoot   RunAsAny   RunAsAny   <none>
privileged         true      []        true      RunAsAny    RunAsAny           RunAsAny   RunAsAny   <none>
restricted         false     []        false     MustRunAs   MustRunAsRange     RunAsAny   RunAsAny   <none>
----
====

The definition for each SCC is also viewable by cluster administrators using the
CLI. For example, for the privileged SCC:

====
----
# oc export scc/privileged

allowHostDirVolumePlugin: true
allowHostIPC: true
allowHostNetwork: true
allowHostPID: true
allowHostPorts: true
allowPrivilegedContainer: true
allowedCapabilities: null
apiVersion: v1
fsGroup: <1>
  type: RunAsAny
groups: <2>
- system:cluster-admins
- system:nodes
kind: SecurityContextConstraints
metadata:
  annotations:
    kubernetes.io/description: 'privileged allows access to all privileged and host
      features and the ability to run as any user, any group, any fsGroup, and with
      any SELinux context.  WARNING: this is the most relaxed SCC and should be used
      only for cluster administration. Grant with caution.'
  creationTimestamp: null
  name: privileged
priority: null
runAsUser: <3>
  type: RunAsAny
seLinuxContext: <4>
  type: RunAsAny
supplementalGroups: <5>
  type: RunAsAny
users: <6>
- system:serviceaccount:default:registry
- system:serviceaccount:default:router
- system:serviceaccount:openshift-infra:build-controller
----

<1> The `*FSGroup*` strategy which dictates the allowable values for the
Security Context
<2> The groups that have access to this SCC
<3> The run as user strategy type which dictates the allowable values for the
Security Context
<4> The SELinux context strategy type which dictates the allowable values for
the Security Context
<5> The supplemental groups strategy which dictates the allowable supplemental
groups for the Security Context
<6> The users who have access to this SCC

====

The `*users*` and `*groups*` fields on the SCC control which SCCs can be used.
By default, cluster administrators, nodes, and the build controller are granted
access to the privileged SCC. All authenticated users are granted access to the
restricted SCC.

The privileged SCC:

- allows privileged pods.
- allows host directories to be mounted as volumes.
- allows a pod to run as any user.
- allows a pod to run with any MCS label.
- allows a pod to use the host's IPC namespace.
- allows a pod to use the host's PID namespace.
- allows a pod to use any FSGroup.
- allows a pod to use any supplemental group.

The restricted SCC:

- ensures pods cannot run as privileged.
- ensures pods cannot use host directory volumes.
- requires that a pod run as a user in a pre-allocated range of UIDs.
- requires that a pod run with a pre-allocated MCS label.
- allows a pod to use any FSGroup.
- allows a pod to use any supplemental group.

[NOTE]
====
For more information about each SCC, see the *kubernetes.io/description*
annotation available on the SCC.
====

SCCs are comprised of settings and strategies that control the security features
a pod has access to. These settings fall into three categories:

[cols="1,4"]
|===

|*Controlled by a boolean*
|Fields of this type default to the most restrictive value. For example,
`*AllowPrivilegedContainer*` is always set to *false* if unspecified.

|*Controlled by an allowable set*
|Fields of this type are checked against the set to ensure their value is
allowed.

|*Controlled by a strategy*
a|Items that have a strategy to generate a value provide:

- A mechanism to generate the value, and
- A mechanism to ensure that a specified value falls into the set of allowable
values.

|===

=== SCC Strategies

==== RunAsUser

. *MustRunAs* - Requires a `*runAsUser*` to be configured. Uses the configured
`*runAsUser*` as the default. Validates against the configured `*runAsUser*`.
. *MustRunAsRange* - Requires minimum and maximum values to be defined if not
using pre-allocated values. Uses the minimum as the default. Validates against
the entire allowable range.
. *MustRunAsNonRoot* - Requires that the pod be submitted with a non-zero
`*runAsUser*` or have the `USER` directive defined in the image. No default
provided.
. *RunAsAny* - No default provided. Allows any `*runAsUser*` to be specified.

==== SELinuxContext

. *MustRunAs* - Requires `*seLinuxOptions*` to be configured if not using
pre-allocated values. Uses `*seLinuxOptions*` as the default. Validates against
`*seLinuxOptions*`.
. *RunAsAny* - No default provided. Allows any `*seLinuxOptions*` to be
specified.

==== SupplementalGroups

. *MustRunAs* - Requires at least one range to be specified if not using
pre-allocated values. Uses the minimum value of the first range as the default.
Validates against all ranges.
. *RunAsAny* - No default provided. Allows any `*supplementalGroups*` to be
specified.

==== FSGroup

. *MustRunAs* - Requires at least one range to be specified if not using
pre-allocated values. Uses the minimum value of the first range as the default.
Validates against the first ID in the first range.
. *RunAsAny* - No default provided. Allows any `*fsGroup*` ID to be specified.

[[admission]]

=== Admission
_Admission control_ with SCCs allows for control over the creation of resources
based on the capabilities granted to a user.

In terms of the SCCs, this means that an admission controller can inspect the
user information made available in the context to retrieve an appropriate set of
SCCs. Doing so ensures the pod is authorized to make requests about its
operating environment or to generate a set of constraints to apply to the pod.

The set of SCCs that admission uses to authorize a pod are determined by the
user identity and groups that the user belongs to. Additionally, if the pod
specifies a service account, the set of allowable SCCs includes any constraints
accessible to the service account.

Admission uses the following approach to create the final security context for
the pod:

. Retrieve all SCCs available for use.
. Generate field values for security context settings that were not specified
on the request.
. Validate the final settings against the available constraints.

If a matching set of constraints is found, then the pod is accepted. If the
request cannot be matched to an SCC, the pod is rejected.

A pod must validate every field against the SCC. The following are examples for
just two of the fields that must be validated:

[NOTE]
====
These examples are in the context of a strategy using the preallocated values.
====

*A FSGroup SCC Strategy of MustRunAs*

If the pod defines a `*fsGroup*` ID, then that ID must equal the default
`*FSGroup*` ID. Otherwise, the pod is not validated by that SCC and the next SCC
is evaluated. If the `*FSGroup*` strategy is *RunAsAny* and the pod omits a
`*fsGroup*` ID, then the pod matches the SCC based on `*FSGroup*` (though other
strategies may not validate and thus cause the pod to fail).

*A SupplementalGroups SCC Strategy of MustRunAs*

If the pod specification defines one or more `*SupplementalGroups*` IDs, then
the pod's IDs must equal one of the IDs in the namespace's
*openshift.io/sa.scc.supplemental-groups* annotation. Otherwise, the pod is not
validated by that SCC and the next SCC is evaluated. If the
`*SupplementalGroups*` setting is *RunAsAny* and the pod specification omits a
`*SupplementalGroups*` ID, then the pod matches the SCC based on
`*SupplementalGroups*` (though other strategies may not validate and thus cause
the pod to fail).

==== SCC Prioritization

SCCs have a priority field that affects the ordering when attempting to
validate a request by the admission controller.  A higher priority
SCC is moved to the front of the set when sorting.  When the complete set
of available SCCs are determined they are ordered by:

. Highest priority first, nil is considered a 0 priority
. If priorities are equal, the SCCs will be sorted from most restrictive to least restrictive
. If both priorities and restrictions are equal the SCCs will be sorted by name

By default, the anyuid SCC granted to cluster administrators is given priority
in their SCC set.  This allows cluster administrators to run pods as any
user by without specifying a `RunAsUser` on the pod's `SecurityContext`.  The
administrator may still specify a `RunAsUser` if they wish.

==== Understanding Pre-allocated Values and Security Context Constraints

The admission controller is aware of certain conditions in the security context
constraints that trigger it to look up pre-allocated values from a namespace and
populate the security context constraint before processing the pod. Each SCC
strategy is evaluated independently of other strategies, with the pre-allocated
values (where allowed) for each policy aggregated with pod specification values
to make the final values for the various IDs defined in the running pod.

The following SCCs cause the admission controller to look for pre-allocated
values when no ranges are defined in the pod specification:

. A `*RunAsUser*` strategy of *MustRunAsRange* with no minimum or maximum set.
Admission looks for the *openshift.io/sa.scc.uid-range* annotation to populate
range fields.
. An `*SELinuxContext*` strategy of *MustRunAs* with no level set. Admission
looks for the *openshift.io/sa.scc.mcs* annotation to populate the level.
. A `*FSGroup*` strategy of *MustRunAs*. Admission looks for the
*openshift.io/sa.scc.supplemental-groups* annotation.
. A `*SupplementalGroups*` strategy of *MustRunAs*. Admission looks for the
*openshift.io/sa.scc.supplemental-groups* annotation.

During the generation phase, the security context provider will default any
values that are not specifically set in the pod. Defaulting is based on the
strategy being used:

. `*RunAsAny*` and `*MustRunAsNonRoot*` strategies do not provide default
values. Thus, if the pod needs a field defined (for example, a group ID), this
field must be defined inside the pod specification.
. `*MustRunAs*` (single value) strategies provide a default value which is
always used. As an example, for group IDs: even if the pod specification defines
its own ID value, the namespace's default field will also appear in the pod's
groups.
. `*MustRunAsRange*` and `*MustRunAs*` (range-based) strategies provide the
minimum value of the range. As with a single value `*MustRunAs*` strategy, the
namespace's default value will appear in the running pod. If a range-based
strategy is configurable with multiple ranges, it will provide the minimum value
of the first configured range.

[NOTE]
====
`*FSGroup*` and `*SupplementalGroups*` strategies fall back to the
*openshift.io/sa.scc.uid-range* annotation if the
*openshift.io/sa.scc.supplemental-groups* annotation does not exist on the
namespace. If neither exist, the SCC will fail to create.
====

[NOTE]
====
By default, the annotation-based `*FSGroup*` strategy configures itself with a
single range based on the minimum value for the annotation. For example, if your
annotation reads *1/3*, the `*FSGroup*` strategy will configure itself with a
minimum and maximum of *1*. If you want to allow more groups to be accepted for
the `*FSGroup*` field, you can configure a custom SCC that does not use the
annotation.
====

[NOTE]
====
The *openshift.io/sa.scc.supplemental-groups* annotation accepts a comma
delimited list of blocks in the format of `<start>/<length` or `<start>-<end>`.
The *openshift.io/sa.scc.uid-range* annotation accepts only a single block.
====


[DOCS] Add rollups to `Tune for disk usage` (#60436)

[[tune-for-disk-usage]]
== Tune for disk usage

[discrete]
=== Disable the features you do not need

By default Elasticsearch indexes and adds doc values to most fields so that they
can be searched and aggregated out of the box. For instance if you have a numeric
field called `foo` that you need to run histograms on but that you never need to
filter on, you can safely disable indexing on this field in your
<<mappings,mappings>>:

[source,console]
--------------------------------------------------
PUT index
{
  "mappings": {
    "properties": {
      "foo": {
        "type": "integer",
        "index": false
      }
    }
  }
}
--------------------------------------------------

<<text,`text`>> fields store normalization factors in the index in order to be
able to score documents. If you only need matching capabilities on a `text`
field but do not care about the produced scores, you can configure Elasticsearch
to not write norms to the index:

[source,console]
--------------------------------------------------
PUT index
{
  "mappings": {
    "properties": {
      "foo": {
        "type": "text",
        "norms": false
      }
    }
  }
}
--------------------------------------------------

<<text,`text`>> fields also store frequencies and positions in the index by
default. Frequencies are used to compute scores and positions are used to run
phrase queries. If you do not need to run phrase queries, you can tell
Elasticsearch to not index positions:

[source,console]
--------------------------------------------------
PUT index
{
  "mappings": {
    "properties": {
      "foo": {
        "type": "text",
        "index_options": "freqs"
      }
    }
  }
}
--------------------------------------------------

Furthermore if you do not care about scoring either, you can configure
Elasticsearch to just index matching documents for every term. You will
still be able to search on this field, but phrase queries will raise errors
and scoring will assume that terms appear only once in every document.

[source,console]
--------------------------------------------------
PUT index
{
  "mappings": {
    "properties": {
      "foo": {
        "type": "text",
        "norms": false,
        "index_options": "freqs"
      }
    }
  }
}
--------------------------------------------------

[discrete]
[[default-dynamic-string-mapping]]
=== Don't use default dynamic string mappings

The default <<dynamic-mapping,dynamic string mappings>> will index string fields
both as <<text,`text`>> and <<keyword,`keyword`>>. This is wasteful if you only
need one of them. Typically an `id` field will only need to be indexed as a
`keyword` while a `body` field will only need to be indexed as a `text` field.

This can be disabled by either configuring explicit mappings on string fields
or setting up dynamic templates that will map string fields as either `text`
or `keyword`.

For instance, here is a template that can be used in order to only map string
fields as `keyword`:

[source,console]
--------------------------------------------------
PUT index
{
  "mappings": {
    "dynamic_templates": [
      {
        "strings": {
          "match_mapping_type": "string",
          "mapping": {
            "type": "keyword"
          }
        }
      }
    ]
  }
}
--------------------------------------------------

[discrete]
=== Watch your shard size

Larger shards are going to be more efficient at storing data. To increase the size of your shards, you can decrease the number of primary shards in an index by <<indices-create-index,creating indices>> with fewer primary shards, creating fewer indices (e.g. by leveraging the <<indices-rollover-index,Rollover API>>), or modifying an existing index using the <<indices-shrink-index,Shrink API>>.

Keep in mind that large shard sizes come with drawbacks, such as long full recovery times.

[discrete]
[[disable-source]]
=== Disable `_source`

The <<mapping-source-field,`_source`>> field stores the original JSON body of the document. If you don’t need access to it you can disable it. However, APIs that needs access to `_source` such as update and reindex won’t work.

[discrete]
[[best-compression]]
=== Use `best_compression`

The `_source` and stored fields can easily take a non negligible amount of disk
space. They can be compressed more aggressively by using the `best_compression`
<<index-codec,codec>>.

[discrete]
=== Force Merge

Indices in Elasticsearch are stored in one or more shards. Each shard is a Lucene index and made up of one or more segments - the actual files on disk. Larger segments are more efficient for storing data.

The <<indices-forcemerge,`_forcemerge` API>> can be used to reduce the number of segments per shard. In many cases, the number of segments can be reduced to one per shard by setting `max_num_segments=1`.

[discrete]
=== Shrink Index

The <<indices-shrink-index,Shrink API>> allows you to reduce the number of shards in an index. Together with the Force Merge API above, this can significantly reduce the number of shards and segments of an index.

[discrete]
=== Use the smallest numeric type that is sufficient

The type that you pick for <<number,numeric data>> can have a significant impact
on disk usage. In particular, integers should be stored using an integer type
(`byte`, `short`, `integer` or `long`) and floating points should either be
stored in a `scaled_float` if appropriate or in the smallest type that fits the
use-case: using `float` over `double`, or `half_float` over `float` will help
save storage.

[discrete]
=== Use index sorting to colocate similar documents

When Elasticsearch stores `_source`, it compresses multiple documents at once
in order to improve the overall compression ratio. For instance it is very
common that documents share the same field names, and quite common that they
share some field values, especially on fields that have a low cardinality or
a https://en.wikipedia.org/wiki/Zipf%27s_law[zipfian] distribution.

By default documents are compressed together in the order that they are added
to the index. If you enabled <<index-modules-index-sorting,index sorting>>
then instead they are compressed in sorted order. Sorting documents with similar
structure, fields, and values together should improve the compression ratio.

[discrete]
=== Put fields in the same order in documents

Due to the fact that multiple documents are compressed together into blocks,
it is more likely to find longer duplicate strings in those `_source` documents
if fields always occur in the same order.

[discrete]
[[roll-up-historical-data]]
=== Roll up historical data

Keeping older data can useful for later analysis but is often avoided due to
storage costs. You can use data rollups to summarize and store historical data
at a fraction of the raw data's storage cost. See <<xpack-rollup>>.


Update changelog

Update changelog