Compose file version 3 reference
Estimated reading time: 74 minutesReference and guidelines
These topics describe version 3 of the Compose file format. This is the newest version.
Compose and Docker compatibility matrix
There are several versions of the Compose file format – 1, 2, 2.x, and 3.x. The table below is a quick look. For full details on what each version includes and how to upgrade, see About versions and upgrading.
This table shows which Compose file versions support specific Docker releases.
| Compose file format | Docker Engine release |
|---|---|
| 3.7 | 18.06.0+ |
| 3.6 | 18.02.0+ |
| 3.5 | 17.12.0+ |
| 3.4 | 17.09.0+ |
| 3.3 | 17.06.0+ |
| 3.2 | 17.04.0+ |
| 3.1 | 1.13.1+ |
| 3.0 | 1.13.0+ |
| 2.4 | 17.12.0+ |
| 2.3 | 17.06.0+ |
| 2.2 | 1.13.0+ |
| 2.1 | 1.12.0+ |
| 2.0 | 1.10.0+ |
| 1.0 | 1.9.1.+ |
In addition to Compose file format versions shown in the table, the Compose itself is on a release schedule, as shown in Compose releases, but file format versions do not necessarily increment with each release. For example, Compose file format 3.0 was first introduced in Compose release 1.10.0, and versioned gradually in subsequent releases.
Compose file structure and examples
Here is a sample Compose file from the voting app sample used in the Docker for Beginners lab topic on Deploying an app to a Swarm:
version: "3.7"
services:
redis:
image: redis:alpine
ports:
- "6379"
networks:
- frontend
deploy:
replicas: 2
update_config:
parallelism: 2
delay: 10s
restart_policy:
condition: on-failure
db:
image: postgres:9.4
volumes:
- db-data:/var/lib/postgresql/data
networks:
- backend
deploy:
placement:
constraints: [node.role == manager]
vote:
image: dockersamples/examplevotingapp_vote:before
ports:
- "5000:80"
networks:
- frontend
depends_on:
- redis
deploy:
replicas: 2
update_config:
parallelism: 2
restart_policy:
condition: on-failure
result:
image: dockersamples/examplevotingapp_result:before
ports:
- "5001:80"
networks:
- backend
depends_on:
- db
deploy:
replicas: 1
update_config:
parallelism: 2
delay: 10s
restart_policy:
condition: on-failure
worker:
image: dockersamples/examplevotingapp_worker
networks:
- frontend
- backend
deploy:
mode: replicated
replicas: 1
labels: [APP=VOTING]
restart_policy:
condition: on-failure
delay: 10s
max_attempts: 3
window: 120s
placement:
constraints: [node.role == manager]
visualizer:
image: dockersamples/visualizer:stable
ports:
- "8080:8080"
stop_grace_period: 1m30s
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
deploy:
placement:
constraints: [node.role == manager]
networks:
frontend:
backend:
volumes:
db-data:
The topics on this reference page are organized alphabetically by top-level key
to reflect the structure of the Compose file itself. Top-level keys that define
a section in the configuration file such as build
, deploy
, depends_on
,
networks
, and so on, are listed with the options that support them as
sub-topics. This maps to the <key>: <option>: <value>
indent structure of the
Compose file.
Service configuration reference
The Compose file is a YAML file defining
services,
networks and
volumes.
The default path for a Compose file is ./docker-compose.yml
.
Tip: You can use either a
.yml
or.yaml
extension for this file. They both work.
A service definition contains configuration that is applied to each
container started for that service, much like passing command-line parameters to
docker container create
. Likewise, network and volume definitions are analogous to
docker network create
and docker volume create
.
As with docker container create
, options specified in the Dockerfile, such as CMD
,
EXPOSE
, VOLUME
, ENV
, are respected by default - you don’t need to
specify them again in docker-compose.yml
.
You can use environment variables in configuration values with a Bash-like
${VARIABLE}
syntax - see
variable substitution for full details.
This section contains a list of all configuration options supported by a service definition in version 3.
build
Configuration options that are applied at build time.
build
can be specified either as a string containing a path to the build
context:
version: "3.7"
services:
webapp:
build: ./dir
Or, as an object with the path specified under context and optionally Dockerfile and args:
version: "3.7"
services:
webapp:
build:
context: ./dir
dockerfile: Dockerfile-alternate
args:
buildno: 1
If you specify image
as well as build
, then Compose names the built image
with the webapp
and optional tag
specified in image
:
build: ./dir
image: webapp:tag
This results in an image named webapp
and tagged tag
, built from ./dir
.
Note: This option is ignored when deploying a stack in swarm mode with a (version 3) Compose file. The
docker stack
command accepts only pre-built images.
context
Either a path to a directory containing a Dockerfile, or a url to a git repository.
When the value supplied is a relative path, it is interpreted as relative to the location of the Compose file. This directory is also the build context that is sent to the Docker daemon.
Compose builds and tags it with a generated name, and uses that image thereafter.
build:
context: ./dir
dockerfile
Alternate Dockerfile.
Compose uses an alternate file to build with. A build path must also be specified.
build:
context: .
dockerfile: Dockerfile-alternate
args
Add build arguments, which are environment variables accessible only during the build process.
First, specify the arguments in your Dockerfile:
ARG buildno
ARG gitcommithash
RUN echo "Build number: $buildno"
RUN echo "Based on commit: $gitcommithash"
Then specify the arguments under the build
key. You can pass a mapping
or a list:
build:
context: .
args:
buildno: 1
gitcommithash: cdc3b19
build:
context: .
args:
- buildno=1
- gitcommithash=cdc3b19
Note: In your Dockerfile, if you specify
ARG
before theFROM
instruction,ARG
is not available in the build instructions underFROM
. If you need an argument to be available in both places, also specify it under theFROM
instruction. See Understand how ARGS and FROM interact for usage details.
You can omit the value when specifying a build argument, in which case its value at build time is the value in the environment where Compose is running.
args:
- buildno
- gitcommithash
Note: YAML boolean values (
true
,false
,yes
,no
,on
,off
) must be enclosed in quotes, so that the parser interprets them as strings.
cache_from
Note: This option is new in v3.2
A list of images that the engine uses for cache resolution.
build:
context: .
cache_from:
- alpine:latest
- corp/web_app:3.14
labels
Note: This option is new in v3.3
Add metadata to the resulting image using Docker labels. You can use either an array or a dictionary.
We recommend that you use reverse-DNS notation to prevent your labels from conflicting with those used by other software.
build:
context: .
labels:
com.example.description: "Accounting webapp"
com.example.department: "Finance"
com.example.label-with-empty-value: ""
build:
context: .
labels:
- "com.example.description=Accounting webapp"
- "com.example.department=Finance"
- "com.example.label-with-empty-value"
shm_size
Added in version 3.5 file format
Set the size of the /dev/shm
partition for this build’s containers. Specify
as an integer value representing the number of bytes or as a string expressing
a byte value.
build:
context: .
shm_size: '2gb'
build:
context: .
shm_size: 10000000
target
Added in version 3.4 file format
Build the specified stage as defined inside the Dockerfile
. See the
multi-stage build docs for
details.
build:
context: .
target: prod
cap_add, cap_drop
Add or drop container capabilities.
See man 7 capabilities
for a full list.
cap_add:
- ALL
cap_drop:
- NET_ADMIN
- SYS_ADMIN
Note: These options are ignored when deploying a stack in swarm mode with a (version 3) Compose file.
cgroup_parent
Specify an optional parent cgroup for the container.
cgroup_parent: m-executor-abcd
Note: This option is ignored when deploying a stack in swarm mode with a (version 3) Compose file.
command
Override the default command.
command: bundle exec thin -p 3000
The command can also be a list, in a manner similar to dockerfile:
command: ["bundle", "exec", "thin", "-p", "3000"]
configs
Grant access to configs on a per-service basis using the per-service configs
configuration. Two different syntax variants are supported.
Note: The config must already exist or be defined in the top-level
configs
configuration of this stack file, or stack deployment fails.
For more information on configs, see configs.
Short syntax
The short syntax variant only specifies the config name. This grants the
container access to the config and mounts it at /<config_name>
within the container. The source name and destination mountpoint are both set
to the config name.
The following example uses the short syntax to grant the redis
service
access to the my_config
and my_other_config
configs. The value of
my_config
is set to the contents of the file ./my_config.txt
, and
my_other_config
is defined as an external resource, which means that it has
already been defined in Docker, either by running the docker config create
command or by another stack deployment. If the external config does not exist,
the stack deployment fails with a config not found
error.
Note:
config
definitions are only supported in version 3.3 and higher of the compose file format.
version: "3.7"
services:
redis:
image: redis:latest
deploy:
replicas: 1
configs:
- my_config
- my_other_config
configs:
my_config:
file: ./my_config.txt
my_other_config:
external: true
Long syntax
The long syntax provides more granularity in how the config is created within the service’s task containers.
source
: The name of the config as it exists in Docker.target
: The path and name of the file to be mounted in the service’s task containers. Defaults to/<source>
if not specified.uid
andgid
: The numeric UID or GID that owns the mounted config file within in the service’s task containers. Both default to0
on Linux if not specified. Not supported on Windows.mode
: The permissions for the file that is mounted within the service’s task containers, in octal notation. For instance,0444
represents world-readable. The default is0444
. Configs cannot be writable because they are mounted in a temporary filesystem, so if you set the writable bit, it is ignored. The executable bit can be set. If you aren’t familiar with UNIX file permission modes, you may find this permissions calculator useful.
The following example sets the name of my_config
to redis_config
within the
container, sets the mode to 0440
(group-readable) and sets the user and group
to 103
. The redis
service does not have access to the my_other_config
config.
version: "3.7"
services:
redis:
image: redis:latest
deploy:
replicas: 1
configs:
- source: my_config
target: /redis_config
uid: '103'
gid: '103'
mode: 0440
configs:
my_config:
file: ./my_config.txt
my_other_config:
external: true
You can grant a service access to multiple configs and you can mix long and short syntax. Defining a config does not imply granting a service access to it.
container_name
Specify a custom container name, rather than a generated default name.
container_name: my-web-container
Because Docker container names must be unique, you cannot scale a service beyond 1 container if you have specified a custom name. Attempting to do so results in an error.
Note: This option is ignored when deploying a stack in swarm mode with a (version 3) Compose file.
credential_spec
Note: This option was added in v3.3. Using group Managed Service Account (gMSA) configurations with compose files is supported in Compose version 3.8.
Configure the credential spec for managed service account. This option is only
used for services using Windows containers. The credential_spec
must be in the
format file://<filename>
or registry://<value-name>
.
When using file:
, the referenced file must be present in the CredentialSpecs
subdirectory in the Docker data directory, which defaults to C:\ProgramData\Docker\
on Windows. The following example loads the credential spec from a file named
C:\ProgramData\Docker\CredentialSpecs\my-credential-spec.json
:
credential_spec:
file: my-credential-spec.json
When using registry:
, the credential spec is read from the Windows registry on
the daemon’s host. A registry value with the given name must be located in:
HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Virtualization\Containers\CredentialSpecs
The following example load the credential spec from a value named my-credential-spec
in the registry:
credential_spec:
registry: my-credential-spec
Example gMSA configuration
When configuring a gMSA credential spec for a service, you only need
to specify a credential spec with config
, as shown in the following example:
version: "3.8"
services:
myservice:
image: myimage:latest
credential_spec:
config: my_credential_spec
configs:
my_credentials_spec:
file: ./my-credential-spec.json|
depends_on
Express dependency between services, Service dependencies cause the following behaviors:
-
docker-compose up
starts services in dependency order. In the following example,db
andredis
are started beforeweb
. -
docker-compose up SERVICE
automatically includesSERVICE
’s dependencies. In the following example,docker-compose up web
also creates and startsdb
andredis
. -
docker-compose stop
stops services in dependency order. In the following example,web
is stopped beforedb
andredis
.
Simple example:
version: "3.7"
services:
web:
build: .
depends_on:
- db
- redis
redis:
image: redis
db:
image: postgres
There are several things to be aware of when using
depends_on
:
depends_on
does not wait fordb
andredis
to be “ready” before startingweb
- only until they have been started. If you need to wait for a service to be ready, see Controlling startup order for more on this problem and strategies for solving it.Version 3 no longer supports the
condition
form ofdepends_on
.The
depends_on
option is ignored when deploying a stack in swarm mode with a version 3 Compose file.
deploy
Version 3 only.
Specify configuration related to the deployment and running of services. This
only takes effect when deploying to a swarm with
docker stack deploy, and is
ignored by docker-compose up
and docker-compose run
.
version: "3.7"
services:
redis:
image: redis:alpine
deploy:
replicas: 6
update_config:
parallelism: 2
delay: 10s
restart_policy:
condition: on-failure
Several sub-options are available:
endpoint_mode
Specify a service discovery method for external clients connecting to a swarm.
Version 3.3 only.
-
endpoint_mode: vip
- Docker assigns the service a virtual IP (VIP) that acts as the front end for clients to reach the service on a network. Docker routes requests between the client and available worker nodes for the service, without client knowledge of how many nodes are participating in the service or their IP addresses or ports. (This is the default.) -
endpoint_mode: dnsrr
- DNS round-robin (DNSRR) service discovery does not use a single virtual IP. Docker sets up DNS entries for the service such that a DNS query for the service name returns a list of IP addresses, and the client connects directly to one of these. DNS round-robin is useful in cases where you want to use your own load balancer, or for Hybrid Windows and Linux applications.
version: "3.7"
services:
wordpress:
image: wordpress
ports:
- "8080:80"
networks:
- overlay
deploy:
mode: replicated
replicas: 2
endpoint_mode: vip
mysql:
image: mysql
volumes:
- db-data:/var/lib/mysql/data
networks:
- overlay
deploy:
mode: replicated
replicas: 2
endpoint_mode: dnsrr
volumes:
db-data:
networks:
overlay:
The options for endpoint_mode
also work as flags on the swarm mode CLI command
docker service create. For a
quick list of all swarm related docker
commands, see Swarm mode CLI
commands.
To learn more about service discovery and networking in swarm mode, see Configure service discovery in the swarm mode topics.
labels
Specify labels for the service. These labels are only set on the service, and not on any containers for the service.
version: "3.7"
services:
web:
image: web
deploy:
labels:
com.example.description: "This label will appear on the web service"
To set labels on containers instead, use the labels
key outside of deploy
:
version: "3.7"
services:
web:
image: web
labels:
com.example.description: "This label will appear on all containers for the web service"
mode
Either global
(exactly one container per swarm node) or replicated
(a
specified number of containers). The default is replicated
. (To learn more,
see Replicated and global
services
in the swarm topics.)
version: "3.7"
services:
worker:
image: dockersamples/examplevotingapp_worker
deploy:
mode: global
placement
Specify placement of constraints and preferences. See the docker service create documentation for a full description of the syntax and available types of constraints and preferences.
version: "3.7"
services:
db:
image: postgres
deploy:
placement:
constraints:
- node.role == manager
- engine.labels.operatingsystem == ubuntu 14.04
preferences:
- spread: node.labels.zone
replicas
If the service is replicated
(which is the default), specify the number of
containers that should be running at any given time.
version: "3.7"
services:
worker:
image: dockersamples/examplevotingapp_worker
networks:
- frontend
- backend
deploy:
mode: replicated
replicas: 6
resources
Configures resource constraints.
Note: This replaces the older resource constraint options for non swarm mode in Compose files prior to version 3 (
cpu_shares
,cpu_quota
,cpuset
,mem_limit
,memswap_limit
,mem_swappiness
), as described in Upgrading version 2.x to 3.x.
Each of these is a single value, analogous to its docker service create counterpart.
In this general example, the redis
service is constrained to use no more than
50M of memory and 0.50
(50% of a single core) of available processing time (CPU),
and has 20M
of memory and 0.25
CPU time reserved (as always available to it).
version: "3.7"
services:
redis:
image: redis:alpine
deploy:
resources:
limits:
cpus: '0.50'
memory: 50M
reservations:
cpus: '0.25'
memory: 20M
The topics below describe available options to set resource constraints on services or containers in a swarm.
Looking for options to set resources on non swarm mode containers?
The options described here are specific to the
deploy
key and swarm mode. If you want to set resource constraints on non swarm deployments, use Compose file format version 2 CPU, memory, and other resource options. If you have further questions, refer to the discussion on the GitHub issue docker/compose/4513.
Out Of Memory Exceptions (OOME)
If your services or containers attempt to use more memory than the system has available, you may experience an Out Of Memory Exception (OOME) and a container, or the Docker daemon, might be killed by the kernel OOM killer. To prevent this from happening, ensure that your application runs on hosts with adequate memory and see Understand the risks of running out of memory.
restart_policy
Configures if and how to restart containers when they exit. Replaces
restart
.
condition
: One ofnone
,on-failure
orany
(default:any
).delay
: How long to wait between restart attempts, specified as a duration (default: 0).max_attempts
: How many times to attempt to restart a container before giving up (default: never give up). If the restart does not succeed within the configuredwindow
, this attempt doesn’t count toward the configuredmax_attempts
value. For example, ifmax_attempts
is set to ‘2’, and the restart fails on the first attempt, more than two restarts may be attempted.window
: How long to wait before deciding if a restart has succeeded, specified as a duration (default: decide immediately).
version: "3.7"
services:
redis:
image: redis:alpine
deploy:
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
rollback_config
Version 3.7 file format and up
Configures how the service should be rollbacked in case of a failing update.
parallelism
: The number of containers to rollback at a time. If set to 0, all containers rollback simultaneously.delay
: The time to wait between each container group’s rollback (default 0s).failure_action
: What to do if a rollback fails. One ofcontinue
orpause
(defaultpause
)monitor
: Duration after each task update to monitor for failure(ns|us|ms|s|m|h)
(default 0s).max_failure_ratio
: Failure rate to tolerate during a rollback (default 0).order
: Order of operations during rollbacks. One ofstop-first
(old task is stopped before starting new one), orstart-first
(new task is started first, and the running tasks briefly overlap) (defaultstop-first
).
update_config
Configures how the service should be updated. Useful for configuring rolling updates.
parallelism
: The number of containers to update at a time.delay
: The time to wait between updating a group of containers.failure_action
: What to do if an update fails. One ofcontinue
,rollback
, orpause
(default:pause
).monitor
: Duration after each task update to monitor for failure(ns|us|ms|s|m|h)
(default 0s).max_failure_ratio
: Failure rate to tolerate during an update.order
: Order of operations during updates. One ofstop-first
(old task is stopped before starting new one), orstart-first
(new task is started first, and the running tasks briefly overlap) (defaultstop-first
) Note: Only supported for v3.4 and higher.
Note:
order
is only supported for v3.4 and higher of the compose file format.
version: "3.7"
services:
vote:
image: dockersamples/examplevotingapp_vote:before
depends_on:
- redis
deploy:
replicas: 2
update_config:
parallelism: 2
delay: 10s
order: stop-first
Not supported for docker stack deploy
The following sub-options (supported for docker-compose up
and docker-compose run
) are not supported for docker stack deploy
or the deploy
key.
- build
- cgroup_parent
- container_name
- devices
- tmpfs
- external_links
- links
- network_mode
- restart
- security_opt
- userns_mode
Tip: See the section on how to configure volumes for services, swarms, and docker-stack.yml files. Volumes are supported but to work with swarms and services, they must be configured as named volumes or associated with services that are constrained to nodes with access to the requisite volumes.
devices
List of device mappings. Uses the same format as the --device
docker
client create option.
devices:
- "/dev/ttyUSB0:/dev/ttyUSB0"
Note: This option is ignored when deploying a stack in swarm mode with a (version 3) Compose file.
dns
Custom DNS servers. Can be a single value or a list.
dns: <IP_ADDRESS>
dns:
- <IP_ADDRESS>
- <IP_ADDRESS>
dns_search
Custom DNS search domains. Can be a single value or a list.
dns_search: example.com
dns_search:
- dc1.example.com
- dc2.example.com
entrypoint
Override the default entrypoint.
entrypoint: /code/entrypoint.sh
The entrypoint can also be a list, in a manner similar to dockerfile:
entrypoint:
- php
- -d
- zend_extension=/usr/local/lib/php/extensions/no-debug-non-zts-20100525/xdebug.so
- -d
- memory_limit=-1
- vendor/bin/phpunit
Note: Setting
entrypoint
both overrides any default entrypoint set on the service’s image with theENTRYPOINT
Dockerfile instruction, and clears out any default command on the image - meaning that if there’s aCMD
instruction in the Dockerfile, it is ignored.
env_file
Add environment variables from a file. Can be a single value or a list.
If you have specified a Compose file with docker-compose -f FILE
, paths in
env_file
are relative to the directory that file is in.
Environment variables declared in the environment section override these values – this holds true even if those values are empty or undefined.
env_file: .env
env_file:
- ./common.env
- ./apps/web.env
- /opt/secrets.env
Compose expects each line in an env file to be in VAR=VAL
format. Lines
beginning with #
are treated as comments and are ignored. Blank lines are
also ignored.
# Set Rails/Rack environment
RACK_ENV=development
Note: If your service specifies a build option, variables defined in environment files are not automatically visible during the build. Use the args sub-option of
build
to define build-time environment variables.
The value of VAL
is used as is and not modified at all. For example if the
value is surrounded by quotes (as is often the case of shell variables), the
quotes are included in the value passed to Compose.
Keep in mind that the order of files in the list is significant in determining
the value assigned to a variable that shows up more than once. The files in the
list are processed from the top down. For the same variable specified in file
a.env
and assigned a different value in file b.env
, if b.env
is
listed below (after), then the value from b.env
stands. For example, given the
following declaration in docker-compose.yml
:
services:
some-service:
env_file:
- a.env
- b.env
And the following files:
# a.env
VAR=1
and
# b.env
VAR=hello
$VAR
is hello
.
environment
Add environment variables. You can use either an array or a dictionary. Any boolean values; true, false, yes no, need to be enclosed in quotes to ensure they are not converted to True or False by the YML parser.
Environment variables with only a key are resolved to their values on the machine Compose is running on, which can be helpful for secret or host-specific values.
environment:
RACK_ENV: development
SHOW: 'true'
SESSION_SECRET:
environment:
- RACK_ENV=development
- SHOW=true
- SESSION_SECRET
Note: If your service specifies a build option, variables defined in
environment
are not automatically visible during the build. Use the args sub-option ofbuild
to define build-time environment variables.
expose
Expose ports without publishing them to the host machine - they’ll only be accessible to linked services. Only the internal port can be specified.
expose:
- "3000"
- "8000"
external_links
Link to containers started outside this docker-compose.yml
or even outside of
Compose, especially for containers that provide shared or common services.
external_links
follow semantics similar to the legacy option links
when
specifying both the container name and the link alias (CONTAINER:ALIAS
).
external_links:
- redis_1
- project_db_1:mysql
- project_db_1:postgresql
Notes:
If you’re using the version 2 or above file format, the externally-created containers must be connected to at least one of the same networks as the service that is linking to them. Links are a legacy option. We recommend using networks instead.
This option is ignored when deploying a stack in swarm mode with a (version 3) Compose file.
extra_hosts
Add hostname mappings. Use the same values as the docker client --add-host
parameter.
extra_hosts:
- "somehost:<IP_ADDRESS>"
- "otherhost:<IP_ADDRESS>"
An entry with the ip address and hostname is created in /etc/hosts
inside containers for this service, e.g:
<IP_ADDRESS> somehost
<IP_ADDRESS> otherhost
healthcheck
Version 2.1 file format and up.
Configure a check that’s run to determine whether or not containers for this service are “healthy”. See the docs for the HEALTHCHECK Dockerfile instruction for details on how healthchecks work.
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost"]
interval: 1m30s
timeout: 10s
retries: 3
start_period: 40s
interval
, timeout
and start_period
are specified as durations.
Note:
start_period
is only supported for v3.4 and higher of the compose file format.
test
must be either a string or a list. If it’s a list, the first item must be
either NONE
, CMD
or CMD-SHELL
. If it’s a string, it’s equivalent to
specifying CMD-SHELL
followed by that string.
# Hit the local web app
test: ["CMD", "curl", "-f", "http://localhost"]
As above, but wrapped in /bin/sh
. Both forms below are equivalent.
test: ["CMD-SHELL", "curl -f http://localhost || exit 1"]
test: curl -f https://localhost || exit 1
To disable any default healthcheck set by the image, you can use disable:
true
. This is equivalent to specifying test: ["NONE"]
.
healthcheck:
disable: true
image
Specify the image to start the container from. Can either be a repository/tag or a partial image ID.
image: redis
image: ubuntu:14.04
image: tutum/influxdb
image: example-registry.com:4000/postgresql
image: a4bc65fd
If the image does not exist, Compose attempts to pull it, unless you have also specified build, in which case it builds it using the specified options and tags it with the specified tag.
init
Run an init inside the container that forwards signals and reaps processes.
Set this option to true
to enable this feature for the service.
version: "3.7"
services:
web:
image: alpine:latest
init: true
The default init binary that is used is Tini, and is installed in
/usr/libexec/docker-init
on the daemon host. You can configure the daemon to use a custom init binary through theinit-path
configuration option.
isolation
Specify a container’s isolation technology. On Linux, the only supported value
is default
. On Windows, acceptable values are default
, process
and
hyperv
. Refer to the
Docker Engine docs
for details.
labels
Add metadata to containers using Docker labels. You can use either an array or a dictionary.
It’s recommended that you use reverse-DNS notation to prevent your labels from conflicting with those used by other software.
labels:
com.example.description: "Accounting webapp"
com.example.department: "Finance"
com.example.label-with-empty-value: ""
labels:
- "com.example.description=Accounting webapp"
- "com.example.department=Finance"
- "com.example.label-with-empty-value"
links
Warning: The
--link
flag is a legacy feature of Docker. It may eventually be removed. Unless you absolutely need to continue using it, we recommend that you use user-defined networks to facilitate communication between two containers instead of using--link
. One feature that user-defined networks do not support that you can do with--link
is sharing environmental variables between containers. However, you can use other mechanisms such as volumes to share environment variables between containers in a more controlled way.
Link to containers in another service. Either specify both the service name and
a link alias (SERVICE:ALIAS
), or just the service name.
web:
links:
- db
- db:database
- redis
Containers for the linked service are reachable at a hostname identical to the alias, or the service name if no alias was specified.
Links are not required to enable services to communicate - by default, any service can reach any other service at that service’s name. (See also, the Links topic in Networking in Compose.)
Links also express dependency between services in the same way as depends_on, so they determine the order of service startup.
Notes
If you define both links and networks, services with links between them must share at least one network in common to communicate.
This option is ignored when deploying a stack in swarm mode with a (version 3) Compose file.
logging
Logging configuration for the service.
logging:
driver: syslog
options:
syslog-address: "tcp://<IP_ADDRESS>:123"
The driver
name specifies a logging driver for the service’s
containers, as with the --log-driver
option for docker run
(documented here).
The default value is json-file.
driver: "json-file"
driver: "syslog"
driver: "none"
Note: Only the
json-file
andjournald
drivers make the logs available directly fromdocker-compose up
anddocker-compose logs
. Using any other driver does not print any logs.
Specify logging options for the logging driver with the options
key, as with the --log-opt
option for docker run
.
Logging options are key-value pairs. An example of syslog
options:
driver: "syslog"
options:
syslog-address: "tcp://<IP_ADDRESS>:123"
The default driver json-file, has options to limit the amount of logs stored. To do this, use a key-value pair for maximum storage size and maximum number of files:
options:
max-size: "200k"
max-file: "10"
The example shown above would store log files until they reach a max-size
of
200kB, and then rotate them. The amount of individual log files stored is
specified by the max-file
value. As logs grow beyond the max limits, older log
files are removed to allow storage of new logs.
Here is an example docker-compose.yml
file that limits logging storage:
version: "3.7"
services:
some-service:
image: some-service
logging:
driver: "json-file"
options:
max-size: "200k"
max-file: "10"
Logging options available depend on which logging driver you use
The above example for controlling log files and sizes uses options specific to the json-file driver. These particular options are not available on other logging drivers. For a full list of supported logging drivers and their options, see logging drivers.
network_mode
Network mode. Use the same values as the docker client --network
parameter, plus
the special form service:[service name]
.
network_mode: "bridge"
network_mode: "host"
network_mode: "none"
network_mode: "service:[service name]"
network_mode: "container:[container name/id]"
Notes
This option is ignored when deploying a stack in swarm mode with a (version 3) Compose file.
network_mode: "host"
cannot be mixed with links.
networks
Networks to join, referencing entries under the
top-level networks
key.
services:
some-service:
networks:
- some-network
- other-network
aliases
Aliases (alternative hostnames) for this service on the network. Other containers on the same network can use either the service name or this alias to connect to one of the service’s containers.
Since aliases
is network-scoped, the same service can have different aliases on different networks.
Note: A network-wide alias can be shared by multiple containers, and even by multiple services. If it is, then exactly which container the name resolves to is not guaranteed.
The general format is shown here.
services:
some-service:
networks:
some-network:
aliases:
- alias1
- alias3
other-network:
aliases:
- alias2
In the example below, three services are provided (web
, worker
, and db
),
along with two networks (new
and legacy
). The db
service is reachable at
the hostname db
or database
on the new
network, and at db
or mysql
on
the legacy
network.
version: "3.7"
services:
web:
image: "nginx:alpine"
networks:
- new
worker:
image: "my-worker-image:latest"
networks:
- legacy
db:
image: mysql
networks:
new:
aliases:
- database
legacy:
aliases:
- mysql
networks:
new:
legacy:
ipv4_address, ipv6_address
Specify a static IP address for containers for this service when joining the network.
The corresponding network configuration in the
top-level networks section must have an
ipam
block with subnet configurations covering each static address.
If IPv6 addressing is desired, the
enable_ipv6
option must be set, and you must use a version 2.x Compose file. IPv6 options do not currently work in swarm mode.
An example:
version: "3.7"
services:
app:
image: nginx:alpine
networks:
app_net:
ipv4_address: <IP_ADDRESS>
ipv6_address: 2001:3984:3989::10
networks:
app_net:
ipam:
driver: default
config:
- subnet: "<IP_ADDRESS>/24"
- subnet: "2001:3984:3989::/64"
pid
pid: "host"
Sets the PID mode to the host PID mode. This turns on sharing between container and the host operating system the PID address space. Containers launched with this flag can access and manipulate other containers in the bare-metal machine’s namespace and vice versa.
ports
Expose ports.
Note: Port mapping is incompatible with
network_mode: host
Short syntax
Either specify both ports (HOST:CONTAINER
), or just the container
port (an ephemeral host port is chosen).
Note: When mapping ports in the
HOST:CONTAINER
format, you may experience erroneous results when using a container port lower than 60, because YAML parses numbers in the formatxx:yy
as a base-60 value. For this reason, we recommend always explicitly specifying your port mappings as strings.
ports:
- "3000"
- "3000-3005"
- "8000:8000"
- "9090-9091:8080-8081"
- "49100:22"
- "<IP_ADDRESS>:8001:8001"
- "<IP_ADDRESS>:5000-5010:5000-5010"
- "6060:6060/udp"
Long syntax
The long form syntax allows the configuration of additional fields that can’t be expressed in the short form.
target
: the port inside the containerpublished
: the publicly exposed portprotocol
: the port protocol (tcp
orudp
)mode
:host
for publishing a host port on each node, oringress
for a swarm mode port to be load balanced.
ports:
- target: 80
published: 8080
protocol: tcp
mode: host
Note: The long syntax is new in v3.2
restart
no
is the default restart policy, and it does not restart a container under
any circumstance. When always
is specified, the container always restarts. The
on-failure
policy restarts a container if the exit code indicates an
on-failure error.
restart: "no"
restart: always
restart: on-failure
restart: unless-stopped
Note: This option is ignored when deploying a stack in swarm mode with a (version 3) Compose file. Use restart_policy instead.
secrets
Grant access to secrets on a per-service basis using the per-service secrets
configuration. Two different syntax variants are supported.
Note: The secret must already exist or be defined in the top-level
secrets
configuration of this stack file, or stack deployment fails.
For more information on secrets, see secrets.
Short syntax
The short syntax variant only specifies the secret name. This grants the
container access to the secret and mounts it at /run/secrets/<secret_name>
within the container. The source name and destination mountpoint are both set
to the secret name.
The following example uses the short syntax to grant the redis
service
access to the my_secret
and my_other_secret
secrets. The value of
my_secret
is set to the contents of the file ./my_secret.txt
, and
my_other_secret
is defined as an external resource, which means that it has
already been defined in Docker, either by running the docker secret create
command or by another stack deployment. If the external secret does not exist,
the stack deployment fails with a secret not found
error.
version: "3.7"
services:
redis:
image: redis:latest
deploy:
replicas: 1
secrets:
- my_secret
- my_other_secret
secrets:
my_secret:
file: ./my_secret.txt
my_other_secret:
external: true
Long syntax
The long syntax provides more granularity in how the secret is created within the service’s task containers.
source
: The name of the secret as it exists in Docker.target
: The name of the file to be mounted in/run/secrets/
in the service’s task containers. Defaults tosource
if not specified.uid
andgid
: The numeric UID or GID that owns the file within/run/secrets/
in the service’s task containers. Both default to0
if not specified.mode
: The permissions for the file to be mounted in/run/secrets/
in the service’s task containers, in octal notation. For instance,0444
represents world-readable. The default in Docker 1.13.1 is0000
, but is be0444
in newer versions. Secrets cannot be writable because they are mounted in a temporary filesystem, so if you set the writable bit, it is ignored. The executable bit can be set. If you aren’t familiar with UNIX file permission modes, you may find this permissions calculator useful.
The following example sets name of the my_secret
to redis_secret
within the
container, sets the mode to 0440
(group-readable) and sets the user and group
to 103
. The redis
service does not have access to the my_other_secret
secret.
version: "3.7"
services:
redis:
image: redis:latest
deploy:
replicas: 1
secrets:
- source: my_secret
target: redis_secret
uid: '103'
gid: '103'
mode: 0440
secrets:
my_secret:
file: ./my_secret.txt
my_other_secret:
external: true
You can grant a service access to multiple secrets and you can mix long and short syntax. Defining a secret does not imply granting a service access to it.
security_opt
Override the default labeling scheme for each container.
security_opt:
- label:user:USER
- label:role:ROLE
Note: This option is ignored when deploying a stack in swarm mode with a (version 3) Compose file.
stop_grace_period
Specify how long to wait when attempting to stop a container if it doesn’t
handle SIGTERM (or whatever stop signal has been specified with
stop_signal
), before sending SIGKILL. Specified
as a duration.
stop_grace_period: 1s
stop_grace_period: 1m30s
By default, stop
waits 10 seconds for the container to exit before sending
SIGKILL.
stop_signal
Sets an alternative signal to stop the container. By default stop
uses
SIGTERM. Setting an alternative signal using stop_signal
causes
stop
to send that signal instead.
stop_signal: SIGUSR1
sysctls
Kernel parameters to set in the container. You can use either an array or a dictionary.
sysctls:
net.core.somaxconn: 1024
net.ipv4.tcp_syncookies: 0
sysctls:
- net.core.somaxconn=1024
- net.ipv4.tcp_syncookies=0
You can only use sysctls that are namespaced in the kernel. Docker does not support changing sysctls inside a container that also modify the host system. For an overview of supported sysctls, refer to configure namespaced kernel parameters (sysctls) at runtime.
This option requires Docker Engine 19.03 or up when deploying a stack in swarm mode with a (version 3) Compose file.
tmpfs
Version 2 file format and up.
Mount a temporary file system inside the container. Can be a single value or a list.
tmpfs: /run
tmpfs:
- /run
- /tmp
Note: This option is ignored when deploying a stack in swarm mode with a (version 3-3.5) Compose file.
Version 3.6 file format and up.
Mount a temporary file system inside the container. Size parameter specifies the size of the tmpfs mount in bytes. Unlimited by default.
- type: tmpfs
target: /app
tmpfs:
size: 1000
ulimits
Override the default ulimits for a container. You can either specify a single limit as an integer or soft/hard limits as a mapping.
ulimits:
nproc: 65535
nofile:
soft: 20000
hard: 40000
userns_mode
userns_mode: "host"
Disables the user namespace for this service, if Docker daemon is configured with user namespaces. See dockerd for more information.
Note: This option is ignored when deploying a stack in swarm mode with a (version 3) Compose file.
volumes
Mount host paths or named volumes, specified as sub-options to a service.
You can mount a host path as part of a definition for a single service, and
there is no need to define it in the top level volumes
key.
But, if you want to reuse a volume across multiple services, then define a named
volume in the top-level volumes
key. Use
named volumes with services, swarms, and stack
files.
Note: The top-level volumes key defines a named volume and references it from each service’s
volumes
list. This replacesvolumes_from
in earlier versions of the Compose file format. See Use volumes and Volume Plugins for general information on volumes.
This example shows a named volume (mydata
) being used by the web
service,
and a bind mount defined for a single service (first path under db
service
volumes
). The db
service also uses a named volume called dbdata
(second
path under db
service volumes
), but defines it using the old string format
for mounting a named volume. Named volumes must be listed under the top-level
volumes
key, as shown.
version: "3.7"
services:
web:
image: nginx:alpine
volumes:
- type: volume
source: mydata
target: /data
volume:
nocopy: true
- type: bind
source: ./static
target: /opt/app/static
db:
image: postgres:latest
volumes:
- "/var/run/postgres/postgres.sock:/var/run/postgres/postgres.sock"
- "dbdata:/var/lib/postgresql/data"
volumes:
mydata:
dbdata:
Note: See Use volumes and Volume Plugins for general information on volumes.
Short syntax
Optionally specify a path on the host machine
(HOST:CONTAINER
), or an access mode (HOST:CONTAINER:ro
).
You can mount a relative path on the host, that expands relative to
the directory of the Compose configuration file being used. Relative paths
should always begin with .
or ..
.
volumes:
# Just specify a path and let the Engine create a volume
- /var/lib/mysql
# Specify an absolute path mapping
- /opt/data:/var/lib/mysql
# Path on the host, relative to the Compose file
- ./cache:/tmp/cache
# User-relative path
- ~/configs:/etc/configs/:ro
# Named volume
- datavolume:/var/lib/mysql
Long syntax
The long form syntax allows the configuration of additional fields that can’t be expressed in the short form.
type
: the mount typevolume
,bind
,tmpfs
ornpipe
source
: the source of the mount, a path on the host for a bind mount, or the name of a volume defined in the top-levelvolumes
key. Not applicable for a tmpfs mount.target
: the path in the container where the volume is mountedread_only
: flag to set the volume as read-onlybind
: configure additional bind optionspropagation
: the propagation mode used for the bind
volume
: configure additional volume optionsnocopy
: flag to disable copying of data from a container when a volume is created
tmpfs
: configure additional tmpfs optionssize
: the size for the tmpfs mount in bytes
consistency
: the consistency requirements of the mount, one ofconsistent
(host and container have identical view),cached
(read cache, host view is authoritative) ordelegated
(read-write cache, container’s view is authoritative)
version: "3.7"
services:
web:
image: nginx:alpine
ports:
- "80:80"
volumes:
- type: volume
source: mydata
target: /data
volume:
nocopy: true
- type: bind
source: ./static
target: /opt/app/static
networks:
webnet:
volumes:
mydata:
Note: The long syntax is new in v3.2
Volumes for services, swarms, and stack files
When working with services, swarms, and docker-stack.yml
files, keep in mind
that the tasks (containers) backing a service can be deployed on any node in a
swarm, and this may be a different node each time the service is updated.
In the absence of having named volumes with specified sources, Docker creates an anonymous volume for each task backing a service. Anonymous volumes do not persist after the associated containers are removed.
If you want your data to persist, use a named volume and a volume driver that is multi-host aware, so that the data is accessible from any node. Or, set constraints on the service so that its tasks are deployed on a node that has the volume present.
As an example, the docker-stack.yml
file for the
votingapp sample in Docker
Labs defines a service called db
that runs a postgres
database. It is
configured as a named volume to persist the data on the swarm,
and is constrained to run only on manager
nodes. Here is the relevant snip-it from that file:
version: "3.7"
services:
db:
image: postgres:9.4
volumes:
- db-data:/var/lib/postgresql/data
networks:
- backend
deploy:
placement:
constraints: [node.role == manager]
Caching options for volume mounts (Docker Desktop for Mac)
On Docker 17.04 CE Edge and up, including 17.06 CE Edge and Stable, you can
configure container-and-host consistency requirements for bind-mounted
directories in Compose files to allow for better performance on read/write of
volume mounts. These options address issues specific to osxfs
file sharing,
and therefore are only applicable on Docker Desktop for Mac.
The flags are:
-
consistent
: Full consistency. The container runtime and the host maintain an identical view of the mount at all times. This is the default. -
cached
: The host’s view of the mount is authoritative. There may be delays before updates made on the host are visible within a container. -
delegated
: The container runtime’s view of the mount is authoritative. There may be delays before updates made in a container are visible on the host.
Here is an example of configuring a volume as cached
:
version: "3.7"
services:
php:
image: php:7.1-fpm
ports:
- "9000"
volumes:
- .:/var/www/project:cached
Full detail on these flags, the problems they solve, and their
docker run
counterparts is in the Docker Desktop for Mac topic Performance tuning for
volume mounts (shared filesystems).
domainname, hostname, ipc, mac_address, privileged, read_only, shm_size, stdin_open, tty, user, working_dir
Each of these is a single value, analogous to its
docker run counterpart. Note that mac_address
is a legacy option.
user: postgresql
working_dir: /code
domainname: foo.com
hostname: foo
ipc: host
mac_address: 02:42:ac:11:65:43
privileged: true
read_only: true
shm_size: 64M
stdin_open: true
tty: true
Specifying durations
Some configuration options, such as the interval
and timeout
sub-options for
check
, accept a duration as a string in a
format that looks like this:
2.5s
10s
1m30s
2h32m
5h34m56s
The supported units are us
, ms
, s
, m
and h
.
Specifying byte values
Some configuration options, such as the shm_size
sub-option for
build
, accept a byte value as a string in a format
that looks like this:
2b
1024kb
2048k
300m
1gb
The supported units are b
, k
, m
and g
, and their alternative notation kb
,
mb
and gb
. Decimal values are not supported at this time.
Volume configuration reference
While it is possible to declare volumes on the file as part of the
service declaration, this section allows you to create named volumes (without
relying on volumes_from
) that can be reused across multiple services, and are
easily retrieved and inspected using the docker command line or API. See the
docker volume subcommand
documentation for more information.
See Use volumes and Volume Plugins for general information on volumes.
Here’s an example of a two-service setup where a database’s data directory is shared with another service as a volume so that it can be periodically backed up:
version: "3.7"
services:
db:
image: db
volumes:
- data-volume:/var/lib/db
backup:
image: backup-service
volumes:
- data-volume:/var/lib/backup/data
volumes:
data-volume:
An entry under the top-level volumes
key can be empty, in which case it
uses the default driver configured by the Engine (in most cases, this is the
local
driver). Optionally, you can configure it with the following keys:
driver
Specify which volume driver should be used for this volume. Defaults to whatever
driver the Docker Engine has been configured to use, which in most cases is
local
. If the driver is not available, the Engine returns an error when
docker-compose up
tries to create the volume.
driver: foobar
driver_opts
Specify a list of options as key-value pairs to pass to the driver for this volume. Those options are driver-dependent - consult the driver’s documentation for more information. Optional.
volumes:
example:
driver_opts:
type: "nfs"
o: "addr=<IP_ADDRESS>,nolock,soft,rw"
device: ":/docker/example"
external
If set to true
, specifies that this volume has been created outside of
Compose. docker-compose up
does not attempt to create it, and raises
an error if it doesn’t exist.
For version 3.3 and below of the format, external
cannot be used in
conjunction with other volume configuration keys (driver
, driver_opts
,
labels
). This limitation no longer exists for
version 3.4 and above.
In the example below, instead of attempting to create a volume called
[projectname]_data
, Compose looks for an existing volume simply
called data
and mount it into the db
service’s containers.
version: "3.7"
services:
db:
image: postgres
volumes:
- data:/var/lib/postgresql/data
volumes:
data:
external: true
external.name was deprecated in version 3.4 file format use
name
instead.
You can also specify the name of the volume separately from the name used to refer to it within the Compose file:
volumes:
data:
external:
name: actual-name-of-volume
External volumes are always created with docker stack deploy
External volumes that do not exist are created if you use docker stack deploy to launch the app in swarm mode (instead of docker compose up). In swarm mode, a volume is automatically created when it is defined by a service. As service tasks are scheduled on new nodes, swarmkit creates the volume on the local node. To learn more, see moby/moby#29976.
labels
Add metadata to containers using Docker labels. You can use either an array or a dictionary.
It’s recommended that you use reverse-DNS notation to prevent your labels from conflicting with those used by other software.
labels:
com.example.description: "Database volume"
com.example.department: "IT/Ops"
com.example.label-with-empty-value: ""
labels:
- "com.example.description=Database volume"
- "com.example.department=IT/Ops"
- "com.example.label-with-empty-value"
name
Set a custom name for this volume. The name field can be used to reference volumes that contain special characters. The name is used as is and will not be scoped with the stack name.
version: "3.7"
volumes:
data:
name: my-app-data
It can also be used in conjunction with the external
property:
version: "3.7"
volumes:
data:
external: true
name: my-app-data
Network configuration reference
The top-level networks
key lets you specify networks to be created.
-
For a full explanation of Compose’s use of Docker networking features and all network driver options, see the Networking guide.
-
For Docker Labs tutorials on networking, start with Designing Scalable, Portable Docker Container Networks
driver
Specify which driver should be used for this network.
The default driver depends on how the Docker Engine you’re using is configured,
but in most instances it is bridge
on a single host and overlay
on a
Swarm.
The Docker Engine returns an error if the driver is not available.
driver: overlay
bridge
Docker defaults to using a bridge
network on a single host. For examples of
how to work with bridge networks, see the Docker Labs tutorial on Bridge
networking.
overlay
The overlay
driver creates a named network across multiple nodes in a
swarm.
-
For a working example of how to build and use an
overlay
network with a service in swarm mode, see the Docker Labs tutorial on Overlay networking and service discovery. -
For an in-depth look at how it works under the hood, see the networking concepts lab on the Overlay Driver Network Architecture.
host or none
Use the host’s networking stack, or no networking. Equivalent to
docker run --net=host
or docker run --net=none
. Only used if you use
docker stack
commands. If you use the docker-compose
command,
use network_mode instead.
If you want to use a particular network on a common build, use [network] as mentioned in the second yaml file example.
The syntax for using built-in networks such as host
and none
is a little
different. Define an external network with the name host
or none
(that
Docker has already created automatically) and an alias that Compose can use
(hostnet
or nonet
in the following examples), then grant the service access to that
network using the alias.
version: "3.7"
services:
web:
networks:
hostnet: {}
networks:
hostnet:
external: true
name: host
services:
web:
...
build:
...
network: host
context: .
...
services:
web:
...
networks:
nonet: {}
networks:
nonet:
external: true
name: none
driver_opts
Specify a list of options as key-value pairs to pass to the driver for this network. Those options are driver-dependent - consult the driver’s documentation for more information. Optional.
driver_opts:
foo: "bar"
baz: 1
attachable
Note: Only supported for v3.2 and higher.
Only used when the driver
is set to overlay
. If set to true
, then
standalone containers can attach to this network, in addition to services. If a
standalone container attaches to an overlay network, it can communicate with
services and standalone containers that are also attached to the overlay
network from other Docker daemons.
networks:
mynet1:
driver: overlay
attachable: true
enable_ipv6
Enable IPv6 networking on this network.
Not supported in Compose File version 3
enable_ipv6
requires you to use a version 2 Compose file, as this directive is not yet supported in Swarm mode.
ipam
Specify custom IPAM config. This is an object with several properties, each of which is optional:
driver
: Custom IPAM driver, instead of the default.config
: A list with zero or more config blocks, each containing any of the following keys:subnet
: Subnet in CIDR format that represents a network segment
A full example:
ipam:
driver: default
config:
- subnet: <IP_ADDRESS>/16
Note: Additional IPAM configurations, such as
gateway
, are only honored for version 2 at the moment.
internal
By default, Docker also connects a bridge network to it to provide external
connectivity. If you want to create an externally isolated overlay network,
you can set this option to true
.
labels
Add metadata to containers using Docker labels. You can use either an array or a dictionary.
It’s recommended that you use reverse-DNS notation to prevent your labels from conflicting with those used by other software.
labels:
com.example.description: "Financial transaction network"
com.example.department: "Finance"
com.example.label-with-empty-value: ""
labels:
- "com.example.description=Financial transaction network"
- "com.example.department=Finance"
- "com.example.label-with-empty-value"
external
If set to true
, specifies that this network has been created outside of
Compose. docker-compose up
does not attempt to create it, and raises
an error if it doesn’t exist.
For version 3.3 and below of the format, external
cannot be used in
conjunction with other network configuration keys (driver
, driver_opts
,
ipam
, internal
). This limitation no longer exists for
version 3.4 and above.
In the example below, proxy
is the gateway to the outside world. Instead of
attempting to create a network called [projectname]_outside
, Compose
looks for an existing network simply called outside
and connect the proxy
service’s containers to it.
version: "3.7"
services:
proxy:
build: ./proxy
networks:
- outside
- default
app:
build: ./app
networks:
- default
networks:
outside:
external: true
external.name was deprecated in version 3.5 file format use
name
instead.
You can also specify the name of the network separately from the name used to refer to it within the Compose file:
version: "3.7"
networks:
outside:
external:
name: actual-name-of-network
name
Set a custom name for this network. The name field can be used to reference networks which contain special characters. The name is used as is and will not be scoped with the stack name.
version: "3.7"
networks:
network1:
name: my-app-net
It can also be used in conjunction with the external
property:
version: "3.7"
networks:
network1:
external: true
name: my-app-net
configs configuration reference
The top-level configs
declaration defines or references
configs that can be granted to the services in this
stack. The source of the config is either file
or external
.
file
: The config is created with the contents of the file at the specified path.external
: If set to true, specifies that this config has already been created. Docker does not attempt to create it, and if it does not exist, aconfig not found
error occurs.name
: The name of the config object in Docker. This field can be used to reference configs that contain special characters. The name is used as is and will not be scoped with the stack name. Introduced in version 3.5 file format.
In this example, my_first_config
is created (as
<stack_name>_my_first_config)
when the stack is deployed,
and my_second_config
already exists in Docker.
configs:
my_first_config:
file: ./config_data
my_second_config:
external: true
Another variant for external configs is when the name of the config in Docker
is different from the name that exists within the service. The following
example modifies the previous one to use the external config called
redis_config
.
configs:
my_first_config:
file: ./config_data
my_second_config:
external:
name: redis_config
You still need to grant access to the config to each service in the stack.
secrets configuration reference
The top-level secrets
declaration defines or references
secrets that can be granted to the services in this
stack. The source of the secret is either file
or external
.
file
: The secret is created with the contents of the file at the specified path.external
: If set to true, specifies that this secret has already been created. Docker does not attempt to create it, and if it does not exist, asecret not found
error occurs.name
: The name of the secret object in Docker. This field can be used to reference secrets that contain special characters. The name is used as is and will not be scoped with the stack name. Introduced in version 3.5 file format.
In this example, my_first_secret
is created as
<stack_name>_my_first_secret
when the stack is deployed,
and my_second_secret
already exists in Docker.
secrets:
my_first_secret:
file: ./secret_data
my_second_secret:
external: true
Another variant for external secrets is when the name of the secret in Docker
is different from the name that exists within the service. The following
example modifies the previous one to use the external secret called
redis_secret
.
Compose File v3.5 and above
secrets:
my_first_secret:
file: ./secret_data
my_second_secret:
external: true
name: redis_secret
Compose File v3.4 and under
my_second_secret:
external:
name: redis_secret
You still need to grant access to the secrets to each service in the stack.
Variable substitution
Your configuration options can contain environment variables. Compose uses the
variable values from the shell environment in which docker-compose
is run. For
example, suppose the shell contains POSTGRES_VERSION=9.3
and you supply this
configuration:
db:
image: "postgres:${POSTGRES_VERSION}"
When you run docker-compose up
with this configuration, Compose looks for the
POSTGRES_VERSION
environment variable in the shell and substitutes its value
in. For this example, Compose resolves the image
to postgres:9.3
before
running the configuration.
If an environment variable is not set, Compose substitutes with an empty
string. In the example above, if POSTGRES_VERSION
is not set, the value for
the image
option is postgres:
.
You can set default values for environment variables using a
.env
file, which Compose automatically looks for. Values
set in the shell environment override those set in the .env
file.
Important: The
.env file
feature only works when you use thedocker-compose up
command and does not work withdocker stack deploy
.
Both $VARIABLE
and ${VARIABLE}
syntax are supported. Additionally when using
the 2.1 file format, it is possible to
provide inline default values using typical shell syntax:
${VARIABLE:-default}
evaluates todefault
ifVARIABLE
is unset or empty in the environment.${VARIABLE-default}
evaluates todefault
only ifVARIABLE
is unset in the environment.
Similarly, the following syntax allows you to specify mandatory variables:
${VARIABLE:?err}
exits with an error message containingerr
ifVARIABLE
is unset or empty in the environment.${VARIABLE?err}
exits with an error message containingerr
ifVARIABLE
is unset in the environment.
Other extended shell-style features, such as ${VARIABLE/foo/bar}
, are not
supported.
You can use a $$
(double-dollar sign) when your configuration needs a literal
dollar sign. This also prevents Compose from interpolating a value, so a $$
allows you to refer to environment variables that you don’t want processed by
Compose.
web:
build: .
command: "$$VAR_NOT_INTERPOLATED_BY_COMPOSE"
If you forget and use a single dollar sign ($
), Compose interprets the value
as an environment variable and warns you:
The VAR_NOT_INTERPOLATED_BY_COMPOSE is not set. Substituting an empty string.
Extension fields
It is possible to re-use configuration fragments using extension fields. Those
special fields can be of any format as long as they are located at the root of
your Compose file and their name start with the x-
character sequence.
Note
Starting with the 3.7 format (for the 3.x series) and 2.4 format (for the 2.x series), extension fields are also allowed at the root of service, volume, network, config and secret definitions.
version: '3.4'
x-custom:
items:
- a
- b
options:
max-size: '12m'
name: "custom"
The contents of those fields are ignored by Compose, but they can be inserted in your resource definitions using YAML anchors. For example, if you want several of your services to use the same logging configuration:
logging:
options:
max-size: '12m'
max-file: '5'
driver: json-file
You may write your Compose file as follows:
version: '3.4'
x-logging:
&default-logging
options:
max-size: '12m'
max-file: '5'
driver: json-file
services:
web:
image: myapp/web:latest
logging: *default-logging
db:
image: mysql:latest
logging: *default-logging
It is also possible to partially override values in extension fields using the YAML merge type. For example:
version: '3.4'
x-volumes:
&default-volume
driver: foobar-storage
services:
web:
image: myapp/web:latest
volumes: ["vol1", "vol2", "vol3"]
volumes:
vol1: *default-volume
vol2:
<< : *default-volume
name: volume02
vol3:
<< : *default-volume
driver: default
name: volume-local


--- Title: None of these apply (disabled bus pass) Breadcrumb1_url: section-parking.html Breadcrumb1_title: Parking & Travel Breadcrumb2_url: bus-passes-diamond-travelcard.html Breadcrumb2_title: Free bus pass (Diamond Travelcard) ---
You are not automatically eligible for a disabled person’s bus pass. You may still be eligible if you meet one of these criteria.


Join Windows worker nodes to a swarm
Estimated reading time: 7 minutesThese are the docs for UCP version 3.0.16
To select a different version, use the selector below.
UCP supports worker nodes that run on Windows Server 2016. Only worker nodes are supported on Windows, and all manager nodes in the swarm must run on Linux.
Follow these steps to enable a worker node on Windows.
- Install UCP on a Linux distribution.
- Install Docker Enterprise Edition (Docker EE) on Windows Server 2016.
- Configure the Windows node.
- Join the Windows node to the swarm.
Install UCP
Install UCP on a Linux distribution. Learn how to install UCP on production. UCP requires Docker EE version 17.06 or later.
Install Docker EE on Windows Server 2016
Install Docker EE on a Windows Server 2016 instance to enable joining a swarm that’s managed by UCP.
Configure the Windows node
Follow these steps to configure the docker daemon and the Windows environment.
- Pull the Windows-specific image of
ucp-agent
, which is nameducp-agent-win
. - Run the Windows worker setup script provided with
ucp-agent-win
. - Join the swarm with the token provided by the UCP web UI.
- Create a file
daemon.json
. - Paste following content and save it under
$env:ProgramData\Docker\config\
{ "labels": ["os=windows"] }
- Restart the docker service.
Restart-Service docker
Pull the Windows-specific images
On a manager node, run the following command to list the images that are required on Windows nodes.
docker container run --rm docker/ucp:3.0.16 images --list --enable-windows \
docker/ucp-agent-win:3.0.16 \
docker/ucp-dsinfo-win:3.0.16
On Windows Server 2016, in a PowerShell terminal running as Administrator,
log in to Docker Hub with the docker login
command and pull the listed images.
docker image pull docker/ucp-agent-win:3.0.16
docker image pull docker/ucp-dsinfo-win:3.0.16
Run the Windows node setup script
You need to open ports 2376 and 12376, and create certificates for the Docker daemon to communicate securely. Run this command:
$script = [ScriptBlock]::Create((docker run --rm docker/ucp-agent-win:3.0.16 windows-script | Out-String))
Invoke-Command $script
Docker daemon restart
When you run
windows-script
, the Docker service is unavailable temporarily.
The Windows node is ready to join the swarm. Run the setup script on each instance of Windows Server that will be a worker node.
Compatibility with daemon.json
The script may be incompatible with installations that use a config file at
C:\ProgramData\docker\config\daemon.json
. If you use such a file, make sure
that the daemon runs on port 2376 and that it uses certificates located in
C:\ProgramData\docker\daemoncerts
. If certificates don’t exist in this
directory, run ucp-agent-win generate-certs
, as shown in Step 2 of the
Set up certs for the dockerd service
procedure.
In the daemon.json file, set the tlscacert
, tlscert
, and tlskey
options
to the corresponding files in C:\ProgramData\docker\daemoncerts
:
{
...
"debug": true,
"tls": true,
"tlscacert": "C:\ProgramData\docker\daemoncerts\ca.pem",
"tlscert": "C:\ProgramData\docker\daemoncerts\cert.pem",
"tlskey": "C:\ProgramData\docker\daemoncerts\key.pem",
"tlsverify": true,
...
}
Join the Windows node to the swarm
Now you can join the UCP cluster by using the docker swarm join
command that’s
provided by the UCP web UI. Learn to add nodes to your swarm.
The command looks similar to the following.
docker swarm join --token <token> <ucp-manager-ip>
Run the docker swarm join
command on each instance of Windows Server that
will be a worker node.
Configure a Windows worker node manually
The following sections describe how to run the commands in the setup script
manually to configure the dockerd
service and the Windows environment.
The script opens ports in the firewall and sets up certificates for dockerd
.
To see the script, you can run the windows-script
command without piping
to the Invoke-Expression
cmdlet.
docker container run --rm docker/ucp-agent-win:3.0.16 windows-script
Open ports in the Windows firewall
UCP and Docker EE require that ports 2376 and 12376 are open for inbound TCP traffic.
In a PowerShell terminal running as Administrator, run these commands to add rules to the Windows firewall.
netsh advfirewall firewall add rule name="docker_local" dir=in action=allow protocol=TCP localport=2376
netsh advfirewall firewall add rule name="docker_proxy" dir=in action=allow protocol=TCP localport=12376
Set up certs for the dockerd service
- Create the directory
C:\ProgramData\docker\daemoncerts
. -
In a PowerShell terminal running as Administrator, run the following command to generate certificates.
docker container run --rm -v C:\ProgramData\docker\daemoncerts:C:\certs docker/ucp-agent-win:3.0.16 generate-certs
-
To set up certificates, run the following commands to stop and unregister the
dockerd
service, register the service with the certificates, and restart the service.Stop-Service docker dockerd --unregister-service dockerd -H npipe:// -H <IP_ADDRESS>:2376 --tlsverify --tlscacert=C:\ProgramData\docker\daemoncerts\ca.pem --tlscert=C:\ProgramData\docker\daemoncerts\cert.pem --tlskey=C:\ProgramData\docker\daemoncerts\key.pem --register-service Start-Service docker
The dockerd
service and the Windows environment are now configured to join a UCP swarm.
Tip: If the TLS certificates aren’t set up correctly, the UCP web UI shows the following warning.
Node WIN-NOOQV2PJGTE is a Windows node that cannot connect to its local Docker daemon.
Uninstall UCP from Windows Server
The following steps return the Docker Engine to its original configuration:
-
Unregister the docker service and register it again without the TLS certificates:
Stop-Service docker dockerd --unregister-service dockerd -H npipe:// --register-service Start-Service docker
-
Remove the
certs
directory for the docker service:Remove-Item -Recurse C:\ProgramData\docker\daemoncerts
-
Remove the firewall rules:
netsh advfirewall firewall delete rule name="docker_2376_in" netsh advfirewall firewall delete rule name="docker_12376_in" netsh advfirewall firewall delete rule name="docker_2377_in" netsh advfirewall firewall delete rule name="docker_4789_in" netsh advfirewall firewall delete rule name="docker_4789_out" netsh advfirewall firewall delete rule name="docker_7946_in" netsh advfirewall firewall delete rule name="docker_7946_out"
Windows nodes limitations
Some features are not yet supported on Windows nodes:
- Networking
- The swarm mode routing mesh can’t be used on Windows nodes. You can expose a port for your service in the host where it is running, and use the HTTP routing mesh to make your service accessible using a domain name.
- Encrypted networks are not supported. If you’ve upgraded from a previous
version, you also need to recreate the
ucp-hrm
network to make it unencrypted.
- Secrets
- When using secrets with Windows services, Windows stores temporary secret files on disk. You can use BitLocker on the volume containing the Docker root directory to encrypt the secret data at rest.
- When creating a service which uses Windows containers, the options to specify UID, GID, and mode are not supported for secrets. Secrets are currently only accessible by administrators and users with system access within the container.
- Mounts
- On Windows, Docker can’t listen on a Unix socket. Use TCP or a named pipe instead.


Chapter Eight
Other Related Families
Wood Family
Hannah Wood married Kent Mitchell
about 1765. (See page 170 and 171). Hannah’s father was Isaac Wood and he
married Elizabeth Wilborn. Isaac’s parents were
Joshua Woods who married Mary Unknown abt.
1695 in Baltimore, MD164,165,167. Joshua’s parents were William
Wood and Mary Wicks165. Hannah Wood was Martha Hall
Moose’s 2nd Great Grandmother.
Wilborn /Wellbourne/ Parkine166,167,168
According
to several family trees on One World Tess,
Ancestry.com, Isaac Wood, b. 18 Jan 1700 in Harford, MD, d. 23 Jun 1777
in Havre de Grace, Harford, MD, married
Elizabeth Wilborn, b. abt
1717 in Long Island, NY, d. 1783 in Harford, MD. The marriage date is said to
be 1721 but this cannot be correct if
The
parents of John Milton Hall were Thomas Hall who married Margaret
Hamilton48, abt 1823 in PA and d. on
17 Jun 1866 in
One
un-named tree on Ancestry.com had the following Information:
Minerva Cox, b.abt
1831, probably in
Other Children of
1. Rebecca
COX b: ABT. 1818
2. Josephus
COX I b: ABT. 1820 in VA
3. John
B. COX b: 29 MAY 1823 in
4. Ruth
COX b: ABT. 1827
5. Thomas
COX b: ABT. 1829
6. Minerva
COX b: ABT. 1831
7. Samuel
Gibson COX b: 20 JAN 1832
8. Stephen
COX b: ABT. 1836
9. Baldwin
COX b: ABT. 1838
William and Minerva apparently are the ones living
next to Thomas and Margaret Hamilton Hall in 1850 in
Hoffman
Elizabeth Hoffman,b 28 Dec 1743 in Germany and
died in Northampton, PA, married Johann Jacob Mussgnug,
b. abt 1740 in Solligen,
Germany, d. 1 Oct 1823 in Gettysburg, PA in Northampton, PA on 25 Feb 1766.
Johann Jacob II was reported to be the father of John Moose, Sr1,169,170.
Martin’s
father was Deitrich Hoffman1.
Bowen/Moone/Wilsford/Boeshaar/Carter/Hymphrey
Margaret Bowen is listed as the wife of George
Stephen Ullom in over 30 Family Trees on Ancestry.com. Margaret was said to
be the daughter of Henry Bowen and Anna Moone and
thus tied to a rich history covering the family names above and many more and
going back to
Click here to
return to the Table of Contents


Get these beautiful pink tulips to make your office pop out in that zoom meeting!
Give that special person in your life a bouquet from our beautiful selection of roses.
Straight from T's Hardware Store, we're having a sale on some of the finest trowels around!


BREAKING: Buhari returns from private visit to UK
President Muhammadu Buhari has returned from a private visit to the United Kingdom.
The President returned a day earlier than he was scheduled to.
The Presidential Jet touched down at about 9.45pm Friday night.
Speaking to reporters at the Presidential Wing of the Nnamdi Azikiwe International airport Abuja, President Buhari says he intends to work harder and be accountable to Nigerians.
Details shortly….


ENDS TOMORROW: FREE SHIPPING ON ALL ORDERS
WITH CODE FREESHIP*
this week’s discovery:
Hugo Guinness for J.Crew
Introducing the multitalented Mr. Guinness: illustrator, writer, actor (fun fact: He voiced a role in the Wes Anderson film Fantastic Mr. Fox). But British-born Hugo Guinness is probably best known for the quirky and collectible black-and-white prints he creates in his Brooklyn studio. So after he appeared in the pages of our catalog a few years back (yes, he’s also a sometime model), we couldn’t resist asking him to bring his witty artwork to the walls of our NYC stores. This spring, Guinness adds sweater designer to his repertoire with a new, exclusive-to-us collaboration. The artist picked a few favorites from his archives, including this English bull terrier with a “magnificent, strange-looking” appearance. His prints were then intarsia-knit into our Italian cashmere and hand embroidered for a textured finish. “I really like seeing my work used for different things in different ways,” Guinness says. How fantastic is that?
featuring the Hugo Guinness for J.Crew cashmere terrier sweater
SHOP ALL SWEATERS >
*Free shipping valid on purchases made online at jcrew.com.com through March 3, 2013, 11:59pm ET only. Offer cannot be applied to previous purchases and cannot be redeemed for cash or used in combination with any other offer. To redeem, enter the code FREESHIP in the promo code box at checkout. Orders will be shipped via regular shipping (3-6 business days) and return requests must be made in accordance with our return policy. Offer valid in the U.S. and Canada only. Terms of offer are subject to change.
Please add<EMAIL_ADDRESS>to your address book to ensure our emails reach your inbox. If you received this email from a friend and would like to subscribe to our email list, please go to the following web address -<EMAIL_ADDRESS>To unsubscribe from the email list, please go to the following web address -<EMAIL_ADDRESS>J.Crew Customer Relations | One Ivy Crescent | Lynchburg, VA 24513-1001


Tic-Tac-Toe
Click the button below to start playing!
Player 1
Player 2
Tic-Tac-Toe
1
2
3
4
5
6
7
8
9
Score:


customizeplayers.Rmd
library(dfsOptimizer)
The package includes a lot of methods for customizing individual players in your model (see the “Customizing your Optimizer” vignette for examples)
## Example of how to block players by ID
# Single Player
opt <- block_players_by_id(opt, player_ids = single_player_id)
# Blocking multiple players uses the same syntax
opt <- block_players_by_id(opt, player_ids = multiple_player_ids)
Let’s say you want to do something complicated, like * Remove every player who either is in the bottom 50% of projected value (fpts/salary) OR plays for Montreal * Lock the two wingers with the highest points * Set the maximum exposure of all players to be proportional to their projected value, within teams
You could use the various sets of functions included in this package, But for complex, multi-step customization, the flow I recommend is: 1. Add players to an optimizer object 2. Pull those players out using get_player_data()
3. Use whatever R code you like on the data.table of players 4. Put the table back into the model with add_players_from_df
Here’s how we’d handle the above example:
# Instantiate
mod <- create_optimizer(site = 'DRAFTKINGS', sport = 'HOCKEY', contest_type = 'CLASSIC')
# Add players (using the data set nhl_players, which is included in the package)
mod <- add_players_from_df(mod, nhl_players)
mod
#> Optimizer Object (S4 class)
#> Site: DRAFTKINGS
#> Sport: HOCKEY
#> Contest Type: CLASSIC
#> Number of Players: 308
One can see we have 308 players in our set – Let’s pull the out into a data.table
# Pull those right back out
# This step is necessary if you added players to your model via CSV,
# otherwise you could just manipulate the original data.frame before
# adding it to the model
players <- get_player_data(mod)
head(players)
#> id first_name last_name fullname team position depth salary
#> 1: 14086443 Nathan MacKinnon Nathan MacKinnon COL C 1 8300
#> 2: 14087003 Frederik Andersen Frederik Andersen TOR G 1 8200
#> 3: 14087005 Michael Hutchinson Michael Hutchinson TOR G 1 8000
#> 4: 14087004 Carey Price Carey Price MON G 1 8000
#> 5: 14086445 Auston Matthews Auston Matthews TOR C 1 7900
#> 6: 14087007 Charlie Lindgren Charlie Lindgren MON G 1 7800
#> fpts locked blocked is_injured min_exposure max_exposure variance
#> 1: 19.00 FALSE FALSE FALSE NA NA NA
#> 2: 15.78 FALSE FALSE FALSE NA NA NA
#> 3: 11.26 FALSE FALSE FALSE NA NA NA
#> 4: 12.18 FALSE FALSE FALSE NA NA NA
#> 5: 15.85 FALSE FALSE FALSE NA NA NA
#> 6: 16.10 FALSE FALSE FALSE NA NA NA
Now that we have the players, we can manipulate the table however we want – I’m going to use data.table in this example, but you could just as easily use whatever methods or packages you want.
Remember what we want to do: * Remove every player who either is in the bottom 50% of projected value (fpts/salary) OR plays for Montreal * Lock the two wingers with the highest points * Set the maximum exposure of all players to be proportional to their projected value, within teams
# Remove every player who _either_ is in the bottom 50% of projected value (fpts/salary) OR plays for Montreal
players <-
players[fpts/salary >= median(fpts/salary)][team != 'MON']
# Lock the two wingers with the highest points
players[ id %in% players[position == 'W', ][order(-fpts)][1:2, id], locked := TRUE]
# Set the maximum exposure of all players to be proportional to their projected value, *within teams*
players[, max_exposure := rank(fpts/salary, ties.method = 'first') / .N, by = team]
From here, you just put the data right back in using add_players_from_df
mod <- add_players_from_df(mod, players)
mod
#> Optimizer Object (S4 class)
#> Site: DRAFTKINGS
#> Sport: HOCKEY
#> Contest Type: CLASSIC
#> Number of Players: 132
You could, of course, use this method in conjunction with external data as well. For instance, if you wanted to remove anyone who played the night before, or who hasn’t player more than N games, you could join in an external set with values like “Days since played” or “N Games Played” and use that to filter the data down before setting it back into the model!


Apache Flink ML 2.0.0 Release Announcement
January 7, 2022 - Dong Lin Yun GaoThe Apache Flink community is excited to announce the release of Flink ML 2.0.0! Flink ML is a library that provides APIs and infrastructure for building stream-batch unified machine learning algorithms, that can be easy-to-use and performant with (near-) real-time latency.
This release involves a major refactor of the earlier Flink ML library and introduces major features that extend the Flink ML API and the iteration runtime, such as supporting stages with multi-input multi-output, graph-based stage composition, and a new stream-batch unified iteration library. Moreover, we added five algorithm implementations in this release, which is the start of a long-term initiative to provide a large number of off-the-shelf algorithms in Flink ML with state-of-the-art performance.
We believe this release is an important step towards extending Apache Flink to a wide range of machine learning use cases, especially the real-time machine learning scenarios.
We encourage you to download the release and share your feedback with the community through the Flink mailing lists or JIRA! We hope you like the new release and we’d be eager to learn about your experience with it.
Notable Features #
API and Infrastructure #
Supporting stages requiring multi-input multi-output #
Stages in a machine learning workflow might take multiple inputs and return multiple outputs. For example, a graph embedding algorithm might need to read two tables, which represent the edge and node of the graph respectively. And a workflow might need a stage that splits the input dataset into two output datasets, for training and testing respectively.
With this capability, algorithm developers can assemble a machine learning workflow as a directed acyclic graph (DAG) of pre-defined stages. And this workflow can be configured and deployed without users knowing the implementation details of this graph. This improvement could considerably expand the applicability and usability of Flink ML.
Supporting online learning with APIs exposing model data #
In a native online learning scenario, we have a long-running job that keeps processing training data and updating a machine learning model. And we could have multiple jobs deployed in web servers which do online inference. It is necessary to transmit the latest model data from the training job to those inference jobs in (near-) real-time latency.
The traditional Estimator/Transformer paradigm does not provide APIs to expose this model data in a streaming manner. Users have to repeatedly call fit() to update model data. Although users might be able to update model data once every few minutes, it is likely very inefficient, if not impossible, to update model data once every few seconds with this approach.
With FLIP-173, model data can be exposed as an unbounded stream via the getModelData() API. Then algorithm users can transfer the model data to web servers in real-time and use the up-to-date model data to do online inference. This feature could significantly strengthen Flink ML’s capability to support online learning applications.
Improved usability for managing parameters #
We care a lot about usability and developer velocity in Flink ML. In this release, we refactored and significantly simplified the experience of defining, getting and setting parameters for algorithms.
With FLIP-174, parameters can be defined as static variables of an interface, and any algorithm that implements the interface could inherit these variable definitions without additional work. Commonly used parameter validators are provided as part of the infrastructure.
Tools for composing DAG of stages into a new stage #
One of the most useful tool in the existing ML libraries (e.g. Scikit-learn, Flink, Spark) is Pipeline, which allows users to compose an estimator from an ordered list of estimators and transformers, without having to explicitly implement the fit/transform for the estimator/transformer.
FLIP-175 extended this capability from pipeline to DAG. Users can now compose an estimator from a DAG of estimator and transformers. This capability of composition allows developers to slice a complex workflow into simpler modules and re-use the modules across multiple workflows. We believe this capability could significantly improve the experience of building and deploying complex workflows using Flink ML.
Stream-batch Unified Iteration Library #
To support training machine learning algorithms and adjust the model parameters dynamically based on the prediction result, it is necessary to have native support for processing data iteratively. It is known that Flink uses DAG to describe the process logic, thus we need to provide the iteration library on top of Flink separately. Besides, since we need to support both offline training and online training / adjustment, the iteration library should support both streaming and batch cases.
FLIP-176 implements a stream-batch unified iteration library. It provides the function of transmitting records back to the precedent operators and the ability to track the progress of rounds inside the iteration. Users could directly use DataStream API and Table API to express the execution logic inside the iteration. Besides, the new iteration library also extends Flink’s checkpointing mechanism to also support exactly-once failover for jobs using iterations.
Python SDK #
Nowadays many machine learning practitioners are used to developing machine learning workflows in Python due to its ease-of-use and excellent ecosystem. To meet the needs of these users, a Python package dedicated for Flink ML is created starting from this release. The Python package currently provides APIs similar to their Java counterparts for developing machine learning algorithms.
Users can install Flink ML Python package through pip using the following command:
pip install apache-flink-ml
In the future we will enhance the Python SDK to enable its interoperability with Flink ML’s Java library, for example, allowing users to express machine learning workflows in Python, where workflows consist of a mixture of stages from the Flink ML Java library as well as stages implemented in Python (e.g. a TensorFlow program).
Algorithm Library #
Now that the Flink ML API re-design is done, we started the initiative to add off-the-shelf algorithms in Flink ML. The release of Flink-ML 2.0.0 is closely related to project Alink - an Apache Flink ecosystem project open sourced by Alibaba. The connection between the Flink community and developers of the Alink project dates back to 2017. The project Alink developers have a significant contribution in designing the new Flink ML APIs, refactoring, optimizing and migrating algorithms from Alink to Flink. Our long-term goal is to provide a library of performant algorithms that are easy to use, debug and customize for your needs.
We have implemented five algorithms in this release, i.e. logistic regression, k-means, k-nearest neighbors, naive bayes and one-hot encoder. For now these algorithms focus on validating the APIs and iteration runtime. In addition to adding more and more algorithms, we will also stress test and optimize their performance to make sure these algorithms have state-of-the-art performance. Stay tuned!
Related Work #
Flink ML project moved to a separate repository #
To accelerate the development of Flink ML, the effort has moved to the new repository flink-ml under the Flink project. We here follow a similar approach like the Stateful Functions effort, where a separate repository has helped to speed up the development by allowing for more light-weight contribution workflows and separate release cycles.
Github organization created for Flink ecosystem projects #
To facilitate the community collaboration on ecosystem projects that extend the capability of the Apache Flink, Apache Flink PMC has granted the permission to use flink-extended as the name of this GitHub organization, which provides a neutral place to host the code of ecosystem projects.
Two Flink ML related projects have been moved to this organization. dl-on-flink provides the capability to implement Flink ML stages using TensorFlow. And clink is a library that facilitates the implementation of Flink ML stages using C++ in order to support e.g. real-time feature engineering.
We hope you can join this effort and share your Flink ecosystem projects in this Github organization. And stay tuned for more updates on ecosystem projects.
Upgrade Notes #
Please review this note for a list of adjustments to make and issues to check if you plan to upgrade to Flink ML 2.0.0.
This note discusses any critical information about incompatibilities and breaking changes, performance changes, and any other changes that might impact your production deployment of Flink ML.
-
Module names are changed.
We have replaced the
flink-ml-api
module with theflink-ml-core_2.12
module.For users who have a dependency on
flink-ml-api
, please replace it withflink-ml-core_2.12
-
PipelineStage and its subclasses are changed.
FLIP-173 made major changes to PipelineStage and its subclasses. Changes include class rename, method signature change, method removal etc.
Users who use PipelineStage and its subclasses should use the new APIs introduced in FLIP-173.
-
Param-related classes are changed.
FLIP-174 made major changes to the param-related classes. Changes include class rename, method signature change, method removal etc.
Users who use classes such as Params and WithParams should use the new APIs introduced in FLIP-174.
-
Flink dependency is changed from 1.12 to 1.14.
This change introduces all the breaking changes listed in the Flink 1.14 release notes. One major change is that the DataSet API is not supported anymore.
Users who use DataSet::iterate should switch to using the datastream-based iteration API introduced in FLIP-176.
Release Notes and Resources #
Please take a look at the release notes for a detailed list of changes and new features.
The binary distribution and source artifacts are now available on the updated Downloads page of the Flink website, and the most recent distribution of Flink ML Python package is available on PyPI.
List of Contributors #
The Apache Flink community would like to thank each one of the contributors that have made this release possible:
Yun Gao, Dong Lin, Zhipeng Zhang, huangxingbo, Yunfeng Zhou, Jiangjie (Becket) Qin, weibo, abdelrahman-ik.


This is an instruction paragraph created by a CSS class. It should show up in a smaller than normal italic red font.
The second sentence is marked using a span tag. This sentence has a line through it. This third sentence is not included in the spanned text.
| View default htm | E-mail<EMAIL_ADDRESS>|


Top 5 Songs For John Danielsson
Rank
Artist
Title
Album
1
Pink Floyd
TBS9
2
Kazouma Jinnouchi
Never Forget - Midnight Version
3
Pink Floyd
TBS14
4
Johnny Cash
The Man Comes Around
5
Hans Zimmer
Time


mod-card
These are cards
Properties
| Property | Attribute | Description | Type | Default |
|---|---|---|---|---|
elevation |
elevation |
Elevation value, controls card shadow in gallery | 1 \| 2 \| 3 \| 4 \| 5 \| 6 |
undefined |
image |
-- | Optional prop that accepts images and attempts to render them | { src: string; alt: string; } \| { src: string; } |
undefined |
rounded |
rounded |
Optional variable to control corners of card | boolean |
false |
size |
size |
Optional variable to control the size of the card | "large" \| "medium" |
"large" |
theme |
theme |
Theme used to control the color elements of the card | "abyss" \| "autumn" \| "breeze" \| "minimalist" |
"minimalist" |
Dependencies
Used by
Graph
graph TD;
mod-panel --> mod-card
style mod-card fill:#f9f,stroke:#333,stroke-width:4px


speechmatics.client¶
Wrapper library to interface with Real-time ASR v2 API. Based on http://asyncio.readthedocs.io/en/latest/producer_consumer.html
-
class
speechmatics.client.
WebsocketClient
(connection_settings)[source]¶ Manage a transcription session with the server.
The best way to interact with this library is to instantiate this client and then add a set of handlers to it. Handlers respond to particular types of messages received from the server.
- Parameters
connection_settings (speechmatics.models.ConnectionSettings) – Settings for the WebSocket connection, including the URL of the server.
-
add_event_handler
(event_name, event_handler)[source]¶ Add an event handler (callback function) to handle an incoming message from the server. Event handlers are passed a copy of the incoming message from the server. If event_name is set to ‘all’ then the handler will be added for every event.
For example, a simple handler that just prints out the
speechmatics.models.ServerMessageType.AddTranscript
messages received:>>> client = WebsocketClient( ConnectionSettings(url="wss://localhost:9000")) >>> handler = lambda msg: print(msg) >>> client.add_event_handler(ServerMessageType.AddTranscript, handler)
- Parameters
event_name (str) – The name of the message for which a handler is being added. Refer to
speechmatics.models.ServerMessageType
for a list of the possible message types.event_handler (Callable[[dict], None]) – A function to be called when a message of the given type is received.
- Raises
ValueError – If the given event name is not valid.
-
add_middleware
(event_name, middleware)[source]¶ Add a middleware to handle outgoing messages sent to the server. Middlewares are passed a reference to the outgoing message, which they may alter. If event_name is set to ‘all’ then the handler will be added for every event.
- Parameters
event_name (str) – The name of the message for which a middleware is being added. Refer to the V2 API docs for a list of the possible message types.
middleware (Callable[[dict, bool], None]) – A function to be called to process an outgoing message of the given type. The function receives the message as the first argument and a second, boolean argument indicating whether or not the message is binary data (which implies it is an AddAudio message).
- Raises
ValueError – If the given event name is not valid.
-
async
run
(stream, transcription_config, audio_settings)[source]¶ Begin a new recognition session. This will run asynchronously. Most callers may prefer to use
run_synchronously()
which will block until the session is finished.- Parameters
stream (io.IOBase) – File-like object which an audio stream can be read from.
transcription_config (speechmatics.models.TranscriptionConfig) – Configuration for the transcription.
audio_settings (speechmatics.models.AudioSettings) – Configuration for the audio stream.
- Raises
Exception – Can raise any exception returned by the consumer/producer tasks.
-
stop
()[source]¶ Indicates that the recognition session should be forcefully stopped. Only used in conjunction with run. You probably don’t need to call this if you’re running the client via
run_synchronously()
.
-
update_transcription_config
(new_transcription_config)[source]¶ Updates the transcription config used for the session. This results in a SetRecognitionConfig message sent to the server.
- Parameters
new_transcription_config (speechmatics.models.TranscriptionConfig) – The new config object.


We're sorry but material doesn't work properly without JavaScript enabled. Please enable it to continue.


Japan stresses to China importance of free, open Hong Kong
by ReutersTOKYO (Nov 25): Japanese Foreign Minister Toshimitsu Motegi on Monday told his Chinese counterpart that he is highly concerned about the Hong Kong situation and it is important that a free and open Hong Kong keeps on prospering under the "one country, two systems" model, a Japanese government official said.
The official talked to a media briefing after a meeting in Tokyo between Motegi and China's state councillor and foreign minister, Wang Yi.
Prolonged pro-democracy protests in Hong Kong are posing the biggest popular challenge to Chinese President Xi Jinping since he came to power in 2012.


About Us
This project is the Code Fellows 201 final group project.
We chose to do this project because we want to make an easy-to-use platform for all of the movie lovers out there.
This web app provides you with a list of the most famous movies where you can view and add to your personalized watchlist or favoriteList.


Returns the bounds of the block stack of this, where this block
is at the top of its stack (in other words, it does not take the
bounds of the blocks above it into account).
Returns a list of the stub kinds (or stub genus names) of this; if this genus does not have any stubs,
returns an empty list
FORWARDED FROM BLOCK GENUS


let-modal
to
the template element to access the injected ModalRef
.
afterClosed
to access the returned value.
A dismissal will trigger an error on the observable, similar to rejecting a promise.
ModalRef
inside the content component to access methods such as close and dismiss.
The ModalService supports limitless stacked modals. Each can be closed individually, there is no specific closing order.
In the example below, it is accomplished via components as content. However, it is entirely possible with templates through the same methodology.
Modals make use of a singleton service called ModalService
.
It provides the ability to open()
modal windows.
The open()
method accepts either a TemplateRef or Component as modal content. In addition, you may optionally provide a ModalConfig object to override some system-default modal window behaviours.
When a modal window is opened, the content is positioned in a parent element that is appended to the body.
Importing one of BrowserAnimationsModule
or NoopAnimationsModule
is required.
The first is the standard Angular animation module, it animates content as expected. NoopAnimationsModule mocks the animations and does not render them, potentially increasing performance.


Before Change
ext = buffer_size + 1
slstart = slice(ext)
slend = slice(-ext, None)
slices = [slice(s) for s in image.shape]
for d in range(image.ndim):
slicedim = list(slices)
slicedim[d] = slstart
borders[tuple(slicedim)] = True
slicedim[d] = slend
borders[tuple(slicedim)] = True
// Re-label, in case we are dealing with a binary image
// and to get consistent labeling
labels = label(image, background=0)
number = np.max(labels) + 1
// determine all objects that are connected to borders
borders_indices = np.unique(labels[borders])
indices = np.arange(number + 1)
After Change
raise ValueError("buffer size may not be greater than labels size")
if out is not None:
np.copyto(out, labels, casting="no")
in_place = True
if not in_place:
out = labels.copy()