class: title Docker
Orchestration
Workshop ??? ??? ## Intros - Hello! We are AJ ([@s0ulshake](https://twitter.com/s0ulshake)) & Jérôme ([@jpetazzo](https://twitter.com/jpetazzo)) ??? - This is our collective Docker knowledge: ![Bell Curve](bell-curve.jpg) ??? ## Agenda .small[ - 09:00-09:15 hello - 09:15-10:45 part 1 - 10:45-11:00 coffee break - 11:00-12:30 part 2 - 12:30-13:30 lunch break - 13:30-15:00 part 3 - 15:00-15:15 coffee break - 15:15-16:45 part 4 - 16:45-17:00 Q&A ] - Feel free to interrupt for questions at any time - Live feedback, questions, help on [Slack](http://container.training/chat) ([get an invite](http://lisainvite.herokuapp.com/)) - All the content is publicly available (slides, code samples, scripts) --- ## Chapter 1: getting started - Pre-requirements - VM environment - Our sample application - Running the application - Introducing SwarmKit --- ## Chapter 2: scaling out our app on Swarm - Creating our first Swarm - Running our first Swarm service - Deploying a local registry - Overlay networks - Breaking into an overlay network - Securing overlay networks ??? ## Chapter 3: operating the Swarm - Rolling updates - Secrets management and encryption at rest - Metrics collection --- ## Chapter 3: deeper in Swarm - Dealing with stateful services - Scripting image building and pushing - Integration with Compose - Controlling Docker from a container - Node management - What's next? ??? # Pre-requirements - Computer with network connection and SSH client - on Linux, OS X, FreeBSD... you are probably all set - on Windows, get [putty](http://www.putty.org/), Microsoft [Win32 OpenSSH](https://github.com/PowerShell/Win32-OpenSSH/wiki/Install-Win32-OpenSSH), [Git BASH](https://git-for-windows.github.io/), or [MobaXterm](http://mobaxterm.mobatek.net/) - Some Docker knowledge (but that's OK if you're not a Docker expert!) --- ## Nice-to-haves - [Mosh](https://mosh.org/) if your internet connection tends to lose packets
(available with `(apt|yum|brew) install mosh`; then use it instead of SSH) - [GitHub](https://github.com/join) account
(if you want to fork the repo; also used to join Gitter) - [Docker Hub](https://hub.docker.com) account
(it's one way to distribute images on your Swarm cluster) --- ## Hands-on sections - The whole workshop is hands-on - We will see Docker 1.13 in action - You are invited to reproduce all the demos - All hands-on sections are clearly identified, like the gray rectangle below .exercise[ - This is the stuff you're supposed to do! ] --- # VM environment - To follow along, you need a cluster of five Docker Engines - If you are doing this with an instructor, see next slide - If you are doing (or re-doing) this on your own, you can: - create your own cluster (local or cloud VMs) with Docker Machine ([instructions](https://github.com/ebriand/orchestration-workshop/tree/master/prepare-machine)) - use [Play-With-Docker](http://play-with-docker.com) ([instructions](https://github.com/ebriand/orchestration-workshop#using-play-with-docker)) - create a bunch of clusters for you and your friends ([instructions](https://github.com/ebriand/orchestration-workshop/tree/master/prepare-vms)) --- class: pic ![You get five VMs](you-get-five-vms.jpg) --- ## You get five VMs - Each person gets 5 private VMs (not shared with anybody else) - They'll remain up until the day after the tutorial - You should have a little card with login+password+IP addresses - You can automatically SSH from one VM to another .exercise[ - Log into the first VM (`node1`) with SSH or MOSH - Check that you can SSH (without password) to `node2`: ```bash ssh node2 ``` - Type `exit` or `^D` to come back to node1 ] --- ## If doing or re-doing the workshop on your own ... - If you use Play-With-Docker: - you can't SSH to the machines - you access the terminal directly in your browser - exposing services requires something like [ngrok](https://ngrok.com/) or [supergrok](https://github.com/ebriand/orchestration-workshop#using-play-with-docker) - If you use VMs deployed with Docker Machine: - you won't have pre-authorized SSH keys to bounce across machines - you won't have host aliases --- ## We will (mostly) interact with node1 only - Unless instructed, **all commands must be run from the first VM, `node1`** - We will only checkout/copy the code on `node1` - When we will use the other nodes, we will do it mostly through the Docker API - We will use SSH only for the initial setup and a few "out of band" operations
(checking internal logs, debugging...) --- ## Terminals Once in a while, the instructions will say:
"Open a new terminal." There are multiple ways to do this: - create a new window or tab on your machine, and SSH into the VM; - use screen or tmux on the VM and open a new window from there. You are welcome to use the method that you feel the most comfortable with. ??? ## Tmux cheatsheet - Ctrl-b c → creates a new window - Ctrl-b n → go to next window - Ctrl-b p → go to previous window - Ctrl-b " → split window top/bottom - Ctrl-b % → split window left/right - Ctrl-b Alt-1 → rearrange windows in columns - Ctrl-b Alt-2 → rearrange windows in rows - Ctrl-b arrows → navigate to other windows - Ctrl-b d → detach session - tmux attach → reattach to session --- ## Brand new versions! - Engine 1.13-rc - Compose 1.9.0 - Machine 0.9-rc .exercise[ - Check all installed versions: ```bash docker version docker-compose -v docker-machine -v ``` ] --- # Our sample application - Visit the GitHub repository with all the materials of this workshop:
https://github.com/ebriand/orchestration-workshop - The application is in the [dockercoins]( https://github.com/ebriand/orchestration-workshop/tree/master/dockercoins) subdirectory - Let's look at the general layout of the source code: there is a Compose file [docker-compose.yml]( https://github.com/ebriand/orchestration-workshop/blob/master/dockercoins/docker-compose.yml) ... ... and 4 other services, each in its own directory: - `rng` = web service generating random bytes - `hasher` = web service computing hash of POSTed data - `worker` = background process using `rng` and `hasher` - `webui` = web interface to watch progress ??? --- ## Compose file format version *Particularly relevant if you have used Compose before...* - Compose 1.6 introduced support for a new Compose file format (aka "v2") - Services are no longer at the top level, but under a `services` section - There has to be a `version` key at the top level, with value `"2"` (as a string, not an integer) - Containers are placed on a dedicated network, making links unnecessary - There are other minor differences, but upgrade is easy and straightforward --- ## Links, naming, and service discovery - Containers can have network aliases (resolvable through DNS) - Compose file version 2 makes each container reachable through its service name - Compose file version 1 requires "links" sections - Our code can connect to services using their short name (instead of e.g. IP address or FQDN) --- ## Example in `worker/worker.py` ![Service discovery](service-discovery.png) --- ## What's this application? --- class: pic ![DockerCoins logo](dockercoins.png) (DockerCoins 2016 logo courtesy of [@XtlCnslt](https://twitter.com/xtlcnslt) and [@ndeloof](https://twitter.com/ndeloof). Thanks!) --- ## What's this application? - It is a DockerCoin miner! 💰🐳📦🚢 - No, you can't buy coffee with DockerCoins - How DockerCoins works: - `worker` asks to `rng` to give it random bytes - `worker` feeds those random bytes into `hasher` - each hash starting with `0` is a DockerCoin - DockerCoins are stored in `redis` - `redis` is also updated every second to track speed - you can see the progress with the `webui` --- ## Getting the application source code - We will clone the GitHub repository - The repository also contains scripts and tools that we will use through the workshop .exercise[ - Clone the repository on `node1`: ```bash git clone git://github.com/ebriand/orchestration-workshop ``` ] (You can also fork the repository on GitHub and clone your fork if you prefer that.) --- # Running the application Without further ado, let's start our application. .exercise[ - Go to the `dockercoins` directory, in the cloned repo: ```bash cd ~/orchestration-workshop/dockercoins ``` - Use Compose to build and run all containers: ```bash docker-compose up ``` ] Compose tells Docker to build all container images (pulling the corresponding base images), then starts all containers, and displays aggregated logs. --- ## Lots of logs - The application continuously generates logs - We can see the `worker` service making requests to `rng` and `hasher` - Let's put that in the background .exercise[ - Stop the application by hitting `^C` ] - `^C` stops all containers by sending them the `TERM` signal - Some containers exit immediately, others take longer
(because they don't handle `SIGTERM` and end up being killed after a 10s timeout) --- ## Restarting in the background - Many flags and commands of Compose are modeled after those of `docker` .exercise[ - Start the app in the background with the `-d` option: ```bash docker-compose up -d ``` - Check that our app is running with the `ps` command: ```bash docker-compose ps ``` ] `docker-compose ps` also shows the ports exposed by the application. ??? --- ## Viewing logs - The `docker-compose logs` command works like `docker logs` .exercise[ - View all logs since container creation and exit when done: ```bash docker-compose logs ``` - Stream container logs, starting at the last 10 lines for each container: ```bash docker-compose logs --tail 10 --follow ``` ] Tip: use `^S` and `^Q` to pause/resume log output. ??? ??? ## Upgrading from Compose 1.6 .warning[The `logs` command has changed between Compose 1.6 and 1.7!] - Up to 1.6 - `docker-compose logs` is the equivalent of `logs --follow` - `docker-compose logs` must be restarted if containers are added - Since 1.7 - `--follow` must be specified explicitly - new containers are automatically picked up by `docker-compose logs` --- ## Connecting to the web UI - The `webui` container exposes a web dashboard; let's view it .exercise[ - Open http://[yourVMaddr]:8000/ (from a browser) ] - The app actually has a constant, steady speed (3.33 coins/second) - The speed seems not-so-steady because: - the worker doesn't update the counter after every loop, but up to once per second - the speed is computed by the browser, checking the counter about once per second - between two consecutive updates, the counter will increase either by 4, or by 0 ??? ## Scaling up the application - Our goal is to make that performance graph go up (without changing a line of code!) ?? - Before trying to scale the application, we'll figure out if we need more resources (CPU, RAM...) - For that, we will use good old UNIX tools on our Docker node ??? ## Looking at resource usage - Let's look at CPU, memory, and I/O usage .exercise[ - run `top` to see CPU and memory usage (you should see idle cycles) - run `vmstat 3` to see I/O usage (si/so/bi/bo)
(the 4 numbers should be almost zero, except `bo` for logging) ] We have available resources. - Why? - How can we use them? --- ## Scaling workers on a single node - Docker Compose supports scaling - Let's scale `worker` and see what happens! .exercise[ - Start one more `worker` container: ```bash docker-compose scale worker=2 ``` - Look at the performance graph (it should show a x2 improvement) - Look at the aggregated logs of our containers (`worker_2` should show up) - Look at the impact on CPU load with e.g. top (it should be negligible) ] --- ## Adding more workers - Great, let's add more workers and call it a day, then! .exercise[ - Start eight more `worker` containers: ```bash docker-compose scale worker=10 ``` - Look at the performance graph: does it show a x10 improvement? - Look at the aggregated logs of our containers - Look at the impact on CPU load and memory usage ] ??? # Identifying bottlenecks - You should have seen a 3x speed bump (not 10x) - Adding workers didn't result in linear improvement - *Something else* is slowing us down - ... But what? - The code doesn't have instrumentation - Let's use state-of-the-art HTTP performance analysis!
(i.e. good old tools like `ab`, `httping`...) ??? ## Accessing internal services - `rng` and `hasher` are exposed on ports 8001 and 8002 - This is declared in the Compose file: ```yaml ... rng: build: rng ports: - "8001:80" hasher: build: hasher ports: - "8002:80" ... ``` ??? ## Measuring latency under load We will use `httping`. .exercise[ - Check the latency of `rng`: ```bash httping -c 10 localhost:8001 ``` - Check the latency of `hasher`: ```bash httping -c 10 localhost:8002 ``` ] `rng` has a much higher latency than `hasher`. ??? ## Let's draw hasty conclusions - The bottleneck seems to be `rng` - *What if* we don't have enough entropy and can't generate enough random numbers? - We need to scale out the `rng` service on multiple machines! Note: this is a fiction! We have enough entropy. But we need a pretext to scale out. (In fact, the code of `rng` uses `/dev/urandom`, which never runs out of entropy...
...and is [just as good as `/dev/random`](http://www.slideshare.net/PacSecJP/filippo-plain-simple-reality-of-entropy).) --- ## Clean up - Before moving on, let's remove those containers .exercise[ - Tell Compose to remove everything: ```bash docker-compose down ``` ] --- class: title # Scaling out --- # SwarmKit - [SwarmKit](https://github.com/docker/swarmkit) is an open source toolkit to build multi-node systems - It is a reusable library, like libcontainer, libnetwork, vpnkit ... - It is a plumbing part of the Docker ecosystem - SwarmKit comes with two examples: - `swarmctl` (a CLI tool to "speak" the SwarmKit API) - `swarmd` (an agent that can federate existing Docker Engines into a Swarm) - SwarmKit/swarmd/swarmctl → libcontainer/containerd/container-ctr --- ## SwarmKit features - Highly-available, distributed store based on [Raft]( https://en.wikipedia.org/wiki/Raft_(computer_science))
(more on next slide) - *Services* managed with a *declarative API*
(implementing *desired state* and *reconciliation loop*) - Automatic TLS keying and signing - Dynamic promotion/demotion of nodes, allowing to change how many nodes are actively part of the Raft consensus - Integration with overlay networks and load balancing - And much more! --- ## Where is the key/value store? - Many orchestration systems use a key/value store backed by a consensus algorithm
(k8s→etcd→Raft, mesos→zookeeper→ZAB, etc.) - SwarmKit implements the Raft algorithm directly
(Nomad is similar; thanks [@cbednarski](https://twitter.com/@cbednarski), [@diptanu](https://twitter.com/diptanu) and others for point it out!) - Analogy courtesy of [@aluzzardi](https://twitter.com/aluzzardi): *It's like B-Trees and RDBMS. They are different layers, often associated. But you don't need to bring up a full SQL server when all you need is to index some data.* - As a result, the orchestrator has direct access to the data
(the main copy of the data is stored in the orchestrator's memory) - Simpler, easier to deploy and operate; also faster --- ## SwarmKit concepts (1/2) - A *cluster* will be at least one *node* (preferably more) - A *node* can be a *manager* or a *worker* (Note: in SwarmKit, *managers* are also *workers*) - A *manager* actively takes part in the Raft consensus - You can talk to a *manager* using the SwarmKit API - One *manager* is elected as the *leader*; other managers merely forward requests to it --- ## Illustration ![Illustration](swarm-mode.svg) --- ## SwarmKit concepts (2/2) - The *managers* expose the SwarmKit API - Using the API, you can indicate that you want to run a *service* - A *service* is specified by its *desired state*: which image, how many instances... - The *leader* uses different subsystems to break down services into *tasks*:
orchestrator, scheduler, allocator, dispatcher - A *task* corresponds to a specific container, assigned to a specific *node* - *Nodes* know which *tasks* should be running, and will start or stop containers accordingly (through the Docker Engine API) You can refer to the [NOMENCLATURE](https://github.com/docker/swarmkit/blob/master/design/nomenclature.md) in the SwarmKit repo for more details. --- ## Swarm Mode - Since version 1.12, Docker Engine embeds SwarmKit - The Docker CLI features three new commands: - `docker swarm` (enable Swarm mode; join a Swarm; adjust cluster parameters) - `docker node` (view nodes; promote/demote managers; manage nodes) - `docker service` (create and manage services) - The Docker API exposes the same concepts - The SwarmKit API is also exposed (on a separate socket) --- ## You need to enable Swarm mode to use the new stuff - By default, all this new code is inactive - Swarm Mode can be enabled, "unlocking" SwarmKit functions
(services, out-of-the-box overlay networks, etc.) .exercise[ - Try a Swarm-specific command: ```bash docker node ls ``` ] -- You will get an error message: ``` Error response from daemon: This node is not a swarm manager. [...] ``` --- # Creating our first Swarm - The cluster is initialized with `docker swarm init` - This should be executed on a first, seed node - .warning[DO NOT execute `docker swarm init` on multiple nodes!] You would have multiple disjoint clusters. .exercise[ - Create our cluster from node1: ```bash docker swarm init ``` ] If Docker tells you that it `could not choose an IP address to advertise`, see next slide! --- ## IP address to advertise - When running in Swarm mode, each node *advertises* its address to the others
(i.e. it tells them *"you can contact me on <IP_ADDRESS>:2377"*) - If the node has only one IP address (other than <IP_ADDRESS>), it is used automatically - If the node has multiple IP addresses, you **must** specify which one to use
(Docker refuses to pick one randomly) - You can specify an IP address or an interface name
(in the latter case, Docker will read the IP address of the interface and use it) - You can also specify a port number
(otherwise, the default port 2377 will be used) --- ## Which IP address should be advertised? - If your nodes have only one IP address, it's safe to let autodetection do the job .small[(Except if your instances have different private and public addresses, e.g. on EC2, and you are building a Swarm involving nodes inside and outside the private network: then you should advertise the public address.)] - If your nodes have multiple IP addresses, pick an address which is reachable *by every other node* of the Swarm - If you are using [play-with-docker](http://play-with-docker.com/), use the IP address shown next to the node name .small[(This is the address of your node on your private internal overlay network. The other address that you might see is the address of your node on the `docker_gwbridge` network, which is used for outbound traffic.)] Examples: ```bash docker swarm init --advertise-addr <IP_ADDRESS> docker swarm init --advertise-addr eth0:7777 ``` --- ## Token generation - In the output of `docker swarm init`, we have a message confirming that our node is now the (single) manager: ``` Swarm initialized: current node (8jud...) is now a manager. ``` - Docker generated two security tokens (like passphrases or passwords) for our cluster - The CLI shows us the command to use on other nodes to add them to the cluster using the "worker" security token: ``` To add a worker to this swarm, run the following command: docker swarm join \ --token SWMTKN-1-59fl4ak4nqjmao1ofttrc4eprhrola2l87... \ <IP_ADDRESS>:2377 ``` --- ## Checking that Swarm mode is enabled .exercise[ - Run the traditional `docker info` command: ```bash docker info ``` ] The output should include: ``` Swarm: active NodeID: 8jud7o8dax3zxbags3f8yox4b Is Manager: true ClusterID: 2vcw2oa9rjps3a24m91xhvv0c ... ``` --- ## Running our first Swarm mode command - Let's retry the exact same command as earlier .exercise[ - List the nodes (well, the only node) of our cluster: ```bash docker node ls ``` ] The output should look like the following: ``` ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS 8jud...ox4b * node1 Ready Active Leader ``` --- ## Adding nodes to the Swarm - A cluster with one node is not a lot of fun - Let's add `node2`! - We need the token that was shown earlier -- - You wrote it down, right? -- - Don't panic, we can easily see it again 😏 --- ## Adding nodes to the Swarm .exercise[ - Show the token again: ```bash docker swarm join-token worker ``` - Log into `node2`: ```bash ssh node2 ``` - Copy paste the `docker swarm join ...` command
(that was displayed just before) ] ??? --- ## Check that the node was added correctly - Stay logged into `node2`! .exercise[ - We can still use `docker info` to verify that the node is part of the Swarm: ```bash docker info | grep ^Swarm ``` ] - However, Swarm commands will not work; try, for instance: ``` docker node ls ``` - This is because the node that we added is currently a *worker* - Only *managers* can accept Swarm-specific commands --- ## View our two-node cluster - Let's go back to `node1` and see what our cluster looks like .exercise[ - Logout from `node2` (with `exit` or `Ctrl-D` or ...) - View the cluster from `node1`, which is a manager: ```bash docker node ls ``` ] The output should be similar to the following: ``` ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS 8jud...ox4b * node1 Ready Active Leader ehb0...4fvx node2 Ready Active ``` ??? ## Adding nodes using the Docker API - We don't have to SSH into the other nodes, we can use the Docker API - Our nodes expose the Docker API over port 2376/tcp,
protected by TLS mutual authentication - To connect to other nodes with the Docker API, we will use Docker Machine (Our nodes have been suitably pre-configured to be controlled through `node1`) ??? ## Docker Machine - Docker Machine has two primary uses: - provisioning cloud instances running the Docker Engine - managing local Docker VMs within e.g. VirtualBox - Docker Machine is purely optional - It makes it easy to create, upgrade, manage... Docker hosts: - on your favorite cloud provider - locally (e.g. to test clustering, or different versions) - across different cloud providers ??? ## Docker Machine basic usage - We will learn two commands: - `docker-machine ls` (list existing hosts) - `docker-machine env` (switch to a specific host) .exercise[ - List configured hosts: ```bash docker-machine ls ``` ] You should see your 5 nodes. ??? ## How did we make our 5 nodes show up there? *For the curious...* - This was done by our VM provisioning scripts - After setting up everything else, `node1` adds the 5 nodes to the local Docker Machine configuration (located in `$HOME/.docker/machine`) - Nodes are added using [Docker Machine generic driver](https://docs.docker.com/machine/drivers/generic/) (It skips machine provisioning and jumps straight to the configuration phase) - Docker Machine creates TLS certificates and deploys them to the nodes through SSH ??? ## Using Docker Machine to communicate with a node - To select a node, use `eval $(docker-machine env nodeX)` - This sets a number of environment variables - To unset these variables, use `eval $(docker-machine env -u)` .exercise[ - View the variables used by Docker Machine: ```bash docker-machine env node3 ``` ] ??? ## Getting the token - First, let's store the join token in a variable .exercise[ - Make sure we talk to the local node: ```bash eval $(docker-machine env -u) ``` - Get the join token: ```bash TOKEN=$(docker swarm join-token -q worker) ``` ] ??? ## Adding a node through the Docker API - Now, let's use Docker Machine to switch to `node3` and add it to the cluster .exercise[ - Communicate with `node3`: ```bash eval $(docker-machine env node3) ``` - Add `node3` to the Swarm: ```bash docker swarm join --token $TOKEN node1:2377 ``` ] Note: it can be useful to use a [custom shell prompt]( https://github.com/ebriand/orchestration-workshop/blob/master/prepare-vms/scripts/postprep.rc#L68) reflecting the `DOCKER_HOST` variable. ??? ## Checking that our node is here - We have to go back to the local node first .exercise[ - Reset the environment variables: ```bash eval $(docker-machine env -u) ``` - Check that the node is here: ```bash docker node ls ``` ] ??? ## Under the hood: docker swarm init When we do `docker swarm init`: - a keypair is created for the root CA of our Swarm - a keypair is created for the first node - a certificate is issued for this node - the join tokens are created ??? ## Under the hood: join tokens There is one token to *join as a worker*, and another to *join as a manager*. The join tokens have two parts: - a secret key (preventing unauthorized nodes from joining) - a fingerprint of the root CA certificate (preventing MITM attacks) If a token is compromised, it can be rotated instantly with: ``` docker swarm join-token --rotate
``` ??? ## Under the hood: docker swarm join When a node joins the Swarm: - it is issued its own keypair, signed by the root CA - if the node is a manager: - it joins the Raft consensus - it connects to the current leader - it accepts connections from worker nodes - if the node is a worker: - it connects to one of the managers (leader or follower) ??? ## Under the hood: cluster communication - The *control plane* is encrypted with AES-GCM; keys are rotated every 12 hours - Authentication is done with mutual TLS; certificates are rotated every 90 days (`docker swarm update` allows to change this delay or to use an external CA) - The *data plane* (communication between containers) is not encrypted by default (but this can be activated on a by-network basis, using IPSEC, leveraging hardware crypto if available) --- ## Under the hood: I want to know more! Revisit SwarmKit concepts: - Docker 1.12 Swarm Mode Deep Dive Part 1: Topology ([video](https://www.youtube.com/watch?v=dooPhkXT9yI)) - Docker 1.12 Swarm Mode Deep Dive Part 2: Orchestration ([video](https://www.youtube.com/watch?v=_F6PSP-qhdA)) Some presentations from the Docker Distributed Systems Summit in Berlin: - Heart of the SwarmKit: Topology Management ([slides](https://speakerdeck.com/aluzzardi/heart-of-the-swarmkit-topology-management)) - Heart of the SwarmKit: Store, Topology & Object Model ([slides](http://www.slideshare.net/Docker/heart-of-the-swarmkit-store-topology-object-model)) ([video](https://www.youtube.com/watch?v=EmePhjGnCXY)) ??? ## Adding more manager nodes - Right now, we have only one manager (node1) - If we lose it, we're SOL - Let's make our cluster highly available .exercise[ - Get the manager token and use it to add nodes 4 and 5 to the cluster: ```bash TOKEN=$(docker swarm join-token -q manager) for N in 4 5; do eval $(docker-machine env node$N) docker swarm join --token $TOKEN node1:2377 done ``` ] --- ## You can control the Swarm from any manager node .exercise[ - Try the following command on a few different nodes: ```bash ssh nodeX docker node ls ``` ] On manager nodes:
you will see the list of nodes, with a `*` denoting the node you're talking to. On non-manager nodes:
you will get an error message telling you that the node is not a manager. As we saw earlier, you can only control the Swarm through a manager node. --- ## Promoting nodes - Instead of adding a manager node, we can also promote existing workers - Nodes can be promoted (and demoted) at any time .exercise[ - See the current list of nodes: ``` docker node ls ``` - Promote the two worker nodes to be managers: ``` docker node promote XXX YYY ``` ] --- ## How many managers do we need? - 2N+1 nodes can (and will) tolerate N failures
(you can have an even number of managers, but there is no point) -- - 1 manager = no failure - 3 managers = 1 failure - 5 managers = 2 failures (or 1 failure during 1 maintenance) - 7 managers and more = now you might be overdoing it a little bit --- ## Why not have *all* nodes be managers? - Intuitively, it's harder to reach consensus in larger groups - With Raft, each write needs to be acknowledged by the majority of nodes - More nodes = more chance that we will have to wait for some laggard - Bigger network = more latency --- ## What would McGyver do? - If some of your machines are more than 10ms away from each other,
try to break them down in multiple clusters (keeping internal latency low) - Groups of up to 9 nodes: all of them are managers - Groups of 10 nodes and up: pick 5 "stable" nodes to be managers - Groups of more than 100 nodes: watch your managers' CPU and RAM - Groups of more than 1000 nodes: - if you can afford to have fast, stable managers, add more of them - otherwise, break down your nodes in multiple clusters --- ## What's the upper limit? - We don't know! - Internal testing at Docker Inc.: 1000-10000 nodes is fine - deployed to a single cloud region - one of the main take-aways was *"you're gonna need a bigger manager"* - Testing by the community: [4700 heterogenous nodes all over the 'net](https://sematext.com/blog/2016/11/14/docker-swarm-lessons-from-swarm3k/) - it just works - more nodes require more CPU; more containers require more RAM - scheduling of large jobs (70000 containers) is slow, though (working on it!) --- # Running our first Swarm service - How do we run services? Simplified version: `docker run` → `docker service create` .exercise[ - Create a service featuring an Alpine container pinging Google resolvers: ```bash docker service create alpine ping <IP_ADDRESS> ``` - Check where the container was created: ```bash docker service ps
``` ] --- ## Checking container logs - Right now, there is no direct way to check the logs of our container
(unless it was scheduled on the current node) - Look up the `NODE` on which the container is running
(in the output of the `docker service ps` command) .exercise[ - Log into the node *or* use Docker Machine to talk to it: ```bash ssh nodeX ``` ```bash eval $(docker-machine env nodeX) ``` ] --- ## Viewing the logs of the container .exercise[ - See that the container is running and check its ID: ```bash docker ps ``` - View its logs: ```bash docker logs
``` - Go back to `node1` afterwards by logging out of SSH, or by running: ```bash eval $(docker-machine env -u) ``` ] --- ## Scale our service - Services can be scaled in a pinch with the `docker service update` command .exercise[ - Scale the service to ensure 2 copies per node: ```bash docker service update
--replicas 10 ``` - Check that we have two containers on the current node: ```bash docker ps ``` ] --- ## Expose a service - Services can be exposed, with two special properties: - the public port is available on *every node of the Swarm*, - requests coming on the public port are load balanced across all instances. - This is achieved with option `-p/--publish`; as an approximation: `docker run -p → docker service create -p` - If you indicate a single port number, it will be mapped on a port starting at 30000
(vs. 32768 for single container mapping) - You can indicate two port numbers to set the public port number
(just like with `docker run -p`) --- ## Expose ElasticSearch on its default port .exercise[ - Create an ElasticSearch service (and give it a name while we're at it): ```bash docker service create --name search --publish 9200:9200 --replicas 7 \ elasticsearch`:2` ``` - Check what's going on: ```bash watch docker service ps search ``` ] Note: don't forget the **:2**! The latest version of the ElasticSearch image won't start without mandatory configuration. --- ## Tasks lifecycle - If you are fast enough, you will be able to see multiple states: - assigned (the task has been assigned to a specific node) - preparing (right now, this mostly means "pulling the image") - running - When a task is terminated (stopped, killed...) it cannot be restarted (A replacement task will be created) --- ![diagram showing what happens during docker service create, courtesy of @aluzzardi](docker-service-create.svg) --- ## Test our service - We mapped port 9200 on the nodes, to port 9200 in the containers - Let's try to reach that port! .exercise[ - Repeat the following command a few times: ```bash curl localhost:9200 ``` ] You might get a few `Connection refused` errors; just keep trying. Each request should be served by a different ElasticSearch instance.
(You will see each instance advertising a different name.) --- ## Terminate our services - Before moving on, we will remove those services - `docker service rm` can accept multiple services names or IDs - `docker service ls` can accept the `-q` flag - A Shell snippet a day keeps the cruft away .exercise[ - Remove all services with this one liner: ```bash docker service ls -q | xargs docker service rm ``` ] --- class: title # Our app on Swarm --- ## What's on the menu? In this part, we will cover: - building images for our app, - shipping those images with a registry, - running them through the services concept, - enabling inter-container communication with overlay networks. --- ## Why do we need to ship our images? - When we do `docker-compose up`, images are built for our services - Those images are present only on the local node - We need those images to be distributed on the whole Swarm - The easiest way to achieve that is to use a Docker registry - Once our images are on a registry, we can reference them when creating our services --- ## Build, ship, and run, for a single service If we had only one service (built from a `Dockerfile` in the current directory), our workflow could look like this: ``` docker build -t jpetazzo/doublerainbow:v0.1 . docker push jpetazzo/doublerainbow:v0.1 docker service create jpetazzo/doublerainbow:v0.1 ``` We just have to adapt this to our application, which has 4 services! --- ## The plan - Build on our local node (`node1`) - Tag images with a version number (timestamp; git hash; semantic...) - Upload them to a registry - Create services using the images --- ## Which registry do we want to use? .small[ - **Docker Hub** - hosted by Docker Inc. - requires an account (free, no credit card needed) - images will be public (unless you pay) - located in AWS EC2 us-east-1 - **Docker Trusted Registry** - self-hosted commercial product - requires a subscription (free 30-day trial available) - images can be public or private - located wherever you want - **Docker open source registry** - self-hosted barebones repository hosting - doesn't require anything - doesn't come with anything either - located wherever you want ] ??? --- ## Using Docker Hub *If we wanted to use the Docker Hub...* - We would set the following environment variable: ```bash export DOCKER_REGISTRY=jpetazzo ``` (Using *our* Docker Hub login, of course!) - And we would log into the Docker Hub: ```bash docker login ``` ] ??? ??? ## Using Docker Trusted Registry *If we wanted to use DTR, we would...* - Make sure we have a Docker Hub account - [Activate a Docker Datacenter subscription]( https://hub.docker.com/enterprise/trial/) - Install DTR on our machines - Set `DOCKER_REGISTRY` to `dtraddress:port/user` *This is out of the scope of this workshop!* ??? ## Using open source registry - We need to run a `registry:2` container
(make sure you specify tag `:2` to run the new version!) - It will store images and layers to the local filesystem
(but you can add a config file to use S3, Swift, etc.) - Docker *requires* TLS when communicating with the registry - unless for registries on `localhost` - or with the Engine flag `--insecure-registry` - Our strategy: publish the registry container on port 5000,
so that it's available through `localhost:5000` on each node ??? # Deploying a local registry - We will create a single-instance service, publishing its port on the whole cluster .exercise[ - Create the registry service: ```bash docker service create --name registry --publish 5000:5000 registry:2 ``` - Try the following command, until it returns `{"repositories":[]}`: ```bash curl localhost:5000/v2/_catalog ``` ] (Retry a few times, it might take 10-20 seconds for the container to be started. Patience.) ??? ## Testing our local registry - We can retag a small image, and push it to the registry .exercise[ - Make sure we have the busybox image, and retag it: ```bash docker pull busybox docker tag busybox localhost:5000/busybox ``` - Push it: ```bash docker push localhost:5000/busybox ``` ] ??? ## Checking what's on our local registry - The registry API has endpoints to query what's there .exercise[ - Ensure that our busybox image is now in the local registry: ```bash curl http://localhost:5000/v2/_catalog ``` ] The curl command should now output: ```json {"repositories":["busybox"]} ``` --- ## Build, tag, and push our application container images - Scriptery to the rescue! .exercise[ - Set `DOCKER_REGISTRY` and `TAG` environment variables to use our local registry - And run this little for loop: ```bash DOCKER_REGISTRY=jpetazzo TAG=v0.1 for SERVICE in hasher rng webui worker; do docker-compose build $SERVICE docker tag dockercoins_$SERVICE $DOCKER_REGISTRY/dockercoins_$SERVICE:$TAG docker push $DOCKER_REGISTRY/dockercoins_$SERVICE done ``` ] --- # Overlay networks - SwarmKit integrates with overlay networks, without requiring an extra key/value store - Overlay networks are created the same way as before .exercise[ - Create an overlay network for our application: ```bash docker network create --driver overlay dockercoins ``` - Check existing networks: ```bash docker network ls ``` ] --- ## Can you spot the differences? The networks `dockercoins` and `ingress` are different from the other ones. Can you see how? -- - They are using a different kind of ID, reflecting the fact that they are SwarmKit objects instead of "classic" Docker Engine objects. - Their *scope* is `swarm` instead of `local`. - They are using the overlay driver. --- ## Caveats .warning[In Docker 1.12, you cannot join an overlay network with `docker run --net ...`.] Starting with version 1.13, you can, if the network was created with the `--attachable` flag. *Why is that?* Placing a container on a network requires allocating an IP address for this container. The allocation must be done by a manager node (worker nodes cannot update Raft data). As a result, `docker run --net ...` requires collaboration with manager nodes. It alters the code path for `docker run`, so it is allowed only under strict circumstances. --- ## Run the application - First, create the `redis` service; that one is using a Docker Hub image .exercise[ - Create the `redis` service: ```bash docker service create --network dockercoins --name redis redis ``` ] --- ## Run the other services - Then, start the other services one by one - We will use the images pushed previously .exercise[ - Start the other services: ```bash DOCKER_REGISTRY=localhost:5000 TAG=v0.1 for SERVICE in hasher rng webui worker; do docker service create --network dockercoins --name $SERVICE \ $DOCKER_REGISTRY/dockercoins_$SERVICE:$TAG done ``` ] ??? --- ## Wait for our application to be up - We will see later a way to watch progress for all the tasks of the cluster - But for now, a scrappy Shell loop will do the trick .exercise[ - Repeatedly display the status of all our services: ```bash watch "docker service ls -q | xargs -n1 docker service ps" ``` - Stop it once everything is running ] --- ## Expose our application web UI - We need to connect to the `webui` service, but it is not publishing any port - Let's reconfigure it to publish a port .exercise[ - Update `webui` so that we can connect to it from outside: ```bash docker service update webui --publish-add 8000:80 ``` ] Note: to "de-publish" a port, you would have to specify the container port. (i.e. in that case, `--publish-rm 80`) --- ## Connect to the web UI - The web UI is now available on port 8000, *on all the nodes of the cluster* .exercise[ - Point your browser to any node, on port 8000 ] You might have to wait a bit for the container to be up and running. Check its status with `docker service ps webui`. Protip: use `docker service ps webui -a` to see *all* tasks.
(Otherwise you only see the ones currently running.) --- ## Scaling the application - We can change scaling parameters with `docker update` as well - We will do the equivalent of `docker-compose scale` .exercise[ - Bring up more workers: ```bash docker service update worker --replicas 10 ``` - Check the result in the web UI ] You should see the performance peaking at 10 hashes/s (like before). --- ## Global scheduling - We want to utilize as best as we can the entropy generators on our nodes - We want to run exactly one `rng` instance per node - SwarmKit has a special scheduling mode for that, let's use it - We cannot enable/disable global scheduling on an existing service - We have to destroy and re-create the `rng` service --- ## Scaling the `rng` service .exercise[ - Remove the existing `rng` service: ```bash docker service rm rng ``` - Re-create the `rng` service with *global scheduling*: ```bash docker service create --name rng --network dockercoins --mode global \ $DOCKER_REGISTRY/dockercoins_rng:$TAG ``` - Look at the result in the web UI ] Note: if the hash rate goes to zero and doesn't climb back up, try to `rm` and `create` again. --- ## Why do we have to re-create the service to enable global scheduling? - Enabling it dynamically would make rolling updates semantics very complex - This might change in the future (after all, it was possible in 1.12 RC!) - As of Docker Engine 1.13, other parameters requiring to `rm`/`create` the service are: - service name - hostname - network --- ## Checkpoint - We've seen how to set up a Swarm - We've used it to host our own registry - We've built our app container images - We've used the registry to host those images - We've deployed and scaled our application Let's treat ourselves with a nice pat on the back! -- And carry on, we have much more to see and learn! --- ## Interlude: making our app "Swarm-ready" This app was written in June 2015. (One year before Swarm mode was released.) What did we change to make it compatible with Swarm mode? -- .exercise[ - Go to the app directory: ```bash cd ~/orchestration-workshop/dockercoins ``` - See modifications in the code: ```bash git log -p --since "4-JUL-2015" -- . ':!*.yml*' ':!*.html' ``` ] --- ## What did we change in our app since its inception? - Compose files - HTML file (it contains an embedded contextual tweet) - Dockerfiles (to switch to smaller images) - That's it! -- *We didn't change a single line of code in this app since it was written.* -- *The images that were [built in June 2015]( https://hub.docker.com/r/jpetazzo/dockercoins_worker/tags/) (when the app was written) can still run today ...
... in Swarm mode (distributed across a cluster, with load balancing) ...
... without any modification.* --- ## How did we design our app in the first place? - [Twelve-Factor App](https://12factor.net/) principles - Service discovery using DNS names - Initially implemented as "links" - Then "ambassadors" - And now "services" - Existing apps might require more changes! ??? class: title # Operating the Swarm ??? ## Troubleshooting overlay networks - We want to run tools like `ab` or `httping` on the internal network ?? - Ah, if only we had created our overlay network with the `--attachable` flag ... ?? - Oh well, let's use this as an excuse to introduce New Ways To Do Things --- # Rolling updates - We want to release a new version of the worker - We will edit the code ... - ... build the new image ... - ... push it to the registry ... - ... update our service to use the new image --- ## But first... - Restart the workers .exercise[ - Just scale back to 10 replicas: ```bash docker service update worker --replicas 10 ``` - Check that they're running: ```bash docker service ps worker ``` ] --- ## Making changes .exercise[ - Edit `~/orchestration-workshop/dockercoins/worker/worker.py` - Locate the line that has a `sleep` instruction - Reduce the `sleep` from `0.1` to `0.01` - Save your changes and exit ] --- ## Building and pushing the new image .exercise[ - Build the new image: ```bash IMAGE=localhost:5000/dockercoins_worker:v0.01 docker build -t $IMAGE worker ``` - Push it to the registry: ```bash docker push $IMAGE ``` ] Note how the build and push were fast (because caching). --- ## Watching the deployment process - We will need to open a new window for this .exercise[ - Look at our service status: ```bash watch -n1 "docker service ps worker -a | grep -v Shutdown.*Shutdown" ``` ] - `docker service ps worker` gives us all tasks
(including the one whose current or desired state is `Shutdown`) - Then we filter out the tasks whose current **and** desired state is `Shutdown` - Future versions might have fancy filters to make that less tinkerish --- ## Updating to our new image - Keep the `watch ...` command running! .exercise[ - In the other window, bring back the workers (if you had stopped them earlier): ```bash docker service update worker --replicas 10 ``` - Then, update the service to the new image: ```bash docker service update worker --image $IMAGE ``` ] By default, SwarmKit does a rolling upgrade, one instance at a time. --- ## Changing the upgrade policy - We can set upgrade parallelism (how many instances to update at the same time) - And upgrade delay (how long to wait between two batches of instances) .exercise[ - Change the parallelism to 2 and the delay to 5 seconds: ```bash docker service update worker --update-parallelism 2 --update-delay 5s ``` ] The current upgrade will continue at a faster pace. --- ## Rolling back - At any time (e.g. before the upgrade is complete), we can rollback .exercise[ - Rollback to the previous image: ```bash docker service update worker --image $DOCKER_REGISTRY/dockercoins_worker:v0.1 ``` - With Docker 1.13, we can also revert to the previous service specification: ```bash docker service update worker --rollback ``` ] Note: if you updated the roll-out parallelism, *rollback* will not rollback to the previous image; it will rollback to the previous roll-out cadence. --- ## Timeline of an upgrade - SwarmKit will upgrade N instances at a time
(following the `update-parallelism` parameter) - New tasks are created, and their desired state is set to `Ready`
.small[(this pulls the image if necessary, ensures resource availability, creates the container ... without starting it)] - If the new tasks fail to get to `Ready` state, go back to the previous step
.small[(SwarmKit will try again and again, until the situation is addressed or desired state is updated)] - When the new tasks are `Ready`, it sets the old tasks desired state to `Shutdown` - When the old tasks are `Shutdown`, it starts the new tasks - Then it waits for the `update-delay`, and continues with the next batch of instances --- # Dealing with stateful services - First of all, you need to make sure that the data files are on a *volume* - Volumes are host directories that are mounted to the container's filesystem - These host directories can be backed by the ordinary, plain host filesystem ... - ... Or by distributed/networked filesystems - In the latter scenario, in case of node failure, the data is safe elsewhere ... - ... And the container can be restarted on another node without data loss --- ## Building a stateful service experiment - We will use Redis for this example - We will expose it on port 10000 to access it easily .exercise[ - Start the Redis service: ```bash docker service create --name stateful -p 10000:6379 redis ``` - Check that we can connect to it (replace XX.XX.XX.XX with any node's IP address): ```bash docker run --rm redis redis-cli -h `XX.XX.XX.XX` -p 10000 info server ``` ] --- ## Accessing our Redis service easily - Typing that whole command is going to be tedious .exercise[ - Define a shell alias to make our lives easier: ```bash alias redis='docker run --rm redis redis-cli -h `XX.XX.XX.XX` -p 10000' ``` - Try it: ```bash redis info server ``` ] --- ## Basic Redis commands .exercise[ - Check that the `foo` key doesn't exist: ```bash redis get foo ``` - Set it to `bar`: ```bash redis set foo bar ``` - Check that it exists now: ```bash redis get foo ``` ] --- ## Local volumes vs. global volumes - Global volumes exist in a single namespace - A global volume can be mounted on any node
.small[(bar some restrictions specific to the volume driver in use; e.g. using an EBS-backed volume on a GCE/EC2 mixed cluster)] - Attaching a global volume to a container allows to start the container anywhere
(and retain its data wherever you start it!) - Global volumes require extra *plugins* (Flocker, Portworx...) - Docker doesn't come with a default global volume driver at this point - Therefore, we will fall back on *local volumes* --- ## Local volumes - We will use the default volume driver, `local` - As the name implies, the `local` volume driver manages *local* volumes - Since local volumes are (duh!) *local*, we need to pin our container to a specific host - We will do that with a *constraint* .exercise[ - Add a placement constraint to our service: ```bash docker service update stateful --constraint-add node.hostname==$HOSTNAME ``` ] --- ## Where is our data? - If we look for our `foo` key, it's gone! .exercise[ - Check the `foo` key: ```bash redis get foo ``` - Adding a constraint caused the service to be redeployed: ```bash docker service ps stateful -a ``` ] Note: even if the constraint ends up being a no-op (i.e. not moving the service), the service gets redeployed. This ensures consistent behavior. --- ## Setting the key again - Since our database was wiped out, let's populate it again .exercise[ - Set `foo` again: ```bash redis set foo bar ``` - Check that it's there: ```bash redis get foo ``` ] --- ## Service updates cause containers to be replaced - Let's try to make a trivial update to the service and see what happens .exercise[ - Set a memory limit to our Redis service: ```bash docker service update stateful --limit-memory 100M ``` - Try to get the `foo` key one more time: ```bash redis get foo ``` ] The key is blank again! --- ## Service volumes are ephemeral by default - Let's highlight what's going on with volumes! .exercise[ - Check the current list of volumes: ```bash docker volume ls ``` - Carry a minor update to our Redis service: ```bash docker service update stateful --limit-memory 200M ``` ] Again: all changes trigger the creation of a new task, and therefore a replacement of the existing container; even when it is not strictly technically necessary. --- ## The data is gone again - What happened to our data? .exercise[ - The list of volumes is slightly different: ```bash docker volume ls ``` ] (You should see one extra volume.) --- ## Assigning a persistent volume to the container - Let's add an explicit volume mount to our service, referencing a named volume .exercise[ - Update the service with a volume mount: ```bash docker service update stateful \ --mount-add type=volume,source=foobarstore,target=/data ``` - Check the new volume list: ```bash docker volume ls ``` ] Note: the `local` volume driver automatically creates volumes. --- ## Checking that persistence actually works across service updates .exercise[ - Store something in the `foo` key: ```bash redis set foo barbar ``` - Update the service with yet another trivial change: ```bash docker service update stateful --limit-memory 300M ``` - Check that `foo` is still set: ```bash redis get foo ``` ] --- ## Recap - The service must commit its state to disk when being shutdown.red[*] (Shutdown = being sent a `TERM` signal) - The state must be written on files located on a volume - That volume must be specified to be persistent - If using a local volume, the service must also be pinned to a specific node (And losing that node means losing the data, unless there are other backups) .footnote[
.red[*]Until recently, the Redis image didn't automatically persist data. Beware!] --- ## Cleaning up .exercise[ - Remove the stateful service: ```bash docker service rm stateful ``` - Remove the associated volume: ```bash docker volume rm foobarstore ``` ] Note: we could keep the volume around if we wanted. --- # Integration with Compose - We will now see how to streamline service creation (i.e. get rid of the `for SERVICE in ...; do docker service create ...` part) .warning[This is experimental and subject to change!] --- ## Compose file version 3 (New in Docker Engine 1.13) - Almost identical to version 2 - Can be directly used by a Swarm cluster through `docker stack ...` commands - Introduces a `deploy` section to pass Swarm-specific parameters - Resource limits are moved to this `deploy` section - See [here](https://github.com/aanand/docker.github.io/blob/8524552f99e5b58452fcb1403e1c273385988b71/compose/compose-file.md#upgrading) for the complete list of changes - Supersedes *Distributed Application Bundles* (JSON payload describing an application; could be generated from a Compose file) --- ## Removing everything - Before deploying using "stacks," let's get a clean slate .exercise[ - Remove *all* the services: ```bash docker service ls -q | xargs docker service rm ``` ] --- ## Our first stack We need a registry to move images around. Before, we deployed it with the following command: ```bash docker service create --publish 5000:5000 registry:2 ``` Now, we are going to deploy it with the following stack file: ```yaml version: "3" services: registry: image: registry:2 ports: - "5000:5000" ``` --- ## Checking our stack files - All the stack files that we will use are in the `stacks` directory .exercise[ - Go to the `stacks` directory: ```bash cd ~/orchestration-workshop/stacks ``` - Check `registry.yml`: ```bash cat registry.yml ``` ] --- ## Deploying our first stack - All stack manipulation commands start with `docker stack` - Under the hood, they map to `docker service` commands - Stacks have a *name* (which also serves as a namespace) - Stacks are specified with the aforementioned Compose file format version 3 .exercise[ - Deploy our local registry: ```bash docker stack deploy registry --compose-file registry.yml ``` ] --- ## Inspecting stacks - `docker stack ps` shows the detailed state of all services of a stack .exercise[ - Check that our registry is running correctly: ```bash docker stack ps registry ``` - Confirm that we get the same output with the following command: ```bash docker service ps registry_registry ``` ] --- ## Specifics of stack deployment Our registry is not *exactly* identical to the one deployed with `docker service create`! - Each stack gets its own overlay network - Services of the task are connected to this network
(unless specified differently in the Compose file) - Services get network aliases matching their name in the Compose file
(just like when Compose brings up an app specified in a v2 file) - Services are explicitly named `
_
` - Services and tasks also get an internal label indicating which stack they belong to --- ## Building and pushing stack services - We are going to use the `build` + `image` trick that we showed earlier: ```bash docker-compose -f my_stack_file.yml build docker-compose -f my_stack_file.yml push docker stack deploy my_stack --compose-file my_stack_file.yml ``` .exercise[ - Try it: ```bash docker-compose -f dockercoins.yml build ``` ] -- It doesn't work!?! --- ## Upgrading Compose - Compose file format version 3 is not supported in Compose 1.9 - We have to use 1.10 (which is not released yet) .exercise[ - Upgrade Compose, using the `master` branch: ```bash pip install git+git://github.com/docker/compose ``` - Re-hash our `$PATH`: ```bash hash docker-compose ``` ] --- ## Trying again - We can now build and push our container images .exercise[ - Build our application: ```bash docker-compose -f dockercoins.yml build ``` - Push the images to our registry: ```bash docker-compose -f dockercoins.yml push ``` ] Let's have a look at the `dockercoins.yml` file while this is building and pushing. --- ```yaml version: "3" services: rng: build: dockercoins/rng image: ${REGISTRY_SLASH-localhost:5000/}rng${COLON_TAG-:latest} logging: driver: gelf options: gelf-address: udp://localhost:12201 deploy: mode: global ... redis: image: redis ... worker: build: dockercoins/worker image: ${REGISTRY_SLASH-localhost:5000/}worker${COLON_TAG-:latest} ... deploy: replicas: 10 ``` --- ## Deploying the application - Now that the images are on the registry, we can deploy our application stack .exercise[ - Create the application stack: ```bash docker stack deploy dockercoins --compose-file dockercoins.yml ``` ] We can now connect to any of our nodes on port 8000, and we will see the familiar hashing speed graph. --- ## Caveats - Compose file format v3 is very new, and not written in the stone yet - It exists because nobody wants to maintain redundant files if they can avoid it - Some features are not fully supported yet: - logging, see [#29116](https://github.com/docker/docker/issues/29116) - secrets - You can re-run `docker stack deploy` to update a stack - ... But unsupported features will be wiped each time you redeploy (!) (This will likely be fixed/improved soon) --- ## Maintaining multiple environments There are many ways to handle variations between environments. - Compose loads `docker-compose.yml` and (if it exists) `docker-compose.override.yml` - Compose can load alternate file(s) by setting the `-f` flag or the `COMPOSE_FILE` environment variable - Compose files can *extend* other Compose files, selectively including services: ```yaml web: extends: file: common-services.yml service: webapp ``` See [this documentation page](https://docs.docker.com/compose/extends/) for more details about these techniques. --- ## Disk space management: `docker system df` - Shows disk usage for images, containers, and volumes - Breaks down between *active* and *reclaimable* categories .exercise[ - Check how much disk space is used at the end of the workshop: ```bash docker system df ``` ] Note: `docker system` is new in Docker Engine 1.13. --- ## Reclaiming unused resources: `docker system prune` - Removes stopped containers - Removes dangling images (that don't have a tag associated anymore) - Removes orphaned volumes - Removes empty networks .exercise[ - Try it: ```bash docker system prune -f ``` ] Note: `docker system prune -a` will also remove *unused* images. --- class: title # Thanks!
Questions?


Dust
Save Level
Load Level
Paused
Too Much Dust :
'
(
Sand
Oil
Fire
Water
Solid
Lava
Spring
Volcanic
Oil Well
Life Itself
C4 ;)
Eraser
Infectant
Brushes
Circle
Square
Boxfill
Press space to pause,
R to refresh


My Team
Roger
Manager
ID: 1
Email:
<EMAIL_ADDRESS>Office number: 1
Steph
Engineer
ID: 3
Email:
<EMAIL_ADDRESS>GitHub:
steph1
Jorge
Intern
ID: 2
Email:
<EMAIL_ADDRESS>School: U of M


French Toast
Recipe from Natasha Kravchuk
Ingredients
- 6 large eggs
- 2 large egg yolks
- 1 cup whole milk
- 1/4 tsp salt
- 2 tsp vanilla extract
- 1 tsp ground cinnamon
- 1 Tbsp warm honey
- 1 lb bread such as Challah, Brioche, or Texas Toast (about 8-10, 3/4”-thick slices)
- 3 Tbsp unsalted butter to saute toasts


WITNESS LIST State Recreational Resources Committee (Subcommittee on Fishing Licenses) May 17, 2000-10:00A Fishing Licenses On: Burgdorf, Jayna (Texas Parks and Wildlife Department) Kurzawski, Ken (Texas Parks and Wildlife Department)


| Opcode | Instruction | 64-Bit Mode | Compat/Leg Mode | Description |
|---|---|---|---|---|
| DE /2 | FICOM m16int | Valid | Valid | Compare ST(0) with m16int. |
| DA /2 | FICOM m32int | Valid | Valid | Compare ST(0) with m32int. |
| DE /3 | FICOMP m16int | Valid | Valid | Compare ST(0) with m16int and pop stack register. |
| DA /3 | FICOMP m32int | Valid | Valid | Compare ST(0) with m32int and pop stack register. |
Compares the value in ST(0) with an integer source operand and sets the condition code flags C0, C2, and C3 in the FPU status word according to the results (see table below). The integer value is converted to double extended-precision floating-point format before the comparison is made.
| Condition | C3 | C2 | C0 |
|---|---|---|---|
| ST(0) > SRC | 0 | 0 | 0 |
| ST(0) < SRC | 0 | 0 | 1 |
| ST(0) = SRC | 1 | 0 | 0 |
| Unordered | 1 | 1 | 1 |
These instructions perform an “unordered comparison.” An unordered comparison also checks the class of the numbers being compared (see “FXAM—Examine Floating-Point” in this chapter). If either operand is a NaN or is in an undefined format, the condition flags are set to “unordered.”
The sign of zero is ignored, so that –0.0 ← +0.0.
The FICOMP instructions pop the register stack following the comparison. To pop the register stack, the processor marks the ST(0) register empty and increments the stack pointer (TOP) by 1.
This instruction’s operation is the same in non-64-bit modes and 64-bit mode.
CASE (relation of operands) OF ST(0) > SRC: C3, C2, C0 ← 000; ST(0) < SRC: C3, C2, C0 ← 001; ST(0) = SRC: C3, C2, C0 ← 100; Unordered: C3, C2, C0 ← 111; ESAC; IF Instruction = FICOMP THEN PopRegisterStack; FI;
| C1 | Set to 0. |
| C0, C2, C3 | See table on previous page. |
| #IS | Stack underflow occurred. |
| #IA | One or both operands are NaN values or have unsupported formats. |
#D
One or both operands are denormal values.
| #GP(0) |
If a memory operand effective address is outside the CS, DS, ES, FS, or GS segment limit. If the DS, ES, FS, or GS register contains a NULL segment selector. |
| #SS(0) | If a memory operand effective address is outside the SS segment limit. |
| #NM | CR0.EM[bit 2] or CR0.TS[bit 3] = 1. |
| #PF(fault-code) | If a page fault occurs. |
| #AC(0) | If alignment checking is enabled and an unaligned memory reference is made while the current privilege level is 3. |
| #UD | If the LOCK prefix is used. |
| #GP | If a memory operand effective address is outside the CS, DS, ES, FS, or GS segment limit. |
| #SS | If a memory operand effective address is outside the SS segment limit. |
| #NM | CR0.EM[bit 2] or CR0.TS[bit 3] = 1. |
| #UD | If the LOCK prefix is used. |
| #GP(0) | If a memory operand effective address is outside the CS, DS, ES, FS, or GS segment limit. |
| #SS(0) | If a memory operand effective address is outside the SS segment limit. |
| #NM | CR0.EM[bit 2] or CR0.TS[bit 3] = 1. |
| #PF(fault-code) | If a page fault occurs. |
| #AC(0) | If alignment checking is enabled and an unaligned memory reference is made. |
| #UD | If the LOCK prefix is used. |
Same exceptions as in protected mode.
| #SS(0) | If a memory address referencing the SS segment is in a non-canonical form. |
| #GP(0) | If the memory address is in a non-canonical form. |
| #NM | CR0.EM[bit 2] or CR0.TS[bit 3] = 1. |
| #MF | If there is a pending x87 FPU exception. |
| #PF(fault-code) | If a page fault occurs. |
| #AC(0) | If alignment checking is enabled and an unaligned memory reference is made while the current privilege level is 3. |
| #UD | If the LOCK prefix is used. |


Skippy.aic A high-compatibility SCP Wiki Discord Bot
What did we do for nearly 250,000 years? We huddled in caves and around small fires, fearful of the things that we didn't understand. We called them 'gods' and 'demons', begged them to spare us, and prayed for salvation.
In time, their numbers dwindled and ours rose. The world began to make more sense when there were fewer things to fear, yet the unexplained can never truly go away.
Mankind must not go back to hiding in fear. No one else will protect us, and we must stand up for ourselves.
Secure. Contain. Protect.
Skippy.aic is the highest-compatibility SCP Wiki scraper Discord bot that allows users to display SCP articles in easily readable embeds, along with other useful and fun features.
Features/Commands
Open Source: If you have concerns with the bots you add to your servers, you can rest easy knowing that Skippy.aic’s code is completely free to view in order to ensure privacy (no matter how poo poo the code is). If you want, you can even download all the files and host your own version of the bot. Check out Skippy.aic’s GitHub!
SCP Embeds: Just by typing ./scp and a number, Skippy.aic can retrieve all the SCP’s information, including the Description, Object Class, Disruption Class, and Risk Class at a moment’s notice.
Article Refinement: Skippy.aic allows for JSON file overrides, which means that bugged article embeds can be fixed easily and great articles can look even better in Discord embeds.
Webhook Enhancement: Using the Foundation’s finest interdimensional travel technology and thaumaturgy, Skippy.aic can dynamically change it’s own name and profile picture to match the other-worldly embed content perfectly using secure and immediately-disposed-of webhooks.
Auto-Embeds: Skippy.aic can automatically detect compatible SCP Wiki URLs and respond with that article’s embed. If you share an article with friends, Skippy.aic can help to ensure the article is readable for everyone without having to click off of Discord.
Privacy Note: The Auto-Embed feature is optional. It’s code is completely open-source, free to read, and does not store detected messages. This feature can be disabled by going to the bot’s “Skippy.aic” role and removing the “Add Reactions” permission. This will cause the Auto-Embed function to immediately cancel and not view, reply to, or attempt to process any messages.
Highest-Compatibility: Skippy.aic supports a vast majority of typical formats used by SCP Wiki authors, including the Anomaly Classification System, the Flops Disruption Class header, the Woed Classified Bar, and, of course, the default Object Class system.


{{ data.meta.total_entries }} premises
-
{% for premises in data.entries %}
- {{premises.entry.name}} {% endfor %}
The list of FSA approved slaughterhouses, game handling establishments and cutting plants. The requirements for this list are laid down in Regulation (EC) 853/2004.
{{ data.meta.total_entries }} premises


Pizza Size
Small
Medium
Large
Extra Large
Pick your veggies:
Roma Tomatoes
Red Onions
Spinach
Jalapenos
Olives
Green Peppers
Mushrooms
Garlic
Pineapple
DriedTomatoes
Red Peppers
Zucchini
Pick your Meats:
anchovies
Red Beacon
Chicken
Ground Beef
Canadian Bacon
Sausage
Pepperoni
Salami
Chorizo
Capicollo
Steak
Lamb
Pick your Cheese:
Feta
Parmesan
Four Cheese Blend
Ground Goat Cheese
Mozzarella
Dairy Free
Extra Cheese
🍕 🍕 🍕 Place Order 🍕 🍕 🍕
Hello,
Here is your order:
Size:
Pizza
Toppings:
Total:
Update Order
New Order


--- layout: member-haha title: Laura Meyer category: member ---
MPP Candidate, Class of 2015
The University of Chicago Harris School of Public Policy
I am a current second-year student at Chicago Harris focused on monitoring and evaluation, international development, and education.
In particular, I am interested in ways to deploy new technology and better data to evaluate the impact of policies, programs, and interventions.
I am currently seeking full-time employment in evaluation and/or international development with a start date in Summer 2015. For more information,
current resume, connect with me on LinkedIn, or explore the links below
I look forward to getting in touch!
Experience and Skills |
|
|---|---|
| |
|
| |
|


Guide
Hit play: Use the arrow keys to move the Joystick and Spacebar to grab!
Player name
Score:
0
00:
00
Grab
Switch
Play
Claw Machine
×
You have more coins remaining , want to play again ?
Yes
No
Claw Machine
×
You just won a prize !
Share
No thanks


Tue or False: When writing a for-each loop, we need a colon (not a semicolon) to sepearate the variable and the collection of items
True, these two components go inside the loop's parentheses separated by a colon
- False, we separate these two components with a semicolon
- False, we separate these two components with four spaces
- False, we do not separate these two components with any punctuation
There are no hints for this question


Xiaomi Mi Note 10 to arrive in Spain on November 6th - Gizchina.com
by Efe UdinXiaomi Mi Note 10 to arrive in Spain on November 6th
3.5 (69.09%) 11 votes
Chinese manufacturer, Xiaomi, has an event on November 5th in China where it will release a couple of products which include Xiaomi CC9 Pro, Xiaomi Watch and Xiaomi TV 5. The CC9 Pro is considered as a version of the Xiaomi Mi Note 10 and earlier reports claim that the Note 10 will arrive in Spain on November 11. However. a new report from GadgetsFlix claims that it will come earlier. In fact, just a day after the CC9 Pro launch in China – November 6th.
According to previous information, the Mi Note 10 series will have two versions – Xiaomi Mi Note 10 and Note 10 Pro. Both phones already have NBTC and EEC certifications which means that they will arrive in parts of Europe and Asia. The Mi Note 10 just like the CC9 Pro will be the first smartphone with a 108MP camera.
According to Geekbench listing, the Mi Note 10 comes with model number “M1910F4G” while the Pro model is “M1910F4S”. The information from Geekbench basically confirms that the Mi Note 10 is the foreign version of Xiaomi CC9 Pro. Amongst other features, the Note 10 Pro will come with a Snapdragon 855+ SoC, 6.7-inch AMOLED display and 40W fast charging.


| Package | com.pblabs.tweaker |
| Class | public class GoogleSpreadsheetTweaker |
| Inheritance | GoogleSpreadsheetTweaker EntityComponent |
A big part of any game are the values that define how it behaves. Small changes in timing, forces, or scores can have big impacts on how fun a game is. The process of making a game fun involves endless tweaking and testing. Often the people doing that work are not programmers, so editing source code and recompiling can be a big barrier for them.
The GoogleSpreadsheetTweaker lets the game programmer map values on named objects and in templates to cells on a Google Spreadsheet.
<entity name="Tweaker"> <component name="Tweaker" type="com.pblabs.tweaker.GoogleSpreadsheetTweaker"> <SpreadsheetUrl>http://spreadsheets.google.com/feeds/cells/abc73bd7db73b73cb73b7c/od7/public/basic</SpreadsheetUrl> <ProxyUrl>http://mydomain.com/game/GoogleSpreadsheetProxy.php</ProxyUrl> <Config> <!-- Wave 1 --> <_><Cell>B3</Cell><Property>!Waves1.WaveManager.Waves.0.ObserverDescription</Property></_> </Config> </component> </entity>
| Property | Defined by | ||
|---|---|---|---|
| config : Array
List of TweakerMapEntry instances mapping cells to properties.
| GoogleSpreadsheetTweaker | ||
| groups : Array
Groups let you assign patterns of cells and properties to multiple items.
| GoogleSpreadsheetTweaker | ||
| isRegistered : Boolean
Whether or not the component is currently registered with an entity.
| EntityComponent | ||
| name : String
The name given to the component when it is added to an entity.
| EntityComponent | ||
| owner : IEntity
A reference to the entity that this component currently belongs to.
| EntityComponent | ||
| proxyUrl : String = "http://yoururl.com/yourproxyscript.php"
The URL for the proxy script, described in the web subfolder of this
project.
| GoogleSpreadsheetTweaker | ||
| spreadsheetUrl : String = ""
The feed for a publicly accessible Google Spreadsheet.
| GoogleSpreadsheetTweaker |
| Method | Defined by | ||
|---|---|---|---|
|
onAdd():void
| GoogleSpreadsheetTweaker | ||
|
onRemove():void
This is called when the component is removed from an entity.
| EntityComponent | ||
|
onReset():void
This is called anytime a component is added or removed from the owner entity.
| EntityComponent |
| config | property |
public var config:Array
List of TweakerMapEntry instances mapping cells to properties.
| groups | property |
public var groups:Array
Groups let you assign patterns of cells and properties to multiple items.
| proxyUrl | property |
public var proxyUrl:String = "http://yoururl.com/yourproxyscript.php"
The URL for the proxy script, described in the web subfolder of this project.
| spreadsheetUrl | property |
public var spreadsheetUrl:String = ""
The feed for a publicly accessible Google Spreadsheet.
The naming convention for these feeds is described at http://code.google.com/apis/spreadsheets/docs/3.0/reference.html#ConstructingURIs
Example of a working URL (key changed to protect the innocent): http://spreadsheets.google.com/feeds/cells/pZ6iqteeevF7uf4J123yqSw/od6/public/basic This gets the first sheet of the specified document.
| onAdd | () | method |
protected override function onAdd():void


Presenting Team 6
Backend Engineer
Current Automation engineer. Aspiring Devops engineer, Proficiency in Infrstructure and Cloud Computing.
sabidi1@mail.sfsu.edu


GODZILLA
Reviewed by Harvey Karten, Ph.D. TriStar Pictures Director: Roland Emmerich Writer: Roland Emmerich, Dean Devlin Cast: Matthew Broderick, Jean Reno, Maria Pitillo, Hank Azaria, Kevin Dunn, Michael Lerner, Arabella Field, Vicki Lewis, Doug Savant, Harry Shearer
The next time someone tells you that irradiated food is harmless, send him to this movie. And if he doesn't get around to seeing it, no problem: you can be sure there will be a "Godzilla 2," or more accurately a "Godzilla 24," within two years. Though the well-known leapin' lizard never gained the popularity here of E.T., he has been the subject of over a score of films, mostly Japanese. They're the pictures which those of us lucky to be of a certain age thrilled to as early as the 1954 Japanese production "Gojira" directed by Terry Morse and starring Ishiro Honda and Takashi Shimura, with Raymond Burr later added as an insert for the American crowds. To no-one's surprise, critics agreed that the special effects were the star in that one. Nothing's changed, nor does TriStar attempt to market the current version as a character-driven story, unless, you of course, you consider a big green guy to be a character. Come to think of it, Godzilla himself is about the only believable character in the picture. He's certainly larger than life and acts better than Matthew Broderick, also to no one's surprise.
There's much to admire here, however. For one thing there's the fact that you can blame the overgrown lizard on French and not American nuclear testing. For another, there's an attempt at humor (there aren't many) by featuring Roger Ebert lookalike Michael Lerner as Mayor Ebert, a grandiose political animal whose comeuppance arrives near the end when his campaign chief gives him the big thumbs down sign. The fact that a good deal of Manhattan Island is destroyed is either good or bad depending on your geopolitical nature, though to appease real New Yorkers, Madison Square Garden comes in for total destruction as well. Yet another plus is the presence of Jean Reno, allegedly a French insurance investigator who, along with Godzilla, is the only credible performer--one whose antipathy toward American coffee and American doughnuts holds water and proves that two things in France are better than what we've got over here.
Filmed in Hawaii to stand in for French Polynesia and also on location in New Jersey, New York and California, "Godzilla" opens on a French nuclear test, whose mushroom cloud is looked upon with fascination by three cute lizzies. (Why only one mutates into the big beast is anyone's guess.) Shortly thereafter, research scientist Dr. Niko Tatopoulos (Matthew Broderick), aka "the worm guy," is whisked away by State Department personnel from his Ukraine project investigating fat earthworms in Chernobyl to advise a group of military planners, politicians and scholars on a sushi-loving creature that has wreaked havoc with a Japanese fishing boat in Polynesia. He soon meets up with Audrey (Maria Pitillo), a former sweetheart who walked out on him eight years before, and with her two friends Luci (Arabella Field) and Victor (Hank Azaria), who work for a TV news station bossed by Charles Caiman (Harry Shearer). Advising the military that it's better to have the creature come to them than to try to dig him out of his New York hiding place, he gathers up thousands of fish as bait from the Fulton fish market and awaits the return. His reappearance provides just what the audience has been waiting for: the destructive force of a dragon-like heathen who, when he a tail unfolds, sweeps out one hundred windows on an upper floor of an office building and whose mighty claw crushes many of those who do not have a starring role in the movie. Meanwhile, Mayor Ebert has probably given up on the idea that anyone in his town will adopt the creature. After all, where would you get a pooper- scooper that's the right size?
Though the company claims to have re-invented the Godzilla legend by use of cutting-edge technology, the picture does not look a whole lot different from the one the Japanese made forty-four years ago. For good or evil, the kitsch is gone. Godzilla does look almost real, thanks to modern CGI (computer graphics imagery) that's de rigueur with high-tech companies that want you to buy a new model each year, but when he growls and moans, we feel kind of sorry for him. It's an E.T. thing. The guy didn't ask to be born like this and you can't blame him for leaving French territory where he would have been condemned--as Woody "Love and Death" Allen would say--to spend his life eating croissants and food with rich sauces. He did not travel by foot all the way to Manhattan to see the movie "Godzilla," nor was he much interested in Knicks games despite his visit to Madison Square Garden. And after all, he was probably aiming to get lawyers, not innocent people, when he swished out those windows around the Flatiron building and coaxed the military to bomb the Chrysler building.
Like Beethoven's Fifth, "Godzilla" has more than one ending. In the most dramatic, the lizard stages a one-day strike against a taxi, showing how easy it is to catch a cab in Manhattan if you're assertive enough.
To show sympathy or animosity toward the creature, huge audiences will surely throng to the 7,000+ screens in the U.S. for its opening on May 20, folks who might well overlook the competing "Bulworth" making its nationwide debut two days later. Never mind that Mayor Ebert cannot match wits in the slightest with Senator Jay Billington Bulworth--the best Hizzoner can do is to scream that evacuated New Yorkers should be let back into the city even while the monster is taking his paces in midtown.
Matthew Broderick, who shows empathy for Godzilla despite his frequent, frightened, wide-eyed gaze each time the monster approaches, is still Ferris Bueller--the boy next door you'd love to set up with your adorable, curly-haired blond daughter. The only human show-stopper here is Jean Reno as French agent Philippe Roache, still donning that four-day beard, whom you will recognize from his dazzling role in Luc Besson's "The Professional." He's the man who puts an artistic touch on the movie by forcing director Roland Emmerich to insert French subtitles. He does a mean job with a southern American accent as well.
Maybe New York, the country's most liberal big city, should rethink its mission to allow visitors in without regard to size, color, or manner of birth. Rated PG-13. Running Time: 139 minutes. (C) 1998 Harvey Karten
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


Hi there, I am a student on Earth. I am currently working on this blog - blue-dot.github.io - for school. I am also working on various other things, which may be revieled in the blog posts.
Click here to see my scratch page Enjoy!


4 references to Done
System.Management.Automation (4)
engine\MinishellParameterBinderController.cs (1)
330
serializer.
Done
();
engine\NativeCommandProcessor.cs (1)
2137
_xmlSerializer.
Done
();
engine\remoting\common\fragmentor.cs (1)
1084
serializer.
Done
();
engine\serialization.cs (1)
117
serializer.
Done
();


Members
Organization
Space
Organizations Loading...
Spaces Loading...
Member
Org Manager
Org Billing Manager
Org Auditor
Delete
Members of organization loading...
Member
Space Manager
Space Developer
Space Auditor
Members of space loading...
Cancel


@* * Copyright 2023 HM Revenue & Customs * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. *@ @import controllers.authenticated.routes._ @import models.Mode @import uk.gov.hmrc.govukfrontend.views.html.components.FormWithCSRF @this( layout: views.LayoutProvider, formHelper: FormWithCSRF, errorSummary: playComponents.error_summary, inputYesNo: playComponents.input_yes_no, heading: playComponents.heading, submitButton: playComponents.submit_button ) @(form: Form[_], mode: Mode, employers: Seq[String])(implicit request: Request[_], messages: Messages) @layout( pageTitle = s"${errorPrefix(form)} ${messages("yourEmployer.title")}" ) { @formHelper(action = YourEmployerController.onSubmit(), 'autoComplete -> "off") { @errorSummary(form.errors) @heading(messages("yourEmployer.heading"))


{% extends 'base.html' %} {% block content %} {% include 'navbars/navbar_prob_ampl.html' %}
Suppose that \(N(j)\) represents the number of people that are \(j\) years old in a room. We cannot look inside the room but instead can call people out one at a time. After asking them what their ages are we can plot the following histogram:
ages = [14,15,16,22,24,25]
frequency = [1,1,3,2,2,5]
plot(x=ages,y=frequency,vbars="True")
Note for instance that \(N(17)\) is zero while we see that: \(N(14) = 1\), \(N(15) = 1\), \(N(16) = 3\), \(N(22) = 2\), \(N(24) = 2\), and \(N(25) = 5\) The total number of people in a room is therefore: \[N = \sum_{j=0}^{\infty} N(j)\]
We can then ask what is the probaility \(P(j)\) of selecting one person of age \(j\). Which in general is: \[P(j) = \frac{N(j)}{N}\] In particular, the sum of *all* probabilities is 1. We are certain to get someone of some age: \[ \sum_{j=1}^{\infty} P(j) = 1\]
With this code snippet we show that the total probabillity does really sum to 1.
data = zip(ages,frequency)
N = sum(frequency)
sum = 0
for age, freq in data:
sum += freq/N
data = {(22, 2),(25, 5),(16, 3),(15, 1),(14, 1),(24, 2)} N = 14 sum = 0 ... sum = 1
If we want to think about what is the most probable age, we need to think about the age value that we expect to see when we call someone out from the room. In quantum mechanics we denote this via the "bra" \(⟨\) and "ket" \(⟩\) symbols around our variable of interest: \[ ⟨j⟩ \] In general, we will write the 'expectation' value of \(j\) as: \[ ⟨j⟩ = \frac{\sum j N(j)}{N} = \sum_{j=0}^{\infty} j P(j) \] Notice that this expression can give us values that are not included in our original dataset. While this is numerically the same as the average value, in the context of quantum mechanics this is termed the expectation value. It is mathematically equivalent to a weighted average. We can extend this expression to also ask: what is the average of the squares of the age? We can write this as: \[ ⟨j^2⟩ = \frac{\sum j^2 N(j)}{N} = \sum_{j=0}^{\infty} j^2 P(j) \] In general, we can also ask this question for any function of \(j\): \[ ⟨f(j)⟩ = \sum_{j=0}^{\infty} f(j) P(j) \]
With this code snippet we calculate the expectation value of ages and the square of the ages.
E_j = 0
for age, freq in data:
E_j = E_j + (age)*freq/N
E_j2 = 0
for age, freq in data:
E_j2 = E_j2 + (age**2)*freq/N
E_j = 0 ... E_j = 21.0 E_j2 = 0 ... E_j2 =<PHONE_NUMBER>7142856
It is also important to ask how a distribution looks. Take for example these two distributions:
x1, y1 = [4,5,6], [1,7,1]
x2, y2 = [1,2,3,4,5,6,7,8,9], [1,1,1,1,2,1,1,1,1]
plot(x=x1,y=y1,vbars="True")
plot(x=x2,y=y2,vbars="True")
By visual inspection, we can determine that the graphs are both centered around the same value. Thus, it is reasonable to say that we expect to obtain the same average. We can measure the spread of each distribution with the variance, which we can write using our special notation as: \[ \sigma^2 = ⟨j^2⟩ - ⟨j⟩^2 \] This seems really simple because we can just calculate \(⟨j⟩^2\) and then \(⟨j^2⟩\) using the formulas from before and use it for the standard deviation: \[ \sigma = \sqrt{⟨j^2⟩ - ⟨j⟩^2} \]
With this code snippet we calculate the standard deviation of the two distributions.
N1, N2 = sum(y1), sum(y2)
E1_j = 0
for x,y in zip(x1,y1):
E1_j = E1_j + (x)*y/N1
E2_j = 0
for x,y in zip(x2,y2):
E2_j = E2_j + (x)*y/N2
E1_j2 = 0
for x,y in zip(x1,y1):
E1_j2 = E1_j2 + (x**2)*y/N1
E2_j2 = 0
for x,y in zip(x2,y2):
E2_j2 = E2_j2 + (x**2)*y/N2
sigma_1 = sqrt(E1_j2 - E1_j**2)
sigma_2 = sqrt(E2_j2 - E2_j**2)
N1, N2 = 9, 10 E1_j = 0 ... E1_j = 5.0 E2_j = 0 ... E2_j = 5.0 E1_j2 = 0 ... E1_j2 = 25.222222 E2_j2 = 0 ... E2_j2 = 31 sigma_1 = 0.47140452079103085 sigma_2 = 2.449489742783178
It is simple enough to generalize what we have learned to the continous case. What if we ask:
To be able to answer these questions we need to refer to infinitesimal intervals: The probability that the chosen value is between \(x\) and \(x+dx\) is given by \(\rho(x)dx\)
The continous distribution is made from infinitesimally small \(dx\) elements that make up the line representing the function \(f(x)\). The important thing to note here is that this analogy extends to the other components of the distribution. The function \(\rho(x)\) is termed the probability density. The probability that \(x\) lies between \(a\) and \(b\) is thus given by the integral of \(\rho(x)\) for the interval \([a,b]\) \[ P_{[a,b]} = \int_a^b \rho(x) dx \]
Our investigation from the previous section is also applicable in this section in an straightforward way. The discrete summations are integrals and the discrete values that are summed become infinitesimal intervals. \[\int_{-\infty}^\infty \rho(x) dx = 1, ⟨x⟩ = \int_{-\infty}^\infty x \rho(x) dx, ⟨f(x)⟩ = \int_{-\infty}^\infty f(x) \rho(x) dx\]
We have to be careful to understand the units of \(\rho(x)\). If we look at the integral: \[\int_{-\infty}^\infty \rho(x) dx = 1 \] We notice that the integration leads to a unitless quantity. Thus, we can deduce that \( \rho(x) dx \) must have no units. If \(dx\) has units of \(distance\) for example, then \(\rho(x)\) must have units of \(\frac{1}{distance}\)
We have to be careful of the fact that \(\int_{-\infty}^\infty \rho(x) dx = 1\). This means that not every function that we can come up with can be a probability density function. In fact, this statement is a statement of normalization. It is analogous to the fact that the sum of all the probabilites in a discrete problem should add up to one. Quantum mechanics enforces the normalization of the porbability density functions. So we have to learn to normalize functions
References:


Happy Hacktober to everyone, I'm Francesco Mistri and want to deliver a message to the world:
You can find me
here
.
Backend is better
Go back to home
Happy Hacking
Dario Ulin
.


Hey there!
Bravo for trying to stay safe from monsters by disabling
javascript, but you're going to need to turn it on to use this.