[Guide] Installing Docker Swarm on HypriotOS

Swarm and Machine

This guide will setup a virtual environment with Docker Swarm on Raspberry Pis running HypriotOS. The guide is based on the following hypriot guide.

If you don't know what Docker Machine and Docker Swarm is, take a look at the Docker Machine introduction and the Docker Swarm tutorial or read the documentation for Machine and Swarm.

Create ssh key for cluster

First an SSH-key on the master Raspberry Pi is generated.

$ ssh root@192.168.1.21
$ ssh-keygen -t rsa -C "root@node21"

Afterwards the key is installed on the slaves e.g. node22 as seen below.

$ ssh-copy-id root@192.168.1.22

Now node21 can ssh into node22.
Repeat this procedure for the rest of the slave nodes (23, 24, ...)

Swarm token

The next step is to create a token for the cluster. Still on the master node install bc (a calculator) and generate a token.

$ sudo apt-get install bc
$ export TOKEN=$(for i in $(seq 1 32); do echo -n $(echo "obase=16;$(($RANDOM % 16))" | bc); done; echo)
$ echo $TOKEN

Now $TOKEN exists as an environment variable.
You can add it to a file and source it into your environment in the future if you like.

$ mkdir swarm
$ cd swarm
$ nano tokenenv
$ source tokenenv

In the tokenenv file you write (where '610...' is your token):

export TOKEN=6102302A23718A7353E035CBF88A957D  

Creating a (swarm-)master

$ docker-machine create -d hypriot --swarm --swarm-master --swarm-discovery token://$TOKEN --hypriot-ip-address 192.168.1.21 node21
$ docker-machine  ls

We got some errors during this step because of "Too many retries" and an error with certificates. From Hypriot's comment section it looks as if the Docker Swarm Driver for Hypriot expects an ssh-key in the '~/.ssh' folder.

Creating slaves

For all slave nodes an agent must be started (e.g. node22):

$ docker-machine create -d hypriot --swarm --swarm-discovery token://$TOKEN --hypriot-ip-address 192.168.1.22 node22

This will start a container running a swarm-agent each node (e.g. node22)

$ docker ps
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS              PORTS                  NAMES  
f8ed1ab56fba        hypriot/rpi-swarm:latest    "/swarm join --advert"   19 minutes ago      Up 19 minutes       2375/tcp               swarm-agent  

Using the virtual environment

When all nodes are added you can set your current machine to a swarm instead of a machine (therefore --swarm). This swarm is a virtual environment that can consist of different machines for instance from different cloud vendors and local machines.

$ eval $(docker-machine env --swarm node21)

To illustrate what this virtual environment is, try running the following

$ docker ps

Usually you would see the containers running locally, but now you see all the containers on all your machines. The image below shows the agents from all the machines.

$ docker ps
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS              PORTS                                   NAMES  
e387c3f0ff9d        hypriot/rpi-swarm:latest    "/swarm join --advert"   23 minutes ago      Up 23 minutes       2375/tcp                                node23/swarm-agent  
eecc9736dce1        hypriot/rpi-swarm:latest    "/swarm join --advert"   27 minutes ago      Up 27 minutes       2375/tcp                                node24/swarm-agent  
f8ed1ab56fba        hypriot/rpi-swarm:latest    "/swarm join --advert"   30 minutes ago      Up 30 minutes       2375/tcp                                node22/swarm-agent  
692eb77cf238        hypriot/rpi-swarm:latest    "/swarm join --advert"   49 minutes ago      Up 49 minutes       2375/tcp                                node21/swarm-agent  
7951c27a00b5        hypriot/rpi-swarm:latest    "/swarm manage --tlsv"   49 minutes ago      Up 49 minutes       2375/tcp, 192.168.1.21:3376->3376/tcp   node21/swarm-agent-master  

To illustrate further you can use docker run as usual, but where is your container now located? Let's try running a simple HTTP-server to find out.

$ docker run -d -p 8080:80 hypriot/rpi-busybox-httpd

The result of the docker run command on a swarm starts a container and a docker ps afterwards results in the following:

$ docker ps
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS              PORTS                                   NAMES  
baef270a9733        hypriot/rpi-busybox-httpd   "/bin/busybox httpd -"   10 minutes ago      Up 10 minutes       192.168.1.22:8080->80/tcp               node22/hungry_poincare  
e387c3f0ff9d        hypriot/rpi-swarm:latest    "/swarm join --advert"   33 minutes ago      Up 33 minutes       2375/tcp                                node23/swarm-agent  
eecc9736dce1        hypriot/rpi-swarm:latest    "/swarm join --advert"   37 minutes ago      Up 37 minutes       2375/tcp                                node24/swarm-agent  
f8ed1ab56fba        hypriot/rpi-swarm:latest    "/swarm join --advert"   40 minutes ago      Up 40 minutes       2375/tcp                                node22/swarm-agent  
692eb77cf238        hypriot/rpi-swarm:latest    "/swarm join --advert"   59 minutes ago      Up 59 minutes       2375/tcp                                node21/swarm-agent  
7951c27a00b5        hypriot/rpi-swarm:latest    "/swarm manage --tlsv"   59 minutes ago      Up 59 minutes       2375/tcp, 192.168.1.21:3376->3376/tcp   node21/swarm-agent-master  

It is seen that the container was created on node22.
Repeating the command, in this case, results in a container on node24.

$ docker ps
CONTAINER ID        IMAGE                       COMMAND                  CREATED              STATUS              PORTS                                   NAMES  
0912c25e92b3        hypriot/rpi-busybox-httpd   "/bin/busybox httpd -"   About a minute ago   Up About a minute   192.168.1.24:8080->80/tcp               node24/goofy_varahamihira  
baef270a9733        hypriot/rpi-busybox-httpd   "/bin/busybox httpd -"   12 minutes ago       Up 12 minutes       192.168.1.22:8080->80/tcp               node22/hungry_poincare  
e387c3f0ff9d        hypriot/rpi-swarm:latest    "/swarm join --advert"   35 minutes ago       Up 35 minutes       2375/tcp                                node23/swarm-agent  
eecc9736dce1        hypriot/rpi-swarm:latest    "/swarm join --advert"   39 minutes ago       Up 39 minutes       2375/tcp                                node24/swarm-agent  
f8ed1ab56fba        hypriot/rpi-swarm:latest    "/swarm join --advert"   42 minutes ago       Up 42 minutes       2375/tcp                                node22/swarm-agent  
692eb77cf238        hypriot/rpi-swarm:latest    "/swarm join --advert"   About an hour ago    Up About an hour    2375/tcp                                node21/swarm-agent  
7951c27a00b5        hypriot/rpi-swarm:latest    "/swarm manage --tlsv"   About an hour ago    Up About an hour    2375/tcp, 192.168.1.21:3376->3376/tcp   node21/swarm-agent-master  

There are different ways of scheduling with multiple strategies and filters available.

You can also print information about the swarm.

$ docker info
Containers: 11  
Images: 9  
Role: primary  
Strategy: spread  
Filters: health, port, dependency, affinity, constraint  
Nodes: 4  
 node21: 192.168.1.21:2376
  └ Status: Healthy
  └ Containers: 3
  └ Reserved CPUs: 0 / 4
  └ Reserved Memory: 0 B / 971.8 MiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.1.12-hypriotos-v7+, operatingsystem=Raspbian GNU/Linux 8 (jessie), provider=hypriot, storagedriver=overlay
 node22: 192.168.1.22:2376
  └ Status: Healthy
  └ Containers: 3
  └ Reserved CPUs: 0 / 4
  └ Reserved Memory: 0 B / 971.8 MiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.1.12-hypriotos-v7+, operatingsystem=Raspbian GNU/Linux 8 (jessie), provider=hypriot, storagedriver=overlay
 node23: 192.168.1.23:2376
  └ Status: Healthy
  └ Containers: 2
  └ Reserved CPUs: 0 / 4
  └ Reserved Memory: 0 B / 971.8 MiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.1.12-hypriotos-v7+, operatingsystem=Raspbian GNU/Linux 8 (jessie), provider=hypriot, storagedriver=overlay
 node24: 192.168.1.24:2376
  └ Status: Healthy
  └ Containers: 3
  └ Reserved CPUs: 0 / 4
  └ Reserved Memory: 0 B / 971.8 MiB
  └ Labels: executiondriver=native-0.2, kernelversion=4.1.12-hypriotos-v7+, operatingsystem=Raspbian GNU/Linux 8 (jessie), provider=hypriot, storagedriver=overlay
CPUs: 16  
Total Memory: 3.796 GiB  
Name: 7951c27a00b5  

Now Docker Swarm should be up and running, and thereby this guide is finished.