Swarm scales docker for free

14 Dec 2015 · Four minute read · on Gianluca's blog

News! Tech related notes are NOW published to ShippingBytes. See you there! I always felt this was not the right place for me to write consistently about tech and tools. So if you want to read more about that see you at the other side

Gourmet is a work in progress application that allow you to execute little applications on an isolated environment, it dowloads your manifest and runs it in a container. I start this application to improve my go knowledge and to work with the Docker API I am happy to share my idea and my tests with Swam an easy way to scale this type of application.

Gourmet exposes an HTTP API available at the /project endpoint that accept a JSON request body like:

    "img": "gourmet/php",
    "source": "https://ramdom-your-source.net/gourmet.zip",
    "env": [

During my test I use this php script that send a message on SQS.

Your script has a console entrypoint executables in this path /bin/console and gourmet uses it to run your program.

To integrate it with Docker I used fsouza/go-dockerclient an open source library written in go.

container, err := dr.Docker.CreateContainer(docker.CreateContainerOptions{
        Image:        img,
        Cmd:          []string{"sleep", "1000"},
        WorkingDir:   "/tmp",
        AttachStdout: false,
        AttachStderr: false,
        Env:          envVars,

This is a snippet that can be used to create a new container. With the container started I use the exec feature to extract your source and to run it.

exec, err := dr.Docker.CreateExec(docker.CreateExecOptions{
    Container:    containerId,
    AttachStdin:  true,
    AttachStdout: true,
    AttachStderr: true,
    Tty:          false,
    Cmd:          command,

if err != nil {
    return err;

err = dr.Docker.StartExec(exec.ID, docker.StartExecOptions{
    Detach:      false,
    Tty:         false,
    RawTerminal: true,
    OutputStream: dr.Stream,
    ErrorStream:  dr.Stream,

After each build Gourmet cleans all and destroies the environment.

err := dr.Docker.KillContainer(docker.KillContainerOptions{ID: containerId})
err = dr.Docker.RemoveContainer(docker.RemoveContainerOptions{ID: containerId, RemoveVolumes: true})
if(err != nil) {
    return err;
return nil

At the moment it is gourmet, It could be different hypothetical use cases:

A microservice to work with docker container easily.

I thought about an easy way to scale this application and I found Swarm, it is a native cluster for docker and it seems awesome in first because it is compatibile with the docker api.


A Docker Swarm’s cluster is very easy to setup, I worked on this project vagrant-swarm to create a local environment but the official documentation is easy to follow.

Swarm’s cluster has two actors:

During this example we will work with 1 master and 2 nodes. Build this machine with virtualbox , with another tool, or in cloud is not a problem and install docker.

Into the master pull swarm and create a cluster identifier.

docker pull swarm
docker run --rm swarm create
docker run --name swarm_master -d -p <manager_port>:2375 swarm manage token://<cluster_id>

swarm create returns a cluster_id use them to start the manager and the manager_ip is the ip of your master server.

Now go into the node, because we must do few things.

docker daemon -H tcp:// -H unix:///var/run/docker.sock
docker run -d swarm join --addr=<node_ip:2375> token://<cluster_id>

When cluster_id is the id created in the previous step and the node_id is the ip of your current node. Enter into the master and restart your manager container

docker restart swarm_master

Now we are ready to test if all it’s up.

docker -H tcp:// info

Replace with your master ip if you are in the same server. You’ll wait this type of response

$. sudo docker -H tcp:// info
Containers: 1
Images: 1
Role: primary
Strategy: spread
Filters: health, port, dependency, affinity, constraint
Nodes: 2
  └ Status: Healthy
  └ Containers: 1
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 513.5 MiB
  └ Labels: executiondriver=native-0.2, kernelversion=3.19.0-43-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
  └ Status: Healthy
  └ Containers: 0
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 513.5 MiB
  └ Labels: executiondriver=native-0.2, kernelversion=3.19.0-43-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
CPUs: 1
Total Memory: 513.5 MiB
Name: f5e23167339e

Gourmet is a set of environment variables to create a connection with docker api, in particular this function NewClientFromEnv and the DOCKER_HOST parameter.

Docker Swarm supports the same Docker API in this way gourmet uses more nodes.

$ DOCKER_HOST="tcp://" ./gourmet api
Something weird with this website? Let me know.