Swarm scales docker for free

14 Dec 2015 · Four minute read · on Gianluca's blog

Gourmet is a work in progress application that allow you to execute little applications on an isolated environment, it dowloads your manifest and runs it in a container. I start this application to improve my go knowledge and to work with the Docker API I am happy to share my idea and my tests with Swam an easy way to scale this type of application.

Gourmet exposes an HTTP API available at the /project endpoint that accept a JSON request body like:

    "img": "gourmet/php",
    "source": "https://ramdom-your-source.net/gourmet.zip",
    "env": [

During my test I use this php script that send a message on SQS.

Your script has a console entrypoint executables in this path /bin/console and gourmet uses it to run your program.

To integrate it with Docker I used fsouza/go-dockerclient an open source library written in go.

container, err := dr.Docker.CreateContainer(docker.CreateContainerOptions{
        Image:        img,
        Cmd:          []string{"sleep", "1000"},
        WorkingDir:   "/tmp",
        AttachStdout: false,
        AttachStderr: false,
        Env:          envVars,

This is a snippet that can be used to create a new container. With the container started I use the exec feature to extract your source and to run it.

exec, err := dr.Docker.CreateExec(docker.CreateExecOptions{
    Container:    containerId,
    AttachStdin:  true,
    AttachStdout: true,
    AttachStderr: true,
    Tty:          false,
    Cmd:          command,

if err != nil {
    return err;

err = dr.Docker.StartExec(exec.ID, docker.StartExecOptions{
    Detach:      false,
    Tty:         false,
    RawTerminal: true,
    OutputStream: dr.Stream,
    ErrorStream:  dr.Stream,

After each build Gourmet cleans all and destroies the environment.

err := dr.Docker.KillContainer(docker.KillContainerOptions{ID: containerId})
err = dr.Docker.RemoveContainer(docker.RemoveContainerOptions{ID: containerId, RemoveVolumes: true})
if(err != nil) {
    return err;
return nil

At the moment it is gourmet, It could be different hypothetical use cases:

A microservice to work with docker container easily.

I thought about an easy way to scale this application and I found Swarm, it is a native cluster for docker and it seems awesome in first because it is compatibile with the docker api.


A Docker Swarm’s cluster is very easy to setup, I worked on this project vagrant-swarm to create a local environment but the official documentation is easy to follow.

Swarm’s cluster has two actors:

During this example we will work with 1 master and 2 nodes. Build this machine with virtualbox , with another tool, or in cloud is not a problem and install docker.

Into the master pull swarm and create a cluster identifier.

docker pull swarm
docker run --rm swarm create
docker run --name swarm_master -d -p <manager_port>:2375 swarm manage token://<cluster_id>

swarm create returns a cluster_id use them to start the manager and the manager_ip is the ip of your master server.

Now go into the node, because we must do few things.

docker daemon -H tcp:// -H unix:///var/run/docker.sock
docker run -d swarm join --addr=<node_ip:2375> token://<cluster_id>

When cluster_id is the id created in the previous step and the node_id is the ip of your current node. Enter into the master and restart your manager container

docker restart swarm_master

Now we are ready to test if all it’s up.

docker -H tcp:// info

Replace with your master ip if you are in the same server. You’ll wait this type of response

$. sudo docker -H tcp:// info
Containers: 1
Images: 1
Role: primary
Strategy: spread
Filters: health, port, dependency, affinity, constraint
Nodes: 2
  └ Status: Healthy
  └ Containers: 1
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 513.5 MiB
  └ Labels: executiondriver=native-0.2, kernelversion=3.19.0-43-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
  └ Status: Healthy
  └ Containers: 0
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 513.5 MiB
  └ Labels: executiondriver=native-0.2, kernelversion=3.19.0-43-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
CPUs: 1
Total Memory: 513.5 MiB
Name: f5e23167339e

Gourmet is a set of environment variables to create a connection with docker api, in particular this function NewClientFromEnv and the DOCKER_HOST parameter.

Docker Swarm supports the same Docker API in this way gourmet uses more nodes.

$ DOCKER_HOST="tcp://" ./gourmet api
Something weird with this website? Let me know.