From PM2 to Docker: Cluster Mode

After publishing my previous article in the From PM2 to Docker series, I've received a few questions:

What about clustering? That's very easy with PM2, but how would you do that with Docker?

Can Docker utilise all cores available?

Is Docker also easily scalable?

These are very legitimate questions. After all, cluster mode in PM2 is a commonly used feature in Node.js applications.

This article answers these questions. By the end of it, you will be able to scale an application with Docker in a way that utilises all available CPU cores for maximum performance.

You will also learn the architectural differences between scaling with PM2 and scaling with Docker, and the benefits the latter brings to the table.

Horizontal scaling

For your application to be able to scale horizontally, it has to be stateless and share nothing. Any data that needs to persist must be stored in a stateful backing, typically a database.

To scale an application with Docker, you simply start multiple container instances. Because containers are just processes, you end up with multiple processes of an application. This is somewhat similar to what you get when you use cluster mode in PM2.

The difference with PM2 is that it uses the Node.js cluster module. PM2 creates multiple processes and the cluster module is responsible for distributing incoming traffic to each process. With Docker, distribution of traffic is handled by a load balancer, which we'll talk about in a bit.

A benefit of this approach is that you are not only able to scale on a single server but across multiple servers as well. The cluster module can only distribute traffic on a single machine, whereas a load balancer can distribute traffic to other servers.

To get the maximum server performance and use all available CPU cores (vCPUs), you want to have one container per core. Starting multiple containers of the same application is simple. You just have to give them different names each time you use the docker run command:

# Start four container instances of the same application
docker run -d --name app_1 app
docker run -d --name app_2 app
docker run -d --name app_3 app
docker run -d --name app_4 app

We'll run into an issue if we want to use the same port for all containers:

$ docker run -d -p 80:3000 --name app_1 app
06fbad4394aefeb45ad2fda6007b0cdb1caf15856a2c800fb9c002dba7304896
$ docker run -d -p 80:3000 --name app_2 app
d5e3959defa0d4571de304d6b09498567da8a6a38ac6247adb96911a302172c8
docker: Error response from daemon: driver failed programming external connectivity on endpoint app_2 (d408c39433627b00183bb27897fb5b3ddc05e189d2a94db8096cfd5105364e6b): Bind for 0.0.0.0:80 failed: port is already allocated.

The clue is at the end: Bind for 0.0.0.0:80 failed: port is already allocated.. A port can be assigned to only one container/process at a time. If web traffic comes in on port 80, how do we spread it across all instances?

We would need a process that receives incoming traffic and distributes it among several other processes, that's what a load balancer does.

Load balancing

A load balancer sits in front of your application and routes client requests to all instances of that application. A load balancing algorithm determines how to distribute traffic. The most common load balancing algorithm is round-robin — requests are distributed sequentially among a group of instances. That's the default for most load balancers and it's what the cluster module in Node.js uses for the distribution of traffic.

From all the load balancers out there, Nginx is the most popular in the Node.js community. Nginx can do more than load balancing traffic — it can also terminate SSL encryption and serve static files. Nginx is more efficient at those than Node.js. Shifting that responsibility away from the application frees up resources for handling more client requests.

Nginx configuration goes in a file named nginx.conf. Let's look at an example specific to load balancing. If you want to learn more about Nginx, the official documentation is a great place to start.

# General configuration
user nginx;
worker_processes 1;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
  worker_connections 1024;
}

# Load balancing configuration starts here
http {
  # Define a group of servers named "app" and use the default round-robin distribution algorithm
  upstream app {
    server app_1:3000;
    server app_2:3000;
    server app_3:3000;
    server app_4:3000;
  }

  # Start a proxy server listening on port 80 that proxies traffic to the "app" server group
  server {
    listen 80;

    location / {
      proxy_pass http://app;
    }
  }
}

We define a server group named app using the upstream directive. Inside the directive, we have a server definition for each container instance of our application. The addresses match the names we gave the containers and the port is the same port the Node.js server is listening on.

Below that, we define a proxy server that listens on port 80 and proxies all incoming traffic to the app server group.

While it's not inherently wrong to install Nginx directly on the host system, it's much easier to communicate with other containers if we use Nginx inside a container. Having the entire application stack inside containers also makes it easier to manage collectively using Docker Compose. You'll see how that works in the next section.

Let's use the official Nginx image from Docker Hub to start an Nginx container that will handle the load balancing for your application.

# Start an Nginx container configured for load balancing
docker run -d --name nginx -p 80:80 -v $(pwd)/nginx.conf:/etc/nginx/nginx.conf:ro nginx

We mount our configuration file inside the container using the -v flag. Additionally, we map port 80 on the host to port 80 inside the container. Port 80 on the host is where internet traffic arrives, and port 80 inside the container is what the Nginx proxy server listens to.

Let's confirm all containers are up and running using docker ps:

$ docker ps
CONTAINER ID        IMAGE        COMMAND                  CREATED             STATUS              PORTS                NAMES
0dc2055e0195        app          "docker-entrypoint.s…"   25 hours ago        Up 25 hours                              app_4
dea61045c74e        app          "docker-entrypoint.s…"   25 hours ago        Up 25 hours                              app_3
827a2a7e429b        app          "docker-entrypoint.s…"   25 hours ago        Up 25 hours                              app_2
eb2bd86b0b59        app          "docker-entrypoint.s…"   25 hours ago        Up 25 hours                              app_1
ba33b8db60d7        nginx        "nginx -g 'daemon of…"   25 hours ago        Up 32 minutes       0.0.0.0:80->80/tcp   nginx

That's four app servers and one nginx load balancer listening on port 80. We resolved the port conflict, and traffic is now distributed across all our application instances in a round-robin fashion. Perfect!

Bringing it all together with Docker Compose

Instead of manually starting four containers and one load balancer, you can do it much quicker with a single command:

$ docker-compose up -d --scale app=4
Creating network "playground_default" with the default driver
Creating playground_app_1 ... done
Creating playground_app_2 ... done
Creating playground_app_3 ... done
Creating playground_app_4 ... done
Creating playground_nginx_1 ... done

Docker Compose brings the entire application stack together in one docker-compose.yml configuration file. You define all the services you need — a database, a backend, a frontend, a load balancer, networks, volumes, etc. — and control them as a single unit. Start everything up with docker-compose up, and bring everything down with docker-compose down. That's how easy it is.

Head over to this Github repository to see the docker-compose.yml used in the example above along with a Node.js sample project. Compare with your project to figure out what's missing.