Learn Docker by Building a Node / Express App

Docker has rapidly become the de facto standard for packaging and deploying modern applications. According to DataDog‘s 2018 Docker Adoption Report, adoption of Docker increased to 24% in 2018, up from 20% the previous year ^1. A 2019 survey by StackRox found that 87% of respondents were using containers in production ^2.

So why has Docker seen such explosive growth in popularity? The answer lies in the benefits of containerization:

1. Consistency – Containers package an application with all its dependencies, ensuring it runs reliably across different environments
2. Portability – Containers are lightweight and portable across different OS and infrastructure
3. Efficiency – Containers use OS-level virtualization and share the host kernel, making them more resource-efficient than virtual machines
4. Scalability – Containers allow for easy horizontal scaling of applications

In this in-depth tutorial, we‘ll walk through how to leverage Docker to containerize a Node.js and Express web application. We‘ll start with a basic single-container setup, then gradually level up the complexity by integrating a MongoDB database and a Redis cache for session management.

Setting Up a Simple Node/Express App

First, let‘s create a basic "Hello World" Express app. Make a new directory for the project and add the following files:

// app.js
const express = require(‘express‘);
const app = express();
const port = 3000;

app.get(‘/‘, (req, res) => {
  res.send(‘Hello World!‘);
})

app.listen(port, () => {
  console.log(`Example app listening at http://localhost:${port}`);
})

// package.json
{
  "name": "hello-docker",  
  "version": "1.0.0",
  "description": "",
  "main": "app.js",
  "scripts": {
    "start": "node app.js"
  },
  "dependencies": {    
    "express": "^4.17.1"
  }
}

Next, let‘s add a Dockerfile to define our application container:

# Dockerfile
FROM node:14-alpine

WORKDIR /app

COPY package*.json ./

RUN npm install --only=production

COPY . .

EXPOSE 3000

CMD ["npm", "start"]

A few best practices to note in this Dockerfile:

• Use an Alpine-based Node image to minimize container size. The full Node 14 image is 944MB vs 117MB for Alpine variant ^3.
• Only install production dependencies with npm install --only=production
• Copy package.json and package-lock.json separately to leverage Docker‘s build cache
• Use the JSON array form of CMD to avoid shell string munging ^4

We can now build and run the container:

$ docker build -t hello-docker .
$ docker run -p 3000:3000 hello-docker  

Navigate to http://localhost:3000 in your browser and you should see "Hello World!"

Integrating MongoDB

In a real-world application, you‘ll likely need to integrate a database. Let‘s add MongoDB to our app using a separate container.

First, update app.js to connect to MongoDB:

const express = require(‘express‘); 
const app = express();
const port = 3000;
const mongoose = require(‘mongoose‘);

mongoose.connect(‘mongodb://mongo:27017/mydb‘, {
  useNewUrlParser: true,
  useUnifiedTopology: true
});

app.get(‘/‘, (req, res) => {
  res.send(‘Hello MongoDB!‘);
})

app.listen(port, () => {
  console.log(`App listening at http://localhost:${port}`); 
})

Notice that we‘re connecting to a MongoDB server at mongodb://mongo:27017. "mongo" is the hostname we‘ll give to our MongoDB container.

Next, create a docker-compose.yml file to define our multi-container setup:

version: "3"
services:
  web:
    build: .
    ports:
      - "3000:3000"
  mongo:
    image: mongo
    ports:
      - "27017:27017"

This docker-compose file defines two services: "web" (our Node app) and "mongo" (the MongoDB database). We specify that the web service should be built from the current directory, while the mongo service uses the official mongo image pulled from Docker Hub.

Now we can start both containers with a single command:

$ docker-compose up  

Our app container can communicate with the MongoDB container using the hostname "mongo". This works because Docker Compose automatically sets up a single network for our app where each container can refer to the others by their service names.

Adding Redis for Session Handling

To demonstrate further how to work with multiple containers, let‘s add session management to our app using Redis.

Update the app.js file again:

const express = require(‘express‘); 
const app = express();
const session = require(‘express-session‘);
const redis = require(‘redis‘);
const redisStore = require(‘connect-redis‘)(session);
const mongoose = require(‘mongoose‘);

const redisClient = redis.createClient({ 
  host: ‘redis‘,
  port: 6379  
});

app.use(
  session({
    store: new redisStore({ client: redisClient }), 
    secret: "my-secret",
    resave: false, 
    saveUninitialized: true
  })  
);

mongoose.connect(‘mongodb://mongo:27017/mydb‘, {
  useNewUrlParser: true,
  useUnifiedTopology: true  
});

app.get(‘/‘, (req, res) => {
  req.session.myVar = req.session.myVar || 0;
  req.session.myVar++;  
  res.send(`Hello, you have visited this page ${req.session.myVar} times`);  
});

app.listen(3000, () => {
  console.log(‘Server started on port 3000‘);
});

We‘ve added express-session to handle sessions, connect-redis to store session data in Redis, and the redis client library to connect to our Redis instance.

Notice that the Redis hostname is set to simply ‘redis‘. Similar to the MongoDB setup, we‘ll add a redis service in docker-compose.yml:

version: "3"
services:  
  web:
    build: . 
    ports:
      - "3000:3000"
  mongo:
    image: mongo
    ports:
      - "27017:27017"
  redis:
    image: redis 
    ports:
      - "6379:6379"

Rebuild the containers and test it out:

$ docker-compose up --build

Each time you refresh the page, the visit counter should increment, demonstrating that sessions are being stored in Redis.

Managing Secrets

One thing we glossed over in the Redis example is how to securely manage sensitive information like database passwords. Hardcoding secrets in Dockerfiles or application code is a big anti-pattern.

There are a few different approaches to solving this in the Docker ecosystem:

1. Docker secrets – With Docker swarm mode, you can use the built-in secrets management system. Secrets are only available to services that have been explicitly granted access. The values are mounted into the container filesystem at /run/secrets/ ^5.

2. Environment variables – A simple approach is to pass secrets via environment variables. You can set these with the docker run -e flag or in docker-compose files. Be aware though that environment variable values are visible in the output of docker inspect.

3. Mounting secrets from the host – Another option is to store secrets in files on the Docker host, then mount these files into the container at runtime. This approach works well when using a secrets management tool like Hashicorp Vault.

As a general rule, aim to keep application code free of secrets. Specify them at deploy time with orchestration tooling. And always use strong, randomly generated secrets!

Orchestration: Swarm vs Kubernetes

We briefly touched on using Docker Swarm for production deployments. Swarm is the built-in orchestration engine for Docker. Some key features:

• Secure by default
• Has a straightforward, easy to use API
• Integrates secrets management, rolling updates, service discovery

However, Kubernetes has emerged as the more dominant orchestration platform. According to a 2019 CNCF survey, 78% of respondents were using Kubernetes vs 23% for Swarm ^6. Some advantages of Kubernetes:

• Has a large, vibrant open source community
• More flexible; supports more configuration options
• Better suited for very large scale deployments

So which should you choose? If you‘re looking for a batteries-included solution that‘s simple to set up and administer, Swarm may be the better option. If you need flexibility and power for a large deployment, Kubernetes is likely the way to go. Many organizations use both in different contexts.

Monitoring Containerized Apps

Monitoring is crucial for running containerized applications reliably in production. Some key metrics to watch:

• Container resource usage (CPU, memory, disk I/O)
• Application performance (response times, error rates)
• Number of running containers for each service

Docker includes some built-in tools for monitoring, like docker stats to view a live stream of container resource usage. However for production use, you‘ll likely want a more robust tool.

Some popular open-source options for monitoring Docker deployments:

• Prometheus (pull-based metrics gathering)
• Grafana (metrics visualization)
• cAdvisor (agent for gathering container metrics)
• Jaeger (distributed tracing)

On the commercial side, DataDog, New Relic, and AppDynamics all have strong container monitoring offerings.

The key is to have observability into both the infrastructure layer (container and host metrics) and the application layer (traces, logs). By combining data from both, you can quickly troubleshoot issues and understand how application changes impact infrastructure and vice versa.

Lessons From The Trenches

To wrap up, I‘ll share a few war stories and takeaways from my experience using Docker for Node.js/Express apps in production:

1. Dockerfile discipline is critical – It‘s easy for Dockerfiles to become bloated, leading to slow build times and large image sizes. Vigilantly apply best practices like leveraging the build cache, minimizing the number of layers, and cleaning up temporary files. Use multi-stage builds where appropriate.

2. Be mindful of memory usage – Node.js apps can be memory hungry, and it‘s easy to hit container memory limits. Use a tool like node-memwatch to profile memory usage and track down leaks. Set appropriate memory limits on containers to avoid runaway processes impacting other containers on the same host.

3. Use health checks – Configure a health check command in your Dockerfile that accurately reflects whether your application is functioning properly. This allows Docker to automatically restart unhealthy containers. I‘ve been burnt in the past by health checks that returned false positives while the application was actually broken.

4. Implement proper logging – Node.js apps running in Docker containers should write all logs to stdout/stderr, rather than to files. This allows Docker to capture and centralize application logs. Make use of 3rd party logging modules like Winston or Bunyan for more advanced log handling.

5. Leverage Docker events – Docker provides an API for subscribing to a stream of events from the daemon. This can be extremely useful for reacting to situations like containers crashing/restarting. I once built a tool to automatically Slack message the team when a production container died.

Hopefully this deep dive has given you a solid foundation for Dockerizing Node.js/Express apps. As with all things, there‘s no substitute for hands-on practice. Go forth and containerize!