Scaling Across Cores

cluster, the SO_REUSEPORT model, and why the platform often does this for you.

The cluster module forks the primary process into multiple workers that share a listening socket, so the OS load-balances incoming connections across them. It is the classic way to use all cores for an I/O-bound HTTP server.

const cluster = require('node:cluster');
const os = require('node:os');

if (cluster.isPrimary) {
  for (let i = 0; i < os.availableParallelism(); i++) cluster.fork();
  cluster.on('exit', (w) => { console.log('worker died, restarting'); cluster.fork(); });
} else {
  require('node:http').createServer((req, res) => res.end('ok')).listen(3000);
}

The honest take

In a container world you often should not use cluster. Running one Node process per container and scaling with replicas (Kubernetes, ECS) gives you the same parallelism plus independent restarts, rolling deploys, and cleaner resource limits. Reach for cluster on bare metal / VMs where you own the whole box and want to fill its cores with one deployable.

Example

// Cluster with graceful, zero-downtime reloads: the primary tells workers
// to stop accepting new connections, drains them, then forks replacements.
const cluster = require('node:cluster');
const os = require('node:os');

if (cluster.isPrimary) {
  const size = os.availableParallelism();
  for (let i = 0; i < size; i++) cluster.fork();

  // Re-fork on unexpected death to keep capacity constant.
  cluster.on('exit', (worker, code, signal) => {
    if (!worker.exitedAfterDisconnect) {
      console.error(`worker ${worker.process.pid} died (${signal || code}); replacing`);
      cluster.fork();
    }
  });

  // Rolling restart on SIGHUP: one worker at a time, no dropped requests.
  process.on('SIGHUP', async () => {
    for (const worker of Object.values(cluster.workers)) {
      worker.disconnect(); // stop new conns, finish in-flight
      await new Promise((r) => worker.once('exit', r));
      cluster.fork();
    }
  });
} else {
  const server = require('node:http').createServer((req, res) => res.end('ok'));
  server.listen(3000);
  // Honor disconnect: close the server, let keep-alive drain.
  process.on('disconnect', () => server.close(() => process.exit(0)));
}

When to use it

A production API server uses the cluster module to fork as many worker processes as CPU cores, multiplying throughput on a multi-core machine.
An ops team uses cluster's master process to detect worker crashes via the `'exit'` event and automatically restart them without downtime.
A zero-downtime deployment script sends `SIGUSR2` to the cluster master, which then replaces workers one by one to apply the new code.

More examples

Fork workers for each CPU core

The classic cluster pattern -- the primary forks one worker per CPU core and automatically restarts any that crash.

const cluster = require('cluster');
const os      = require('os');

if (cluster.isPrimary) {
  const numCPUs = os.cpus().length;
  for (let i = 0; i < numCPUs; i++) cluster.fork();
  cluster.on('exit', (worker) => {
    console.log(`Worker ${worker.process.pid} died`);
    cluster.fork(); // restart automatically
  });
} else {
  require('./server'); // each worker runs the HTTP server
}

Worker inter-process messaging

Shows how the cluster primary sends commands to workers and workers respond with `process.on('message')`.

// primary
for (const id in cluster.workers) {
  cluster.workers[id].send({ cmd: 'reload' });
}

// worker
process.on('message', msg => {
  if (msg.cmd === 'reload') {
    console.log('Reloading config...');
  }
});

Graceful worker restart

Implements a rolling restart triggered by SIGUSR2 -- each worker is replaced with a 2-second stagger to avoid downtime.

if (cluster.isPrimary) {
  const workers = [];
  for (let i = 0; i < 2; i++) workers.push(cluster.fork());

  process.on('SIGUSR2', () => {
    // Rolling restart: replace workers one by one
    workers.forEach((w, i) => {
      setTimeout(() => {
        w.send('shutdown');
        w.disconnect();
        workers[i] = cluster.fork();
      }, i * 2000);
    });
  });
}

Discussion

Be the first to comment on this lesson.