Scaling & Agent Isolation
Scale Jenkins by adding agents and isolate builds in containers for clean, reproducible environments.
As usage grows, a single Jenkins gets overloaded. Two key techniques keep it healthy: scaling out and isolation.
Scaling out
- Keep the controller for coordination only; run builds on agents.
- Add more agents to increase capacity.
- Use ephemeral agents (Docker or Kubernetes) that spin up per build and disappear after.
Agent isolation
Running each build in a fresh container means no leftover state, consistent tool versions, and no interference between builds. The agent { docker } or Kubernetes plugin makes this automatic.
Don't build on the controller
Set the controller's executor count to 0 in production so it never runs builds directly — that keeps it responsive and secure.
Example
// Each build runs in a throwaway container = perfect isolation
pipeline {
agent {
docker {
image 'node:20-alpine'
args '-v $HOME/.npm:/root/.npm' // cache npm between builds
}
}
stages {
stage('Build') {
steps {
sh 'npm ci'
sh 'npm run build'
}
}
}
}When to use it
- A team replaces a single heavy agent with ephemeral Docker agents so each build gets a clean container and the host stays uncluttered.
- An organization deploys Jenkins agents on Kubernetes so capacity auto-scales from 0 to 50 agents depending on queue depth.
- A build that previously competed for a single agent now runs on a dedicated labeled agent, cutting queue wait time from 20 minutes to seconds.
More examples
Ephemeral Docker agent per pipeline
Runs the entire pipeline in a disposable Maven container that is deleted automatically after the build.
pipeline {
agent {
docker {
image 'maven:3.9-eclipse-temurin-21'
label 'docker'
args '--rm' // remove container after build
}
}
stages {
stage('Build') { steps { sh 'mvn package' } }
}
}Kubernetes pod template agent
Provisions a Kubernetes pod as an ephemeral Jenkins agent using the Kubernetes plugin.
pipeline {
agent {
kubernetes {
yaml """
apiVersion: v1
kind: Pod
spec:
containers:
- name: maven
image: maven:3.9
command: ['cat']
tty: true
"""
defaultContainer 'maven'
}
}
stages {
stage('Build') { steps { sh 'mvn package' } }
}
}Multiple labeled permanent agents
Defines two permanent agents with different labels via JCasC to handle Linux and Windows builds separately.
# JCasC: two permanent agents with different labels
jenkins:
nodes:
- permanent:
name: agent-linux-1
labelString: 'linux docker'
numExecutors: 4
remoteFS: /home/jenkins/agent
launcher:
ssh:
host: 10.0.1.11
credentialsId: agent-ssh-key
- permanent:
name: agent-windows-1
labelString: 'windows dotnet'
numExecutors: 2
remoteFS: C:\\jenkins\\agent
launcher:
ssh:
host: 10.0.1.20
credentialsId: agent-win-ssh-key
Discussion