Agents, Labels, and Elastic Scaling

Use labels to route work to the right machines, and lean on ephemeral Kubernetes or cloud agents so capacity grows and shrinks with demand.

A single fat controller that also runs builds does not scale and is a security risk. The professional pattern is a lean controller that only schedules, plus ephemeral agents that spin up per build and vanish afterward.

Labels route work

Tag agents with labels — linux, gpu, docker, arm64 — and request them with agent { label 'gpu' }. Labels can be combined (label 'linux && docker') so a stage lands only on a node that satisfies both.

Ephemeral Kubernetes agents

The Kubernetes plugin creates a fresh pod for each build from a podTemplate, runs the pipeline inside it, and deletes it when done. Every build gets a pristine environment, there is no snowflake state to drift, and idle cost is near zero because pods only exist while building.

Keep the controller thin

  • Set the controller's executor count to 0 so no build ever runs on it.
  • Cap concurrency so a burst cannot exhaust the cluster.
  • Use containerized agents so tool versions are declared, not installed by hand.

Example

Example · yaml
# Kubernetes plugin: a pod spec used as an ephemeral agent.
# Jenkins creates this pod per build and tears it down after.
apiVersion: v1
kind: Pod
metadata:
  labels:
    jenkins-agent: build
spec:
  containers:
    - name: node
      image: node:20-alpine
      command: ["sleep"]
      args: ["infinity"]
      resources:
        requests:
          cpu: "500m"
          memory: "512Mi"
        limits:
          cpu: "2"
          memory: "2Gi"
    - name: kaniko          # rootless image builds, no docker daemon
      image: gcr.io/kaniko-project/executor:debug
      command: ["sleep"]
      args: ["infinity"]
  # pods are ephemeral: fresh env every build, zero idle cost
  restartPolicy: Never

When to use it

  • A Kubernetes-native Jenkins deployment auto-provisions a fresh pod agent for each build and terminates it on completion.
  • A company uses cloud agents on AWS EC2 that spin up when the queue depth exceeds 5 and shut down after 30 minutes idle.
  • Build labels like 'gpu', 'windows', and 'high-mem' route specialized workloads to the right agents without manual scheduling.

More examples

Kubernetes pod agent template

Provisions an ephemeral Kubernetes pod with a Maven build container and a Docker-in-Docker sidecar.

Example · groovy
pipeline {
  agent {
    kubernetes {
      yaml """
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: build
    image: maven:3.9-eclipse-temurin-21
    command: [cat]
    tty: true
  - name: docker
    image: docker:24-dind
    securityContext:
      privileged: true
"""
      defaultContainer 'build'
    }
  }
  stages {
    stage('Build') { steps { sh 'mvn package' } }
  }
}

Label-based agent routing

Routes compilation to a Java21 Maven agent and GPU-intensive tests to a labeled GPU agent.

Example · groovy
pipeline {
  agent none
  stages {
    stage('Compile') {
      agent { label 'maven && java21' }
      steps { sh 'mvn compile' }
    }
    stage('GPU Test') {
      agent { label 'gpu' }
      steps { sh 'python3 train_and_test.py' }
    }
  }
}

Cloud agent JCasC config

Configures the Kubernetes cloud plugin via JCasC with ephemeral pod retention and connection settings.

Example · yaml
jenkins:
  clouds:
    - kubernetes:
        name: k8s-cloud
        serverUrl: https://k8s.example.com
        namespace: jenkins-agents
        jenkinsUrl: http://jenkins.jenkins.svc:8080
        connectTimeout: 30
        podRetention: never
        maxRequestsPerHost: 64

Related

jenkins architecturejenkins pro parallel matrix

Discussion

  • Be the first to comment on this lesson.