Deploying from a Pipeline

Applying a manifest or updating an image, waiting for the rollout to become healthy, and running migrations as a separate gated step.

Syntaxkubectl rollout status deployment/<name> --namespace <ns> --timeout=180s

Deploying from a pipeline means a job changes what is running in the cluster. There are two ways. kubectl apply -f k8s/ sends the whole desired state from files in the repository. kubectl set image changes only the image of one container. Applying is usually better, because then the repository describes the cluster and nothing drifts.

Waiting is the whole point

Both commands return as soon as the cluster has accepted the change, not when it has worked. A rollout is Kubernetes replacing old pods with new ones a few at a time. kubectl rollout status deployment/orders-api --timeout=180s watches until every new pod is ready; if they are not ready in time it exits with a non-zero status, which fails the job. Without that step the pipeline reports green while the new pods crash and restart in a loop.

A pod becomes ready only when its readiness probe gets an answer. So the deploy job's verdict is decided by a health endpoint in your own code.

Migrations are a separate step

A migration changes the database schema. Do not run migrations from application start-up: every replica would attempt it, and the application's database login would need permission to alter tables. Instead build a migration bundle in CI, a single executable holding the migrations, and run it as its own job before the rollout. Give that job a database login that may change the schema, and put a human approval in front of it.

Example

Example · csharp
using Microsoft.Extensions.Diagnostics.HealthChecks;

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);

// The readiness probe calls this. Until it answers with a success status,
// Kubernetes does not count the pod as ready, and the deploy job waits.
builder.Services.AddHealthChecks()
    .AddCheck("self", () => HealthCheckResult.Healthy("accepting traffic"));

WebApplication app = builder.Build();

app.MapHealthChecks("/health/ready");
app.MapGet("/orders/1", () => new { Id = 1, Total = 42 });

app.Run();

When to use it

  • Fail the deploy job instead of leaving a release half rolled out and unnoticed.
  • Roll back automatically when the new pods never become ready.
  • Apply a schema change as its own job that a named person approves first.
  • Deploy a specific past commit by giving kubectl set image that commit's tag.

More examples

The deploy job

Three steps: change it, wait for it, undo it if waiting failed. kubectl rollout undo returns the Deployment to its previous revision, which is why the previous image needs a tag of its own. Use set image for a fast image-only deploy, and kubectl apply -f k8s/ instead when the manifests themselves changed.

Example · yaml
jobs:
  deploy:
    runs-on: ubuntu-24.04
    needs: image
    environment: production
    permissions:
      contents: read
      id-token: write      # for a short-lived cloud token, not a stored key
    steps:
      - uses: actions/checkout@v7

      # A cloud login step belongs here; it writes the kubeconfig that the
      # kubectl commands below read.

      # Point the running Deployment at the image this commit produced.
      - name: Update the image
        run: |
          kubectl set image deployment/orders-api \
            orders-api=ghcr.io/contoso/orders-api:${{ github.sha }} \
            --namespace orders

      # The step that decides whether the deploy worked.
      - name: Wait for the rollout
        run: |
          kubectl rollout status deployment/orders-api \
            --namespace orders --timeout=180s

      # Only runs when an earlier step in this job failed.
      - name: Roll back
        if: failure()
        run: kubectl rollout undo deployment/orders-api --namespace orders

Migrations behind an approval

The bundle is built by the same run that built the image, so the schema and the code match. The connection string comes from the deployment system's secret store and belongs to a login that may alter the schema, which the application's own login should not be able to do. Use a SQL script with dotnet ef migrations script --idempotent instead when the SQL has to be reviewed before it runs.

Example · yaml
jobs:
  bundle:
    runs-on: ubuntu-24.04
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-dotnet@v6
        with:
          dotnet-version: 10.0.x
      - run: dotnet tool install --global dotnet-ef
      # One executable holding every migration. It needs neither the SDK
      # nor the source code to run later.
      - run: |
          dotnet ef migrations bundle --self-contained \
            --target-runtime linux-x64 \
            --project src/Orders.Api \
            --output artifacts/efbundle
      - uses: actions/upload-artifact@v7
        with:
          name: efbundle
          path: artifacts/efbundle

  migrate:
    runs-on: ubuntu-24.04
    needs: bundle
    # A GitHub environment with required reviewers. The job stops here
    # until a person approves it.
    environment: production-database
    steps:
      - uses: actions/download-artifact@v8
        with:
          name: efbundle
      - run: chmod +x ./efbundle
      - name: Apply migrations
        env:
          # Without this the bundle runs as Development and may read
          # a developer's user secrets.
          ASPNETCORE_ENVIRONMENT: Production
        run: ./efbundle --connection "${{ secrets.MIGRATION_CONNECTION }}"

The probe the rollout waits on

This is the other end of the example above. If the path is wrong, or the endpoint checks a dependency that is down, the pods never report ready and the deploy job fails on its timeout. That is the correct outcome: a failed deploy with the old pods still serving beats a green pipeline and a broken service.

Example · yaml
# k8s/deployment.yaml, the part that decides whether a rollout finishes
spec:
  template:
    spec:
      containers:
        - name: orders-api
          image: ghcr.io/contoso/orders-api:9f3c1ab2d4e5
          ports:
            - containerPort: 8080
          readinessProbe:
            httpGet:
              path: /health/ready
              port: 8080
            periodSeconds: 5
            failureThreshold: 3

Discussion

  • Be the first to comment on this lesson.