Cron and Consumers on Deploy

Long-lived workers keep the old release's code after the swap, so stop cron and consumers first and restart them through the symlink.

Syntaxbin/magento cron:remove && pkill -f 'bin/magento queue:consumers:start'

Cron and message queue consumers are long-lived PHP processes. A consumer is a worker started by bin/magento queue:consumers:start that reads messages from a queue and handles them. Cron is the scheduler that runs Magento's jobs, including the one that starts consumers. Both resolve their file paths when the process starts, so a worker launched from the previous release keeps running the previous release's code for as long as it lives.

What goes wrong

A job that began before the migration can be writing to a column the migration has just changed. A consumer holding old code can read a message a new publisher produced, with fields it does not know about. Neither fails loudly. You find out later, from rows that make no sense, on a deploy that otherwise looked clean.

The order

Stop the scheduler first with bin/magento cron:remove, which removes only the lines between Magento's markers in the crontab. Then stop the consumer processes. Then migrate, swap the symlink, and put cron back with bin/magento cron:install.

Respawning

By default the consumers_runner cron job starts consumers again, so killing them while cron is still installed achieves nothing. Each consumer exits after max_messages, 10000 by default, and that exit is the restart you want — as long as the command it restarts from points at the current symlink.

Example

Example · bash
# deploy/workers.sh stop|start - run on the node that owns cron and consumers
set -e

MAGENTO=/var/www/current/bin/magento

case "$1" in
stop)
    # the scheduler goes first, or consumers_runner restarts what you kill
    $MAGENTO cron:remove
    pkill -f 'bin/magento queue:consumers:start' || true
    ;;
start)
    # resolved through the symlink, so this is the NEW release
    $MAGENTO cron:install
    ;;
*)
    echo "usage: workers.sh stop|start" >&2
    exit 2
    ;;
esac

# $ deploy/workers.sh stop
# Magento cron tasks have been removed
#
# $ deploy/workers.sh start
# Crontab has been generated and saved

When to use it

  • Stop workers before a migration so no job runs against a half-changed schema.
  • Restart consumers after a swap so they load the new release's handler code.
  • Supervise consumers with systemd without pinning them to one release directory.
  • Keep the queue quiet for a long deploy without editing the crontab by hand.

More examples

Hold the consumers back from inside env.php

Set cron_run to false when you supervise consumers yourself with systemd or supervisord, and during a deploy when you want the queue to stay quiet without touching the crontab. An empty consumers array means every declared consumer runs. A lower max_messages makes each worker short-lived, which is the cheapest way to guarantee that a worker picks up new code soon after a swap. Changing env.php takes effect on the next process start; run bin/magento cache:flush as well, because the configuration cache holds a copy.

Example · php
<?php
// app/etc/env.php - excerpt; this file is shared, not part of a release
return [
    'cron_consumers_runner' => [
        'cron_run' => false,          // cron no longer starts any consumer
        'max_messages' => 1000,       // default is 10000
        'consumers' => [
            'async.operations.all',
            'product_action_attribute.update',
        ],
        'multiple_processes' => [
            'async.operations.all' => 4,   // Commerce 2.4.4 and later
        ],
    ],
];

Supervise one consumer so a restart lands on the new release

Use a template unit when you want each consumer managed and observable: `systemctl stop 'magento-async@*'` before the swap and `start` after it. The two details that matter are the symlink in ExecStart, and --max-messages, which turns the worker into something that recycles on its own. Without Restart=always the consumer stops for good the first time it finishes its message budget.

Example · ini
# /etc/systemd/system/[email protected]
[Unit]
Description=Magento consumer %i
After=network.target

[Service]
Type=simple
User=www-data

# /var/www/current, never /var/www/releases/20261005T0910
ExecStart=/usr/bin/php8.4 /var/www/current/bin/magento \
    queue:consumers:start %i --single-thread --max-messages=1000

# exits after 1000 messages; systemd starts it again from the symlink
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

Find out what you are actually stopping

Run this on the node before writing the deploy script, because the list depends on which modules are installed and grows when you add your own queue. It is also the answer to 'is this message stuck or is nobody consuming it': a consumer that appears here but has no process is a queue nothing is draining. Pair it with `ps -ef | grep queue:consumers:start` to see which release each running worker came from.

Example · bash
bin/magento queue:consumers:list

# product_action_attribute.update
# product_action_attribute.website.update
# exportProcessor
# inventory.source.items.cleanup
# inventory.mass.update
# inventory.reservations.cleanup
# inventory.reservations.update
# inventory.reservations.updateSalabilityStatus
# media.storage.catalog.image.resize
# codegeneratorProcessor
# sales.rule.update.coupon.usage
# async.operations.all

Related

m2 consumersm2 message queuesm2 cronm2 zero downtimem2 deploy upgrade

Discussion

  • Be the first to comment on this lesson.