Scaling Up

Splitting web, cron and consumer roles, moving cache and sessions to Redis, read replicas, and preparing for a sale day.

Syntaxbin/magento setup:config:set --session-save=redis --session-save-redis-host=HOST --session-save-redis-db=2

One server runs everything, and at some traffic level it stops working. Web requests, cron and queue consumers want different resources at different moments, and they interfere with each other on a shared box.

Splitting the roles

Three roles, one codebase, one app/etc/env.php: web nodes behind a load balancer, one cron node, and one or more consumer nodes. Only the cron node runs the crontab. Consumer nodes run bin/magento queue:consumers:start <name> --max-messages=1000 under a process supervisor that restarts them when they exit, and cron_consumers_runner/cron_run is set to false in env.php so cron does not start a second copy of each consumer.

Shared state leaves the node

Sessions move to Redis under session/save. The default cache and the full page cache get their own Redis databases, conventionally 0 and 1, with sessions on 2. Media goes to shared storage or a CDN, and search to OpenSearch. Magento 2.4.9 added Valkey-specific options to the same command, so --cache-backend=valkey sits alongside --cache-backend=redis.

Read replicas are configured under db/slave_connection, named after the master connection they shadow.

A sale day

Reindex and flush before traffic arrives, then request your landing pages so Varnish holds them. If orders back up, add consumer capacity rather than web nodes: the queue, not PHP, is what is behind.

Example

Example · php
<?php
// app/etc/env.php (fragment; identical on every node)
return [
    'session' => [
        'save' => 'redis',
        'redis' => [
            'host' => 'redis.internal',
            'port' => '6379',
            'database' => '2',
            'disable_locking' => '1',
        ],
    ],
    'cache' => [
        'frontend' => [
            'default' => [
                'backend' => 'Magento\\Framework\\Cache\\Backend\\Redis',
                'backend_options' => [
                    'server' => 'redis.internal',
                    'port' => '6379',
                    'database' => '0',
                ],
                'id_prefix' => 'prod_',
            ],
            'page_cache' => [
                'backend' => 'Magento\\Framework\\Cache\\Backend\\Redis',
                'backend_options' => [
                    'server' => 'redis.internal',
                    'port' => '6379',
                    'database' => '1',
                    'compress_data' => '0',
                ],
                'id_prefix' => 'prod_',
            ],
        ],
    ],
    'cron_consumers_runner' => [
        'cron_run' => false,
        'max_messages' => 1000,
    ],
    'http_cache_hosts' => [
        ['host' => 'varnish.internal', 'port' => 6081],
    ],
];

When to use it

  • Move a single-server store onto separate web, cron and consumer nodes.
  • Stop sessions being lost when the load balancer moves a customer between nodes.
  • Work out whether a backlog of orders needs more consumers or a bigger database.
  • Write the hour-before checklist for a Black Friday deployment.

More examples

The CLI shorthand that writes that file

Each option maps to one env.php path, so the command and the hand edit are interchangeable; the command is safer because it validates the values and will not leave you with a file that fails to parse. Use the CLI when provisioning a node and the hand edit when reviewing what a node actually has. Separate Redis databases matter: flushing the page cache should not drop everyone's session, and sharing a database means one FLUSHDB takes all three. Run bin/magento cache:flush afterwards.

Example · bash
# Shorthand: the CLI writes the env.php keys for you
bin/magento setup:config:set \
  --cache-backend=redis \
  --cache-backend-redis-server=redis.internal \
  --cache-backend-redis-db=0 \
  --page-cache=redis \
  --page-cache-redis-server=redis.internal \
  --page-cache-redis-db=1 \
  --session-save=redis \
  --session-save-redis-host=redis.internal \
  --session-save-redis-db=2 \
  --http-cache-hosts=varnish.internal:6081

# Longhand: edit app/etc/env.php by hand. The two produce the same file.
#   --cache-backend-redis-db   ->  cache/frontend/default/backend_options/database
#   --page-cache-redis-db      ->  cache/frontend/page_cache/backend_options/database
#   --session-save-redis-db    ->  session/redis/database
#   --http-cache-hosts         ->  http_cache_hosts

A consumer under systemd on its own node

--max-messages=1000 makes the process exit after a thousand messages and Restart=always brings it straight back, which is how you stop a long-lived PHP process from leaking memory. Without the limit a consumer runs until it is killed. This only works if cron_consumers_runner/cron_run is false in env.php; leave it true and cron spawns its own consumers alongside these, so two sets of processes compete for the same queues and waste memory and database connections. A single message is still delivered to only one of them, so the risk is duplicated processes rather than duplicated work. queue:consumers:list prints the valid consumer names, which are declared in each module's etc/queue_consumer.xml.

Example · ini
; /etc/systemd/system/[email protected]
[Unit]
Description=Magento queue consumer %i
After=network.target

[Service]
Type=simple
User=www-data
WorkingDirectory=/var/www/magento
ExecStart=/usr/bin/php bin/magento queue:consumers:start %i --max-messages=1000
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

; systemctl enable --now [email protected]
; systemctl enable --now [email protected]
; bin/magento queue:consumers:list   lists the names you can pass as %i

Confirming which node is doing what

bin/magento cron:install writes the crontab entries for you, and the marked block it creates is what cron:remove deletes again. Running it on more than one node is the most common scaling mistake: two nodes both claim schedule rows and jobs run twice or deadlock. The two curl calls are the whole cache-warming idea in miniature. The first pays for generation, the second is served by Varnish; if the second is not an order of magnitude faster, the page is not being cached and nothing else you do on a sale day will help.

Example · bash
# On a web node: cron must not be running here
crontab -l -u www-data
# no crontab for www-data

# On the cron node only
bin/magento cron:install
crontab -l -u www-data | head -3
# #~ MAGENTO START d9e6f3
# * * * * * /usr/bin/php /var/www/magento/bin/magento cron:run 2>&1 | grep -v "Ran jobs by schedule" >> /var/www/magento/var/log/magento.cron.log
# #~ MAGENTO END d9e6f3

# Before a sale, on the cron node
bin/magento indexer:reindex
bin/magento cache:flush
curl -s -o /dev/null -w '%{http_code} %{time_total}\n' https://www.example.com/
# 200 1.812
curl -s -o /dev/null -w '%{http_code} %{time_total}\n' https://www.example.com/
# 200 0.041

Related

m2 cache redism2 scaling infram2 deploy consumersm2 monitoring m2m2 perf mysql

Discussion

  • Be the first to comment on this lesson.