Indexers, Cache and Varnish

Update by schedule, a warm full page cache and Varnish in front are what decide storefront speed before your code matters.

Syntaxbin/magento indexer:set-mode {realtime|schedule} [indexer]

Three things outside your module decide how fast a storefront answers: whether the indexes are built, whether the page is cached, and whether something in front of Magento is serving that cache.

Indexers

An indexer copies data that is expensive to read — prices, category membership, stock — into tables shaped for reading. Each indexer has a mode. Update on Save rebuilds the index inside the request that changed the record, so saving a product in the admin blocks until it finishes. Update by Schedule writes changed IDs into a changelog table named after the index with a _cl suffix, and cron reindexes from it. The component that reads those changelogs is called mview.

Production wants schedule mode: bin/magento indexer:set-mode schedule. That command creates MySQL triggers, and triggers only see changes made after they exist, so run bin/magento indexer:reindex once immediately afterwards.

Cache and Varnish

The full_page cache stores rendered HTML. Magento can serve it itself, or hand the job to Varnish, which answers without starting PHP. Magento describes each page's contents in an X-Magento-Tags response header, then sends purge requests by tag when an entity changes. Those requests go to the hosts listed in http_cache_hosts in app/etc/env.php.

The cache nobody warmed

A flush empties the cache for everyone. The next visitor to each URL pays full price. In developer mode the response header X-Magento-Cache-Debug: MISS says so plainly. Flush before traffic arrives, not during it, and request your busiest URLs yourself afterwards.

Example

Example · php
<?php
// app/etc/env.php (fragment; the real file also holds db, crypt and install keys)
return [
    'http_cache_hosts' => [
        [
            'host' => 'varnish-01.internal',
            'port' => 6081,
        ],
        [
            'host' => 'varnish-02.internal',
            'port' => 6081,
        ],
    ],
    'cache_types' => [
        'config' => 1,
        'layout' => 1,
        'block_html' => 1,
        'collections' => 1,
        'full_page' => 1,
    ],
];

When to use it

  • Stop admin product saves from taking thirty seconds on a large catalog.
  • Work out why a price change never reaches the storefront.
  • Put Varnish in front of a storefront and have invalidation actually work.
  • Decide what to do in the ten minutes after a full cache flush on a busy day.

More examples

Checking and fixing indexer modes

indexer:status is the first command to run on a storefront that feels slow when an admin saves anything. Anything reading Save in the Update On column rebuilds inside the save request. Switching one indexer at a time and reindexing it straight away is safer than switching all of them, because the reindex of a large catalog takes minutes and the storefront serves whatever the index last held while it runs.

Example · bash
bin/magento indexer:status

# Title                                     Status       Update On   Schedule
# Design Config Grid                        Ready        Save        
# Customer Grid                             Ready        Save        
# Category Products                         Ready        Schedule    idle
# Product Categories                        Reindex req  Save        
# Catalog Rule Product                      Ready        Schedule    idle
# Product Price                             Ready        Schedule    idle
# Catalog Search                            Ready        Schedule    idle

bin/magento indexer:set-mode schedule catalog_category_product
bin/magento indexer:reindex catalog_category_product
bin/magento cache:flush

Generating a VCL for Varnish

Magento writes the VCL rather than you, because the file has to agree with Magento about the health-check path, the cookies that make a page private, and which hosts may send purge requests. --access-list is that purge allow-list; leave your web nodes out of it and invalidation is silently rejected. The backend host and port are where Varnish forwards a miss, so they point at the web node, not at Varnish itself. After installing the file, reload Varnish and set Stores > Configuration > Advanced > System > Full Page Cache to Varnish, then run bin/magento cache:flush.

Example · bash
bin/magento varnish:vcl:generate \
  --access-list=localhost,10.0.1.0/24 \
  --backend-host=web-01.internal \
  --backend-port=8080 \
  --grace-period=300 \
  --export-version=6 \
  --output-file=/etc/varnish/default.vcl

# VCL file was successfully exported to /etc/varnish/default.vcl

The command abbreviations and the full names

The console accepts any abbreviation that matches one command, so c:f and cache:flush are the same call. The short forms are everywhere in other people's notes and runbooks, which is the reason to recognise them. Write the long form in scripts: adding a module can introduce a second command starting with the same letters, and the abbreviation then fails with an ambiguity error instead of running. Note also that cache:clean deletes every entry belonging to the named enabled Magento cache types and leaves anything else in the storage untouched, while cache:flush purges the whole cache storage, including entries other applications share with it.

Example · bash
# Shorthand: unambiguous abbreviations of each namespace segment.
bin/magento c:f
bin/magento c:c layout block_html
bin/magento i:rei

# Longhand: exactly the same commands.
bin/magento cache:flush
bin/magento cache:clean layout block_html
bin/magento indexer:reindex

Related

m2 indexer modesm2 indexer operatem2 fpcm2 varnishm2 cache types

Discussion

  • Be the first to comment on this lesson.