Reverse Proxies and CDNs

Edge caching explained through the cache key, why invalidation is the hard half, and the problems a CDN will not solve for you.

SyntaxCache-Control: public, max-age=0, s-maxage=600, stale-while-revalidate=60

A reverse proxy sits in front of your application and speaks HTTP on its behalf: TLS termination, compression, routing, and caching. A CDN is the same idea run from hundreds of locations near users, so a cache hit never crosses an ocean and never reaches your origin at all.

The cache key is the whole design

An edge cache stores responses under a key. By default the key is the URL; everything else — which query parameters count, which headers, which cookies — is something you configure. On CloudFront that is the cache policy, and only what the policy lists is part of the key. Forwarding a header to the origin through an origin request policy does not add it to the key, so a response that varies by that header gets served to the next viewer who did not send it.

Invalidation is the hard half

Caching is a one-line header. Getting a wrong object out of a hundred points of presence is the part that goes wrong. Three strategies, in descending order of reliability: content-hashed URLs, so a change produces a new key and nothing needs purging; short s-maxage with stale-while-revalidate, so staleness is bounded and refreshes are off the critical path; and explicit purge, by path or by cache tag.

What it cannot do

A CDN cannot cache a personalised page, cannot make an uncacheable API fast, and cannot fix a slow origin — a cold cache, a purge or a long-tail URL space sends everything straight through to you.

Example

Example · javascript
const express = require("express");
const app = express();

// 1. Content-hashed asset. The hash is in the URL, so the object never needs
// invalidating: a new build is simply a new cache key.
app.get("/static/:hash/:file", (req, res) => {
  res.set("Cache-Control", "public, max-age=31536000, immutable");
  res.sendFile(`/var/www/static/${req.params.file}`);
});

// 2. A page that changes. Shorthand and longhand of the same intent:
//    "public, max-age=600"                 -> 600s in browsers AND at the edge
//    "public, max-age=0, s-maxage=600"     -> 0s in browsers, 600s at the edge
// s-maxage overrides max-age for shared caches only. The second form is
// almost always what you want: a purge can reach the edge, never a browser.
app.get("/p/:id", async (req, res) => {
  const product = await db.product(req.params.id);
  res.set(
    "Cache-Control",
    "public, max-age=0, s-maxage=600, stale-while-revalidate=60"
  );
  // Tag the response so it can later be purged by meaning rather than by URL.
  // CloudFront only reads this header if the distribution has a CacheTagConfig
  // naming it; without that config the header is silently ignored.
  res.set("x-amz-meta-cache-tag", `product:${product.id}, brand:${product.brand}`);
  res.json(product);
});

// 3. Anything per-user must say so explicitly, or an edge will hand one
// account's page to the next visitor.
app.get("/account", requireLogin, (req, res) => {
  res.set("Cache-Control", "private, no-store");
  res.json({ email: req.user.email });
});

When to use it

  • Working out why a logged-in user is seeing another user's cached page.
  • Choosing between hashed asset URLs and explicit purges for a deploy pipeline.
  • Cutting origin bandwidth cost by raising s-maxage on responses that do not really need to be fresh.
  • Diagnosing a low cache hit ratio caused by tracking parameters in the cache key.

More examples

Purging by tag instead of by path

Reach for tags when one logical change touches an unknown set of URLs — a renamed brand, a deleted user's content, a template change. CloudFront allows up to 50 tags per cached object, and tags are ignored entirely on a distribution without CacheTagConfig, which is the usual reason a purge appears to do nothing.

Example · bash
# Path invalidation: you must know every URL the object appears under.
aws cloudfront create-invalidation \
  --distribution-id "$DIST" --paths "/p/123" "/p/123?variant=*"

# Tag invalidation: the origin returned x-amz-meta-cache-tag on these
# responses, and the distribution's CacheTagConfig names that header.
# The '#' prefix marks a tag rather than a path.
aws cloudfront create-invalidation \
  --distribution-id "$DIST" --paths "#brand:acme"

# Invalidation quotas are rates: 150 paths-or-tags per second, and 1 wildcard
# invalidation per second.

Making the cache key explicit

Use an allow-list rather than forwarding everything: each extra key member multiplies the number of stored objects and divides your hit rate, and a tracking parameter like utm_source in the key gives every campaign link a cold cache. Cookies are set to none here, which is also what makes the policy safe to apply to shared pages.

Example · bash
# Everything that legitimately varies the response must be IN the key.
aws cloudfront create-cache-policy --cache-policy-config '{
  "Name": "product-pages",
  "DefaultTTL": 600, "MinTTL": 0, "MaxTTL": 86400,
  "ParametersInCacheKeyAndForwardedToOrigin": {
    "EnableAcceptEncodingGzip": true,
    "EnableAcceptEncodingBrotli": true,
    "HeadersConfig": { "HeaderBehavior": "whitelist",
      "Headers": { "Quantity": 1, "Items": ["CloudFront-Viewer-Country"] } },
    "QueryStringsConfig": { "QueryStringBehavior": "whitelist",
      "QueryStrings": { "Quantity": 1, "Items": ["variant"] } },
    "CookiesConfig": { "CookieBehavior": "none" }
  }
}'

The same caching in a reverse proxy you run yourself

Worth having even behind a CDN: it absorbs the misses the edge sends you and the traffic that bypasses the edge altogether. X-Cache-Status is the first thing to look at when a hit rate is lower than expected, because it tells you whether the key you think you configured is the key being used.

Example · nginx
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=edge:50m
                 max_size=10g inactive=24h;

server {
  location / {
    proxy_cache edge;
    # Spell the key out. The default is $scheme$proxy_host$request_uri, which
    # includes every query parameter, wanted or not.
    proxy_cache_key "$scheme$host$uri$is_args$args";

    proxy_cache_valid 200 301 10m;
    proxy_cache_valid 404 1m;        # cache negatives briefly, or they stampede

    # One request refreshes the entry; everyone else is served the stale copy
    # meanwhile. This is nginx's equivalent of stale-while-revalidate.
    proxy_cache_use_stale updating error timeout http_500 http_502 http_503;
    proxy_cache_lock on;             # collapse a thundering herd into one miss
    add_header X-Cache-Status $upstream_cache_status;  # HIT/MISS/STALE
  }
}

Related

sd load balancerssd invalidationsd cache layerssd object storagesd bandwidth and cost

Discussion

  • Be the first to comment on this lesson.