Caching

A cache stores an answer under a key so the next request can skip the work, until its time to live runs out.

Syntaxawait redis.set('user:42:profile', JSON.stringify(user), 'EX', 300)

A cache is a store of answers you have already worked out. The first request for a user's profile reads the database; the next hundred read the copy. You reach for one when the work is slow and the answer does not change every second.

Two places to keep it

A process is one running copy of your program. An in-process cache is a Map inside it, held in that process's own memory: as fast as reading a variable, empty again after a restart, and private to that copy, so two copies of your service can disagree. A shared cache is a separate server, usually Redis, holding keys in memory for every copy to read. All copies then agree and the entries outlive your restart, but each read is a network call that can fail.

Time to live, and the key

TTL is time to live: how long an entry may be used before it is dropped. Redis enforces it, so await redis.set(key, JSON.stringify(user), 'EX', 300) keeps the entry for five minutes. A short TTL is cheap correctness, because wrong data repairs itself.

The key must name everything that changes the answer: user:42:profile, not user. Put the id, the page number, the locale and a version prefix in it, and you can later delete exactly what a write made wrong.

Two fills you will see are the same thing: cache[key] ??= load(key) is shorthand for if (cache[key] === null || cache[key] === undefined) { cache[key] = load(key) }.

What trips people up

A Map hands back the object it stored rather than a copy, so a caller that edits the result edits the cached entry. A Map that nothing ever deletes from is a memory leak with a friendly name.

Example

Try it yourself
Loading editor…
Press Run to execute the code.

When to use it

  • Serve a product page's category list without querying it on every request.
  • Hold a slow dashboard total for sixty seconds so a refresh costs nothing.
  • Cache a third-party API response you are rate limited against.
  • Keep decoded configuration in memory instead of parsing it per request.

More examples

The same cache in Redis, with ioredis

The shape every cached read takes: check, return on a hit, do the work on a miss, write it back with a TTL. Values cross the wire as strings, so JSON.stringify going in and JSON.parse coming out are not optional. Reach for this when several processes must see the same cached answer.

Example · javascript
import Redis from 'ioredis';
import { Pool } from 'pg';

const redis = new Redis(process.env.REDIS_URL);
const pool = new Pool({ connectionString: process.env.DATABASE_URL });

// Read-through: ask the cache, and on a miss fill it before returning.
export async function getUser(id) {
  const key = `v1:user:${id}:profile`; // v1 lets you bust every entry at once
  const cached = await redis.get(key); // a string, or null on a miss
  if (cached !== null) return JSON.parse(cached);

  const { rows } = await pool.query('SELECT id, name, plan FROM users WHERE id = $1', [id]);
  const user = rows[0] ?? null;
  if (user === null) return null; // do not cache "not found" with a long TTL

  // 'EX' takes seconds. SET key value EX 300 is the Redis command behind it.
  await redis.set(key, JSON.stringify(user), 'EX', 300);
  return user;
}

Bounding an in-process cache

An unbounded Map grows until the process runs out of memory, and the crash looks like a leak rather than a cache. A size cap plus oldest-first eviction is enough for per-process caches of hot keys, and it needs no dependency.

Example · javascript
// A Map keeps insertion order, so the first key it yields is the oldest one.
const MAX_ENTRIES = 1000;
const store = new Map();

function put(key, value) {
  store.delete(key);       // re-insert so a refreshed key counts as newest
  store.set(key, value);
  if (store.size > MAX_ENTRIES) {
    const oldest = store.keys().next().value;
    store.delete(oldest);  // evict one, keeping the cache a fixed size
  }
}

put('user:1:profile', { id: 1 });
put('user:2:profile', { id: 2 });
console.log(store.size); // 2

A key for a query with many parameters

Two requests that mean the same query must land on the same key, or the cache never hits. Sorting the keys before hashing removes the accident of object order. Keep the version prefix outside the hash so you can still read the key in redis-cli.

Example · javascript
import { createHash } from 'node:crypto';

// Search filters make an unwieldy key, so hash a canonical form of them.
function searchKey(filters) {
  const canonical = JSON.stringify(
    Object.keys(filters).sort().map((k) => [k, filters[k]]) // sort: stable order
  );
  const digest = createHash('sha256').update(canonical).digest('hex');
  return `v1:search:${digest.slice(0, 16)}`;
}

// Same filters in a different order produce the same key.
console.log(searchKey({ q: 'boots', page: 2 }) === searchKey({ page: 2, q: 'boots' }));

Related

node cache invalidationnode db queriesnode api paginationnode jsoncrypto createhash

Discussion

  • Be the first to comment on this lesson.