Design a URL Shortener
Key generation, a 100:1 read-to-write ratio, cache-first redirects, custom aliases, and analytics kept off the hot path.
GET /{key} -> 302 Location: <long_url>A URL shortener maps a short opaque key to a long target URL and redirects to it. It exists because links have to survive being typed, printed and pasted into length-limited fields, and because whoever created the link wants to know how often it was followed.
The shape of the load
The workload is extremely read-heavy: a link is written once and followed many times, and a 100:1 read-to-write ratio is the usual planning figure. That shape decides almost everything. The mapping is immutable once created, so every layer can cache it with a long TTL and invalidation stops being a problem. Reads are single primary-key lookups, so one relational table carries the service until the key space itself becomes the constraint.
Keys
Two approaches. Encode a monotonic counter in base62, which guarantees uniqueness without a read; or generate a random key and retry on a unique-constraint violation. Seven base62 characters give about 3.5 trillion keys (62^7). A raw counter is cheapest but the resulting keys are enumerable, so anyone can walk your whole corpus; randomise inside a leased block, or use random keys and accept the collision check. Custom aliases live in the same namespace as generated keys, so uniqueness has to be a database constraint rather than a check-then-insert in application code.
What trips people up
Writing a click row synchronously turns a 2 ms redirect into a database write per request. Append to a buffered stream or queue and aggregate downstream. And choose the status code deliberately: a 301 is cached by browsers and proxies, so your click count goes flat and you can never repoint the link.
Example
// Redirect path. Reads outnumber writes by roughly 100:1, so this path is
// cache-first and performs no synchronous database write at all.
app.get("/:key", async (req, res) => {
const { key } = req.params; // destructuring shorthand for:
// const key = req.params.key;
// 1. Cache first. The mapping is immutable after creation, so there is no
// invalidation problem and a long TTL is safe.
let target = await redis.get(`u:${key}`);
if (target === null) {
// 2. Miss: one primary-key lookup. No scan, no join.
const row = await db.oneOrNone(
"SELECT long_url, expires_at FROM links WHERE key = $1",
[key]
);
if (!row || (row.expires_at && row.expires_at <= new Date())) {
// Cache the negative result too, or a scanner generates unbounded
// database load with garbage keys.
await redis.set(`u:${key}`, "", "EX", 60);
return res.status(404).send("Unknown link");
}
target = row.long_url;
await redis.set(`u:${key}`, target, "EX", 86400);
}
if (target === "") return res.status(404).send("Unknown link");
// 3. Analytics never block the redirect: append to a buffered stream and
// aggregate downstream. Losing a few events costs less than +20 ms here.
clickStream.push({
key,
ts: Date.now(),
ua: req.get("user-agent"),
referer: req.get("referer") ?? null, // ?? keeps an empty string as-is,
// unlike || which would null it
});
// 302, not 301: a 301 is cached by the client and you stop seeing hits.
res.redirect(302, target);
});When to use it
- Shortening outbound marketing links so one campaign's click-through can be measured per channel.
- Generating stable short links for SMS, where every character counts against the segment limit.
- Letting customers claim vanity aliases on a shared domain without two of them taking the same word.
- Serving a redirect at single-digit milliseconds from a cache while the analytics pipeline runs minutes behind.
More examples
Counter-based keys with leased blocks
Encoding a counter removes collision retries entirely, but a shared counter would be a write on every create. Each instance leases a block of ids, so the hot path never talks to the allocator. Reach for this when you want guaranteed-unique keys and can tolerate ids that are sequential within a block; add the OFFSET so every key is the same length from day one.
const BASE62 =
"0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
// 62**6 = 56,800,235,584, so every encoded id is at least 7 characters.
const OFFSET = 56_800_235_584n;
function toBase62(n) { // n is a BigInt
if (n === 0n) return "0";
let out = "";
while (n > 0n) {
out = BASE62[Number(n % 62n)] + out;
n /= 62n; // shorthand for: n = n / 62n;
}
return out;
}
class KeyAllocator {
constructor(db, blockSize = 1000n) {
this.db = db;
this.blockSize = blockSize;
this.cursor = 0n;
this.end = 0n;
}
async next() {
if (this.cursor >= this.end) await this.lease();
const id = this.cursor;
this.cursor += 1n; // shorthand for: this.cursor = this.cursor + 1n;
return toBase62(id + OFFSET);
}
// One UPDATE ... RETURNING per 1000 keys. Atomic, so two instances can
// never be handed the same block.
async lease() {
const { last } = await this.db.one(
"UPDATE key_blocks SET last = last + $1 WHERE id = 1 RETURNING last",
[this.blockSize.toString()]
);
this.end = BigInt(last);
this.cursor = this.end - this.blockSize;
}
}Custom aliases: let the database decide
Shows the uniqueness race that a SELECT-then-INSERT cannot win, and the reserved-word list that stops someone claiming /api or /login. Use this whenever users pick their own keys: the primary key is the only place where 'is this taken' can be answered correctly under concurrency.
// CREATE TABLE links (
// key TEXT PRIMARY KEY,
// long_url TEXT NOT NULL,
// owner_id BIGINT,
// is_custom BOOLEAN NOT NULL DEFAULT false,
// created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
// expires_at TIMESTAMPTZ
// );
// Routes that must stay reachable, plus anything that looks like one.
const RESERVED = new Set(["api", "admin", "health", "static", "login", "docs"]);
async function createAlias(alias, longUrl, ownerId) {
if (!/^[A-Za-z0-9_-]{3,32}$/.test(alias) || RESERVED.has(alias.toLowerCase())) {
throw new HttpError(400, "invalid alias");
}
try {
// No pre-check. Two concurrent requests for the same alias both pass a
// SELECT and both INSERT; only the constraint can arbitrate.
await db.none(
`INSERT INTO links (key, long_url, owner_id, is_custom)
VALUES ($1, $2, $3, true)`,
[alias, longUrl, ownerId]
);
} catch (e) {
if (e.code === "23505") throw new HttpError(409, "alias taken"); // Postgres unique_violation
throw e;
}
return alias;
}Click aggregation off the request path
The redirect writes nothing; a consumer rolls raw click events into per-link, per-hour counters with a single upserting statement. Reach for this when the product needs counts but not exact ones, and when you want redirect latency to be independent of how popular a link becomes.
// Consumer of the click stream. Batches are rolled up into coarse buckets,
// so a link with 10,000 hits in an hour costs one row, not 10,000.
async function consumeClicks(batch) {
const buckets = new Map();
for (const { key, ts } of batch) {
const hour = new Date(Math.floor(ts / 3_600_000) * 3_600_000);
const id = `${key}|${hour.toISOString()}`;
buckets.set(id, (buckets.get(id) ?? 0) + 1);
}
// Idempotent in shape but not in effect: redelivery double-counts. That is
// normally acceptable for analytics; if it is not, carry a batch id and
// record it in the same transaction.
for (const [id, count] of buckets) {
const [key, hour] = id.split("|");
await db.none(
`INSERT INTO click_counts (key, hour, clicks) VALUES ($1, $2, $3)
ON CONFLICT (key, hour) DO UPDATE SET clicks = click_counts.clicks + $3`,
[key, hour, count]
);
}
}
Discussion