What Belongs Outside the Request

Work the user waits for stays in the request; work that merely has to happen goes on a queue.

Syntaxenqueue(jobType, payload) -> messageId // returns before the work is done

A queue exists to separate work the user is waiting for from work that merely has to happen. When someone places an order, the HTTP response has to carry the order id. It does not have to carry the confirmation email, the invoice PDF, the thumbnail, or the webhook delivered to three partners. Doing those inline adds their latency to your response and their failures to yours.

The test

Ask what the caller does with the result. If the response cannot be rendered without it, it stays inline. If nobody notices it happening two seconds later, it goes on a queue. The second question is availability: inline work makes checkout as available as the mail provider, the PDF renderer and the partner's endpoint multiplied together. Enqueueing turns a hard dependency into a retryable one, and turns an unbounded wait into a bounded one.

What it costs

The work is now invisible. Nobody gets a 500; the job simply sits in a backlog, so a queue needs depth and age monitoring from the first day it exists. The work is also eventually consistent — the user may refresh before the thumbnail is there — so the interface needs a placeholder or a status endpoint.

What trips people up

Publishing before the transaction commits. The broker is not part of your database transaction, so if the commit then fails, the job still runs and looks up a row that was never written.

Example

Example · javascript
import express from "express";
import { SQSClient, SendMessageCommand } from "@aws-sdk/client-sqs";
import { pool } from "./db.js";

const sqs = new SQSClient({ region: "eu-west-1" });
const QueueUrl = process.env.ORDER_EVENTS_QUEUE_URL;

export const router = express.Router();

router.post("/orders", async (req, res) => {
  const client = await pool.connect();
  let order;
  try {
    await client.query("BEGIN");
    const { rows } = await client.query(
      "INSERT INTO orders (user_id, total_cents) VALUES ($1, $2) RETURNING id",
      [req.user.id, req.body.totalCents]
    );
    order = rows[0];
    // The order is durable before anything is published. Publishing first would
    // announce an order that a failed commit then erases.
    await client.query("COMMIT");
  } catch (err) {
    await client.query("ROLLBACK");
    throw err;
  } finally {
    client.release();
  }

  // Everything the user does not wait for: receipt email, invoice PDF, partner
  // webhooks. One event; each consumer decides what it cares about.
  await sqs.send(new SendMessageCommand({
    QueueUrl,
    MessageBody: JSON.stringify({ type: "order.placed", orderId: order.id, version: 1 }),
  }));
  // Note the remaining gap: if this send fails after the commit, the order exists
  // and the event does not. Closing that gap is the transactional outbox.

  // The response carries only what the client needs for the next screen.
  res.status(201).json({ id: order.id, status: "confirmed" });
});

When to use it

  • Send the confirmation email after signup without the signup request waiting on SMTP.
  • Generate thumbnails and extract metadata after a file upload returns 201.
  • Deliver one domain event to several partner webhooks, each with its own retries.
  • Build a nightly CSV export and email it, without a 90-second HTTP request.

More examples

The same endpoint with the work inline

The version to recognise in a codebase and move away from. Nothing here is wrong in development, where every dependency is fast and nothing fails; it becomes a p99 of several seconds and an endpoint that returns 500 because a partner is down. Reach for this comparison when someone argues a queue is premature: the question is not throughput, it is whose failures are allowed to become yours.

Example · javascript
// Four dependencies inside one request. The response cannot be sent until all four
// have answered, so latency is their sum and availability is their product.
router.post("/orders", async (req, res) => {
  const order = await createOrder(req.user.id, req.body);

  await mailer.send(receiptEmail(order));       // ~400ms, third-party SMTP
  await pdf.render(invoiceTemplate(order));     // ~1200ms, CPU bound
  await thumbnailer.generate(order.items);      // ~800ms
  for (const sub of await subscribers(order)) {
    // Unbounded: one partner whose endpoint hangs holds this request open for the
    // whole socket timeout, and holds a database connection with it.
    await fetch(sub.url, { method: "POST", body: JSON.stringify(order) });
  }

  res.status(201).json(order);
});

Returning 202 when the caller does need the result

The pattern for long work whose result the user genuinely wants: a status resource created synchronously, the work done asynchronously. It also gives you cancellation and a place to record the failure, neither of which a message in flight offers. Use it for exports, bulk imports, video transcodes and anything where the honest answer to "is it done?" is a database row.

Example · javascript
// Some work cannot be hidden: a CSV export the user will download. Do not block
// the request on it. Accept it, return a resource the client can poll, and let the
// job fill that resource in.
router.post("/exports", async (req, res) => {
  const { rows: [job] } = await pool.query(
    "INSERT INTO exports (user_id, state) VALUES ($1, 'pending') RETURNING id",
    [req.user.id]
  );

  await sqs.send(new SendMessageCommand({
    QueueUrl: process.env.EXPORT_QUEUE_URL,
    MessageBody: JSON.stringify({ exportId: job.id }),
  }));

  res
    .status(202)                       // Accepted, not Created: it is not done yet
    .location(`/exports/${job.id}`)    // where the client polls for state
    .json({ id: job.id, state: "pending" });
});

// GET /exports/:id -> { state: "pending" | "ready" | "failed", downloadUrl? }
// The row exists from the start, so "where is my export" is an ordinary query
// rather than a search through logs for a message id.

One event, several independent consumers

Fan-out is the reason queues outlive the latency argument. The decisive detail is one queue per consumer rather than one shared queue: separate backlogs mean a consumer that is broken or slow is isolated, and you can replay one consumer's dead letters without redelivering to the others. Reach for it the second time someone asks to be notified when an order is placed.

Example · javascript
import { SNSClient, PublishCommand } from "@aws-sdk/client-sns";
const sns = new SNSClient({});

// Publishing once to a topic with one queue per consumer keeps the producer
// ignorant of who cares. Adding a fraud check later is a new subscription, not a
// change to checkout.
await sns.send(new PublishCommand({
  TopicArn: process.env.ORDER_EVENTS_TOPIC_ARN,
  Message: JSON.stringify({ type: "order.placed", orderId: order.id, version: 1 }),
  // Subscription filter policies let each queue select the types it wants, so the
  // email queue is not woken by every inventory change.
  MessageAttributes: {
    eventType: { DataType: "String", StringValue: "order.placed" },
  },
}));

// Each consumer gets its own queue, which means its own backlog, its own retry
// policy and its own dead-letter queue. A slow thumbnailer cannot delay receipts.

Related

sd queuessd api asyncsd queue deliverysd queue sizingsd percentiles

Discussion

  • Be the first to comment on this lesson.