Bulk Endpoints and Partial Success

One request, two hundred items, nineteen bad ones: per-item results, the right status code, and retrying only what failed.

SyntaxPOST /orders/bulk -> 200 { "errors": true, "items": [ { "id": "...", "status": 201 }, ... ] }

A bulk endpoint exists for one reason: 200 round trips cost 200 round-trip times, and over a 60 ms link that is twelve seconds of waiting. The design question is what happens when item 19 is invalid.

All or nothing, or per item

All-or-nothing is one transaction: every item applies or none does, the client resends the whole batch, and the semantics are trivial. That is right when the items form one logical change — a journal entry, a replaced cart. It is wrong for an import, where one bad row would block 199 good ones forever.

Per-item results mean the HTTP status describes the batch, not the items. Elasticsearch's _bulk is the canonical shape: 200 with a top-level errors boolean and an items array carrying each operation's own status and error. The top-level flag matters because a client checking only res.ok otherwise loses data silently. 207 Multi-Status (RFC 4918) is more honest, but enough clients, proxies and dashboards treat anything other than 200 as failure that most public APIs use 200 and document the body.

Retrying only what failed

Require a client-supplied id on every item and echo it in each result. Without it the client cannot map result 19 back to its row, so it resends everything — and now you need idempotency per item, not per request. Cap the batch size and reject oversized batches with 413 rather than timing out halfway through.

Example

Example · javascript
// routes/bulk.js — per-item results, batch-level 200.
const MAX_ITEMS = 200;

router.post('/orders/bulk', async (req, res) => {
  const items = req.body?.items;
  if (!Array.isArray(items) || items.length === 0)
    return res.status(422).json(problem('bulk-items-required', 422));
  if (items.length > MAX_ITEMS)
    // Say no up front rather than timing out at item 900.
    return res.status(413).json(problem('bulk-too-large', 413,
      { detail: `at most ${MAX_ITEMS} items per request` }));

  // Every item carries a client-supplied id so the client can map results back.
  const missing = items.findIndex((i) => typeof i.id !== 'string');
  if (missing !== -1)
    return res.status(422).json(problem('bulk-item-id-required', 422,
      { detail: `items[${missing}].id is required` }));

  const results = [];
  for (const item of items) {
    try {
      // Each item is its own transaction: one failure must not roll back the rest.
      // The item id doubles as the per-item idempotency key.
      const order = await orders.createIdempotent(
        req.principal.accountId, `${req.get('Idempotency-Key')}:${item.id}`, item.body);
      results.push({ id: item.id, status: 201, location: `/orders/${order.id}` });
    } catch (err) {
      const e = err instanceof ApiError ? err : new ApiError(500, 'internal-error');
      results.push({ id: item.id, status: e.status, error: toProblem(e) });
    }
  }

  // The flag a careless client still has a chance of noticing.
  res.status(200).json({
    errors: results.some((r) => r.status >= 400),
    items: results,          // same order as the request
  });
});

When to use it

  • An importer making 5,000 single POSTs that takes an hour because of round-trip latency
  • Deciding whether a batch endpoint should be transactional or report per-item results
  • Letting a client retry the 19 rows that failed instead of resubmitting 200
  • Capping batch size so one request cannot monopolise a worker

More examples

Retrying only the failures

The client-supplied ids are what make this possible: without them the second attempt would have to resend all 200 rows. Note `r.error?.code` — optional chaining, the shorthand for `r.error && r.error.code` — because successful items carry no error object at all.

Example · javascript
async function importOrders(rows, token) {
  // Stable per-row ids: the same row gets the same id on every attempt.
  let pending = rows.map((row, n) => ({ id: `row-${n}`, body: row }));

  for (let attempt = 0; attempt < 4 && pending.length; attempt++) {
    const res = await fetch('/orders/bulk', {
      method: 'POST',
      headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json',
                 'Idempotency-Key': `import-2026-10-02` },
      body: JSON.stringify({ items: pending }),
    });
    if (!res.ok) throw new Error(`batch rejected: ${res.status}`);

    const { errors, items } = await res.json();
    if (!errors) return { imported: rows.length, rejected: [] };   // the flag, checked

    const byId = new Map(pending.map((p) => [p.id, p]));
    // 4xx will never succeed unchanged; 5xx and 429 are worth another attempt.
    const retryable = items.filter((r) => r.status >= 500 || r.status === 429);
    const permanent = items.filter((r) => r.status >= 400 && r.status < 500 && r.status !== 429);

    if (permanent.length) report(permanent.map((r) => ({ row: byId.get(r.id).body,
                                                        code: r.error?.code })));
    pending = retryable.map((r) => byId.get(r.id));
    await sleep(500 * 2 ** attempt + Math.random() * 250);
  }
  return { imported: rows.length - pending.length, rejected: pending };
}

The all-or-nothing variant

This is the simpler contract and the right default when a half-applied batch would be meaningless — a cart holding three of five lines is worse than an unchanged cart. The signal that you have chosen wrong is support tickets about one bad row blocking a whole nightly feed.

Example · javascript
// Use this when the items are one change, not many: a double-entry journal
// where the lines must balance, or a cart whose contents are replaced wholesale.
router.put('/carts/:id/items', async (req, res, next) => {
  const items = req.body.items ?? [];

  try {
    const cart = await db.tx(async (tx) => {
      await tx.query('DELETE FROM cart_items WHERE cart_id = $1', [req.params.id]);
      for (const [n, item] of items.entries()) {
        // Any throw here rolls back everything, including the delete.
        if (!(await tx.oneOrNone('SELECT 1 FROM skus WHERE sku = $1', [item.sku])))
          throw new ApiError(422, 'unknown-sku', { detail: `items[${n}].sku` });
        await tx.query(`INSERT INTO cart_items (cart_id, sku, qty) VALUES ($1,$2,$3)`,
          [req.params.id, item.sku, item.qty]);
      }
      return tx.one('SELECT * FROM carts WHERE id = $1', [req.params.id]);
    });
    res.json(cart);                      // 200: all of it applied
  } catch (err) {
    next(err);                           // 422: none of it applied
  }
});

The response shape, borrowed from _bulk

Each item's error is the same problem+json object a single-item POST would have returned, so a client reuses one error handler for both paths. Elasticsearch's bulk API works this way — HTTP 200 with a top-level `errors` flag and per-item `status` — and its documented pitfall is the same one: code that checks only the HTTP status loses writes without noticing.

Example · http
POST /orders/bulk
{"items": [{"id": "row-0", "body": {...}},
           {"id": "row-1", "body": {"quantity": 0}},
           {"id": "row-2", "body": {...}}]}

HTTP/1.1 200 OK
{
  "errors": true,
  "items": [
    {"id": "row-0", "status": 201, "location": "/orders/9001"},
    {"id": "row-1", "status": 422,
     "error": {"type": ".../problems/validation-failed", "status": 422,
               "code": "validation-failed",
               "errors": [{"pointer": "/quantity", "code": "min"}]}},
    {"id": "row-2", "status": 201, "location": "/orders/9002"}
  ]
}

Related

sd api errorssd api idempotencysd api ratelimitssd api asyncsd queue retries

Discussion

  • Be the first to comment on this lesson.