Design a Payment Flow
Money needs an idempotency key on every hop, an outbox instead of a dual write, and a reconciliation job that assumes the API lied.
stripe.paymentIntents.create(params, { idempotencyKey }) // longhand: the Idempotency-Key: <key> request headerA payment flow is a distributed transaction where you control half the participants. Your service, a payment provider, a card network and eventually a settlement file all hold an opinion about whether a charge happened, and the network between you can fail at the worst possible moment: after the provider charged the card, before the response reached you. Every decision in the design follows from that one ambiguity — a timeout means unknown, never failed.
Idempotency on every hop
Each call that moves money carries a key derived from the business operation, not from the attempt. Stripe reads it from the Idempotency-Key header and replays the first saved response for that key; the stripe-node { idempotencyKey } request option is shorthand for setting that header, and they are the same thing. Your own API owes its callers the same contract, implemented as a unique index on the key plus a fingerprint of the request body, so a retry with different parameters is rejected instead of charging twice.
Outbox instead of a dual write
"Insert the payment row, then publish to the queue" has no atomic form — either step can be the one that survives a crash. Write the event into an outbox table inside the same transaction as the state change, and let a relay publish it: a poller, or change data capture reading the write-ahead log. Delivery becomes at-least-once, which is why every consumer is idempotent too.
What trips people up
Webhooks arrive twice, out of order and signed. Verify the signature, deduplicate on the event id, and refuse any event that moves a payment backwards. Then reconcile daily against the provider's own records, because reconciliation exists to find the rows your code is certain are fine.
Example
// Charge path: one idempotent call out, and our state plus the event we want to
// publish written in a single transaction.
import Stripe from 'stripe';
import { pool } from './db.js';
const stripe = new Stripe(process.env.STRIPE_SECRET_KEY);
export async function chargeOrder({ orderId, amountCents, currency, paymentMethodId }) {
// Derived from the operation, so every retry of THIS charge reuses it. A fresh
// uuid per attempt would be a fresh charge. The :v1 suffix matters: Stripe
// caches the first response for 24h, including a failure, so a legitimate
// change to the request needs a new key version.
const idempotencyKey = `order:${orderId}:charge:v1`;
const intent = await stripe.paymentIntents.create(
{
amount: amountCents, // minor units: 1999 is GBP 19.99
currency,
payment_method: paymentMethodId,
confirm: true,
metadata: { order_id: String(orderId) }, // our id, visible in their dashboard
},
{ idempotencyKey } // sets Idempotency-Key; identical, shorter
);
const client = await pool.connect();
try {
await client.query('BEGIN');
// Unique on provider_intent_id, so re-running this whole function after a
// crash converges instead of inserting a second payment row.
await client.query(
`INSERT INTO payments (order_id, provider_intent_id, status, amount_cents, currency)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (provider_intent_id)
DO UPDATE SET status = EXCLUDED.status, updated_at = now()`,
[orderId, intent.id, intent.status, amountCents, currency]
);
// The outbox row and the payment row commit together or not at all. No
// publish call here: a broker call inside a transaction is the dual write.
await client.query(
`INSERT INTO outbox (aggregate, aggregate_id, type, payload)
VALUES ('order', $1, 'payment.recorded', $2::jsonb)`,
[orderId, JSON.stringify({ orderId, intentId: intent.id, status: intent.status })]
);
await client.query('COMMIT');
} catch (err) {
await client.query('ROLLBACK');
// The charge may well exist at Stripe. Do not surface "payment failed":
// surface "pending" and let reconciliation resolve it.
throw err;
} finally {
client.release();
}
return intent;
}When to use it
- Charging a card from a checkout endpoint that the client will retry after a 504.
- Publishing an order-paid event without a dual write between the database and the message broker.
- Handling a provider webhook that arrives twice, out of order, and sometimes forged.
- Explaining at month end why your recorded revenue differs from the provider's dashboard.
More examples
The outbox relay
Shows why the outbox is cheap: a polling relay is about thirty lines and needs no new infrastructure. Reach for the poller when you already own a relational database and the event rate is modest; swap it for change data capture reading the write-ahead log (PostgreSQL logical decoding, available since 9.4) when polling latency or the dead tuples it generates start to show. Either way the application code only ever does an INSERT.
// The relay is the only thing that publishes. It is allowed to publish the same
// event twice; it is not allowed to lose one or to publish out of order within
// an aggregate.
import { pool } from './db.js';
import { broker } from './broker.js';
export async function drainOutbox(batchSize = 100) {
const client = await pool.connect();
try {
await client.query('BEGIN');
// SKIP LOCKED lets several relay instances run at once without contending.
// Ordering by id preserves per-aggregate order as long as one aggregate's
// events are not split across concurrent batches - hence the FILTER below
// in production: partition by aggregate_id, or run a single relay.
const { rows } = await client.query(
`SELECT id, type, payload
FROM outbox
WHERE published_at IS NULL
ORDER BY id
LIMIT $1
FOR UPDATE SKIP LOCKED`,
[batchSize]
);
if (rows.length === 0) { await client.query('ROLLBACK'); return 0; }
for (const row of rows) {
// Carry the outbox id as the broker-level message id so consumers can
// deduplicate without parsing the payload.
await broker.publish(row.type, row.payload, { messageId: String(row.id) });
}
// If the process dies between publish and this UPDATE, the batch is sent
// again on the next run. At-least-once by construction.
await client.query(
`UPDATE outbox SET published_at = now() WHERE id = ANY($1::bigint[])`,
[rows.map((r) => r.id)]
);
await client.query('COMMIT');
return rows.length;
} catch (err) {
await client.query('ROLLBACK');
throw err;
} finally {
client.release();
}
}Verifying and applying a webhook exactly once
Three separate defences in one handler: signature verification (authenticity plus a replay window), a unique insert (exactly-once effects on top of at-least-once delivery), and a monotonic state guard (order independence). Reach for this shape for any provider webhook. The raw-body requirement is the single most common cause of "signature verification failed" in a working integration.
import express from 'express';
import Stripe from 'stripe';
import { pool } from './db.js';
const stripe = new Stripe(process.env.STRIPE_SECRET_KEY);
const app = express();
// Ranking the states lets us ignore a late event without a timestamp comparison.
const RANK = {
requires_payment_method: 0,
requires_confirmation: 1,
requires_action: 2,
processing: 3,
succeeded: 4,
canceled: 4,
};
app.post(
'/webhooks/stripe',
// The signature covers the exact bytes Stripe sent, so this one route must
// NOT use a JSON body parser. Mounting express.json() globally breaks it.
express.raw({ type: 'application/json' }),
async (req, res) => {
let event;
try {
event = stripe.webhooks.constructEvent(
req.body,
req.headers['stripe-signature'],
process.env.STRIPE_WEBHOOK_SECRET
);
} catch {
// Also throws on a replayed event outside the default 300-second
// tolerance, which is what makes this more than a shared secret check.
return res.status(400).send('invalid signature');
}
// Delivery is at-least-once. The unique index, not the handler, enforces once.
const claim = await pool.query(
`INSERT INTO processed_events (event_id, type) VALUES ($1, $2)
ON CONFLICT (event_id) DO NOTHING
RETURNING event_id`,
[event.id, event.type]
);
if (claim.rowCount === 0) return res.sendStatus(200); // already handled
if (event.type.startsWith('payment_intent.')) {
const intent = event.data.object;
// Events arrive out of order. Never move a payment backwards: a late
// 'processing' must not overwrite a recorded 'succeeded'.
await pool.query(
`UPDATE payments
SET status = $2, updated_at = now()
WHERE provider_intent_id = $1
AND $3 > COALESCE((SELECT rank FROM payment_status_rank WHERE status = payments.status), -1)`,
[intent.id, intent.status, RANK[intent.status] ?? 0]
);
}
// 200 quickly. Slow handlers get retried, which multiplies the load.
res.sendStatus(200);
}
);Nightly reconciliation against the provider
This is the job that catches what idempotency keys cannot: the charge that succeeded at the provider while your transaction rolled back, and the charge your code recorded against an account that never saw it. Reach for it on day one, not after the first discrepancy — the output is also the answer to "why does our revenue number differ from the dashboard" at month end. Settlement files from the bank form a third source to compare against once card volume grows.
// Reconciliation assumes both sides are wrong until proven otherwise. Three
// buckets, and only the first is boring.
import Stripe from 'stripe';
import { pool } from './db.js';
const stripe = new Stripe(process.env.STRIPE_SECRET_KEY);
export async function reconcile(fromUnix, toUnix) {
const theirs = new Map(); // intent id -> amount in minor units
// stripe-node auto-paginates with for await, so a busy day is one loop.
for await (const charge of stripe.charges.list({
created: { gte: fromUnix, lt: toUnix },
limit: 100,
})) {
if (charge.status === 'succeeded') {
theirs.set(charge.payment_intent, charge.amount);
}
}
const { rows } = await pool.query(
`SELECT provider_intent_id, status, amount_cents
FROM payments
WHERE created_at >= to_timestamp($1) AND created_at < to_timestamp($2)`,
[fromUnix, toUnix]
);
const ours = new Map(rows.map((r) => [r.provider_intent_id, r]));
const report = { matched: [], theirsOnly: [], oursOnly: [], amountMismatch: [] };
for (const [intentId, amount] of theirs) {
const mine = ours.get(intentId);
// Charged by them, unknown to us: exactly the timeout case. Either record
// the payment and fulfil, or refund. Never ignore.
if (!mine) { report.theirsOnly.push({ intentId, amount }); continue; }
if (mine.amount_cents !== amount) { report.amountMismatch.push({ intentId }); continue; }
report.matched.push(intentId);
}
for (const [intentId, mine] of ours) {
// We believe it succeeded and they have no record: a bug, or the wrong
// account, or a test key in production. Page someone.
if (mine.status === 'succeeded' && !theirs.has(intentId)) {
report.oursOnly.push({ intentId });
}
}
return report;
}
Discussion