Unrestricted Resource Consumption
OWASP API4 — every request should have a bounded worst case, and most endpoints have never been asked what theirs is.
The question for every endpoint: what is the most expensive request someone can make? If the answer is unbounded, you have API4.
What goes unbounded
| Resource | Unbounded because |
|---|---|
| Memory | no page-size cap; a whole table loaded to serialise |
| CPU | a regex with catastrophic backtracking; an image decode |
| Database | a query with no LIMIT; an N+1 over a large collection |
| Disk | uploads with no quota; logs with no rotation |
| Connections | no timeout on an upstream call |
| Money | a third-party API call per request |
The three that catch people
ReDoS. A regular expression with nested quantifiers can take exponential time on a crafted string. /^(a+)+$/ against forty as and a b hangs a CPU core. Validation regexes are the usual location, which is unfortunate because they run on untrusted input by definition.
Zip bombs and decompression. A few hundred kilobytes expands to gigabytes. Bound the decompressed size as you read, not afterwards.
The N+1 that scales with input. A bulk endpoint that loops over a client-supplied array and issues a query per element gives the client a multiplier on your database load.
Bound everything, explicitly
Request size, page size, array length, string length, JSON depth, query timeout, request timeout, upload size, image dimensions, decompressed size, concurrency per user, and the number of external calls per request. Every one of these should have a number, and the number should be in code rather than in someone's head.
Timeouts at every layer
Client, load balancer, application, database and upstream calls. If the database timeout is longer than the request timeout, a cancelled request leaves the query running — and under load those accumulate until the connection pool is gone.
Example
// The question, asked of one endpoint
app.get('/api/invoices', auth, async (req, res) => {
const invoices = await db('invoices').where({ user_id: req.user.id });
res.json(invoices);
});
// Worst case: a user with 4 million invoices.
// → the database returns 4M rows
// → the driver builds 4M objects
// → JSON.stringify allocates a multi-gigabyte string
// → the process runs out of memory and takes every in-flight request with it
//
// One missing LIMIT, and one user can restart your API at will.When to use it
- An endpoint without a page-size cap is used to load four million rows into memory, taking the process down with every in-flight request.
- A validation regex with nested quantifiers hangs a CPU core on a crafted 40-character input.
- A statement timeout longer than the HTTP timeout leaves abandoned queries running until the connection pool is exhausted.
More examples
ReDoS: finding it and avoiding it
Bounding input length before running the regex is the fix to apply everywhere: it converts an exponential worst case into a bounded one without rewriting patterns.
// Catastrophic backtracking: nested quantifiers over overlapping alternatives.
// The engine tries exponentially many ways to match before giving up.
// ❌ Dangerous patterns — all real, all found in validation code
/^(a+)+$/ // classic nested quantifier
/^(\w+\s?)*$/ // "trim and validate" gone wrong
/^([a-zA-Z0-9]+\.)*[a-zA-Z0-9]+$/ // a naive domain check
/^(\d+,)*\d+$/ // a naive CSV check
/(x+x+)+y/
// The cost, measured:
const evil = /^(\w+\s?)*$/;
console.time('redos');
evil.test('a'.repeat(30) + '!'); // ~2 seconds, one core, blocked
console.timeEnd('redos');
// 40 characters: minutes. In Node this blocks the EVENT LOOP — the whole
// process stops serving anyone.
// ── Fix 1: rewrite without nesting ───────────────────────────────────
/^\w+(\s\w+)*$/ // no quantifier inside a quantifier
/^[a-zA-Z0-9.]+$/ // often a character class is enough
// ── Fix 2: bound the input BEFORE the regex ──────────────────────────
if (input.length > 256) throw new BadRequestError('too_long');
if (!pattern.test(input)) throw new BadRequestError('invalid_format');
// Exponential growth on a bounded input is bounded. This is the cheapest fix
// and it should be applied everywhere regardless.
// ── Fix 3: do not use a regex ────────────────────────────────────────
// Email: a regex cannot correctly validate one. Check for a single '@',
// a non-empty local part and a domain with a dot — then send a verification
// email, which is the only real validation.
function looksLikeEmail(value) {
if (typeof value !== 'string' || value.length > 254) return false;
const at = value.indexOf('@');
if (at < 1 || at !== value.lastIndexOf('@')) return false;
const domain = value.slice(at + 1);
return domain.length > 3 && domain.includes('.') && !domain.startsWith('.');
}
// ── Fix 4: scan for them in CI ───────────────────────────────────────
// npx eslint --rule '{"security/detect-unsafe-regex": "error"}' src/
// npx safe-regex-cli src/**/*.js
// ── Fix 5: a timeout, where the runtime supports it ──────────────────
// Node 20+: new RegExp(pattern, flags) has no timeout, but you can run
// untrusted patterns in a worker thread with a kill timer. Never accept a
// user-supplied regex in the main thread.Bounds on every dimension, in one place
The ordered timeout ladder is the part worth memorising — an out-of-order set of timeouts is worse than none, because it hides the failure until the pool drains.
// Every limit written down, in code, with a number.
export const LIMITS = {
request: { bodyBytes: 100 * 1024, jsonDepth: 20, fields: 200, timeoutMs: 30_000 },
pagination: { defaultLimit: 20, maxLimit: 100, maxOffset: 10_000 },
arrays: { maxItems: 100, maxBulkIds: 100 },
strings: { shortMax: 256, longMax: 5000, searchMax: 100 },
upload: { maxBytes: 5 * 1024 * 1024, maxFiles: 5, maxPixels: 50_000_000,
maxDecompressedBytes: 100 * 1024 * 1024 },
database: { statementTimeoutMs: 10_000, poolMax: 20, maxRowsPerQuery: 10_000 },
upstream: { timeoutMs: 5000, maxResponseBytes: 5 * 1024 * 1024, maxRetries: 3 },
concurrency: { perUser: 10, perTenant: 50 },
};
// ── Timeouts must be ORDERED, or they defeat each other ──────────────
// database statement 10s ← shortest: a cancelled request kills the query
// upstream call 5s
// application request 30s
// load balancer 35s
// client 40s ← longest
// If the DB timeout exceeded the request timeout, abandoned queries would
// accumulate and drain the pool.
// Postgres, per connection
await db.raw('SET statement_timeout = ?', [LIMITS.database.statementTimeoutMs]);
await db.raw('SET idle_in_transaction_session_timeout = ?', [30_000]);
// ── Concurrency per user: one client cannot occupy every worker ──────
import pLimit from 'p-limit';
const userLimiters = new Map();
export function concurrencyLimit(req, res, next) {
const key = req.user?.id ?? req.ip;
if (!userLimiters.has(key)) {
userLimiters.set(key, pLimit(LIMITS.concurrency.perUser));
}
const limiter = userLimiters.get(key);
if (limiter.pendingCount > LIMITS.concurrency.perUser * 2) {
return res.status(429).json({ error: 'too_many_concurrent_requests' });
}
limiter(() => new Promise((resolve) => { res.on('finish', resolve); next(); }));
}
// ── Decompression: bound as you read, not afterwards ─────────────────
import { createGunzip } from 'zlib';
export async function safeGunzip(buffer, maxBytes) {
return new Promise((resolve, reject) => {
const chunks = [];
let total = 0;
const gunzip = createGunzip();
gunzip.on('data', (chunk) => {
total += chunk.length;
if (total > maxBytes) { // a 500KB zip can expand to 10GB
gunzip.destroy();
return reject(new BadRequestError('decompressed_too_large'));
}
chunks.push(chunk);
});
gunzip.on('end', () => resolve(Buffer.concat(chunks)));
gunzip.on('error', reject);
gunzip.end(buffer);
});
}The N+1 that the client controls
Counting queries per request in development is one of the highest-value cheap checks available — N+1s are invisible in a test database with ten rows.
// ❌ The client supplies the array, so the client chooses the query count.
app.post('/api/invoices/bulk', auth, async (req, res) => {
const results = [];
for (const id of req.body.ids) { // 50,000 ids?
const invoice = await db('invoices').where({ id, user_id: req.user.id }).first();
const customer = await db('customers').where({ id: invoice.customer_id }).first();
const lines = await db('invoice_lines').where({ invoice_id: id });
results.push({ ...invoice, customer, lines });
}
res.json(results);
});
// 50,000 ids → 150,000 queries → the database is gone.
// ✅ Bound the input, then batch — a fixed number of queries regardless of size.
const bulkSchema = z.object({
ids: z.array(z.string().uuid())
.min(1)
.max(LIMITS.arrays.maxBulkIds), // ← the cap
}).strict();
app.post('/api/invoices/bulk', auth, validate({ body: bulkSchema }),
async (req, res) => {
// Three queries, whatever the array length.
const invoices = await db('invoices')
.whereIn('id', req.body.ids)
.andWhere({ user_id: req.user.id }) // authorization in the query
.limit(LIMITS.arrays.maxBulkIds);
if (!invoices.length) return res.json([]);
const [customers, lines] = await Promise.all([
db('customers').whereIn('id', invoices.map((i) => i.customer_id)),
db('invoice_lines').whereIn('invoice_id', invoices.map((i) => i.id)),
]);
const byCustomer = new Map(customers.map((c) => [c.id, c]));
const byInvoice = groupBy(lines, 'invoice_id');
res.json(invoices.map((i) => ({
...InvoiceSerializer.public(i),
customer: CustomerSerializer.public(byCustomer.get(i.customer_id)),
lines: (byInvoice[i.id] ?? []).map(LineSerializer.public),
})));
});
// ── Detect the pattern before it reaches production ──────────────────
// Count queries per request in development and fail loudly above a threshold.
if (process.env.NODE_ENV !== 'production') {
app.use((req, res, next) => {
req.queryCount = 0;
const off = db.on('query', () => { req.queryCount++; });
res.on('finish', () => {
off();
if (req.queryCount > 25) {
logger.error({ path: req.path, queries: req.queryCount },
'possible N+1 — investigate before shipping');
}
});
next();
});
}
Discussion