Outbound Request Hygiene

Every call your API makes to someone else is a dependency, a trust boundary, and a way for their bad day to become yours.

Inbound security gets the attention. Outbound calls are where a third party's compromise, outage or schema change becomes your incident.

Bound every call

  • Connect and read timeouts. Without them a slow upstream holds your connections until the pool is exhausted and your API stops responding to anyone.
  • Response size limits. Content-Length is a claim; enforce the limit on the stream.
  • Retry budgets. Retrying without a cap turns a partial outage into a self-inflicted denial of service on your dependency.
  • Circuit breakers. Stop calling something that is failing, and fail fast instead of queueing.

Fail closed on security decisions

If a fraud check, a licence check or an authorization service times out, the answer is no. "The provider was down so we let it through" is how a dependency outage becomes an authorization bypass — and it is written during an incident, by someone under pressure.

Treat responses as untrusted input

Validate the schema. Bound the size. Never render it unescaped. Never concatenate it into a query. A vendor compromise should produce a failed request and a log line, not stored XSS.

Pin what you can

Do not follow redirects to hosts you did not intend to call. Verify TLS properly — never disable certificate validation, not even "temporarily" in staging, because that setting always ships. For high-value integrations, pin the certificate or the issuing CA.

Keep credentials out of the request

A shared HTTP client that attaches an Authorization header by default will happily send your credentials to an attacker-chosen URL the moment an SSRF appears. Attach credentials per destination, deliberately.

Example

Example · javascript
// ❌ No timeout: one slow upstream exhausts your connection pool and your
//    API stops answering anyone at all.
const res = await fetch('https://partner.example.com/accounts');

// ✅ Bounded, validated, and not trusted
const res = await fetch('https://partner.example.com/accounts', {
  signal: AbortSignal.timeout(5000),
  redirect: 'error',                          // a 302 could point anywhere
  headers: { Accept: 'application/json' },
});
if (!res.ok) throw new UpstreamError(res.status);
const data = partnerSchema.parse(await res.json());   // untrusted input

When to use it

  • A payment provider's slow response exhausts the connection pool and takes the whole API down, until timeouts and a circuit breaker are added.
  • A fraud-check timeout is changed from fail-open to fail-closed after a review points out it was an authorization bypass during any outage.
  • A compromised vendor returns a hostile payload that is rejected by schema validation instead of being stored and rendered.

More examples

One configured HTTP client for all outbound calls

Recording duration and error reason per host gives you the data to set a sensible circuit-breaker threshold instead of guessing at one.

Example · javascript
import { Agent, fetch as undiciFetch } from 'undici';

// A single place where the bounds are set, so no call can be created without them.
const agent = new Agent({
  connect: { timeout: 3000 },        // TCP + TLS handshake
  bodyTimeout: 10_000,               // time between body chunks
  headersTimeout: 5000,
  keepAliveTimeout: 4000,
  connections: 50,                   // bound the pool per origin
});

const MAX_RESPONSE_BYTES = 5 * 1024 * 1024;

export async function httpJson(url, {
  method = 'GET', body, headers = {}, timeoutMs = 10_000, schema,
} = {}) {
  const started = Date.now();
  let res;

  try {
    res = await undiciFetch(url, {
      method,
      headers: { Accept: 'application/json', ...headers },
      body: body ? JSON.stringify(body) : undefined,
      dispatcher: agent,
      signal: AbortSignal.timeout(timeoutMs),
      redirect: 'error',              // never follow a redirect blindly
    });
  } catch (err) {
    metrics.increment('outbound.error', { host: new URL(url).host,
      reason: err.name === 'TimeoutError' ? 'timeout' : 'network' });
    throw new UpstreamError('upstream_unavailable', { cause: err });
  } finally {
    metrics.histogram('outbound.duration_ms', Date.now() - started,
      { host: new URL(url).host });
  }

  if (!res.ok) throw new UpstreamError(`upstream_status_${res.status}`);

  // Bound the body as it streams — Content-Length is a claim, not a fact.
  const chunks = [];
  let total = 0;
  for await (const chunk of res.body) {
    total += chunk.length;
    if (total > MAX_RESPONSE_BYTES) throw new UpstreamError('response_too_large');
    chunks.push(chunk);
  }

  const parsed = JSON.parse(Buffer.concat(chunks).toString('utf8'));

  // Validate as strictly as you would validate a browser request.
  return schema ? schema.parse(parsed) : parsed;
}

// ⚠️ Never do this, in any environment:
//   new Agent({ connect: { rejectUnauthorized: false } })
//   process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0'
// It disables certificate validation entirely, and it always ships.

Circuit breaker and retry budget

The idempotency key is what makes retrying a payment safe at all — without it, a retry budget is a policy for charging customers multiple times.

Example · javascript
// Retrying a failing dependency without a budget turns their partial outage
// into a full one, for both of you.

class CircuitBreaker {
  #state = 'closed';          // closed → open → half-open → closed
  #failures = 0;
  #openedAt = 0;

  constructor({ threshold = 5, resetMs = 30_000, name }) {
    Object.assign(this, { threshold, resetMs, name });
  }

  async call(fn) {
    if (this.#state === 'open') {
      if (Date.now() - this.#openedAt < this.resetMs) {
        metrics.increment('circuit.rejected', { name: this.name });
        throw new UpstreamError('circuit_open');    // fail FAST, do not queue
      }
      this.#state = 'half-open';                    // allow one probe through
    }

    try {
      const result = await fn();
      this.#failures = 0;
      this.#state = 'closed';
      return result;
    } catch (err) {
      this.#failures++;
      if (this.#state === 'half-open' || this.#failures >= this.threshold) {
        this.#state = 'open';
        this.#openedAt = Date.now();
        logger.error({ name: this.name }, 'circuit opened');
        metrics.increment('circuit.opened', { name: this.name });
      }
      throw err;
    }
  }
}

const paymentsBreaker = new CircuitBreaker({ name: 'payments', threshold: 5 });

// Retries: bounded, jittered, and ONLY for idempotent operations.
async function withRetry(fn, { attempts = 3, baseMs = 200 } = {}) {
  let lastError;
  for (let i = 0; i < attempts; i++) {
    try {
      return await fn();
    } catch (err) {
      lastError = err;
      // Never retry a 4xx — the request is wrong, and repeating it will not help.
      if (err.status >= 400 && err.status < 500 && err.status !== 429) throw err;
      if (i === attempts - 1) break;

      const delay = baseMs * 2 ** i + Math.random() * baseMs;   // jitter
      await sleep(delay);
    }
  }
  throw lastError;
}

export const chargeCard = (payload) =>
  paymentsBreaker.call(() => withRetry(() =>
    httpJson('https://payments.example.com/charges', {
      method: 'POST',
      body: payload,
      // Idempotency key: retrying must not charge twice.
      headers: { 'Idempotency-Key': payload.orderId },
      schema: chargeResponseSchema,
    })));

Fail closed on security-relevant calls

Writing the fail-open/fail-closed decision into a table per dependency makes it reviewable — otherwise it is decided implicitly by whoever writes the catch block.

Example · javascript
// Not every dependency failure has the same correct answer. Decide per call,
// in advance, and write down why.

// ── FAIL CLOSED: a security decision ─────────────────────────────────
async function checkFraud(order) {
  try {
    return await fraudService.assess(order, { timeoutMs: 2000 });
  } catch (err) {
    // ❌ return { risk: 'low' };
    //    "The fraud service was down so we approved everything" is a decision
    //    an attacker can trigger by degrading the fraud service.
    metrics.increment('fraud.unavailable');
    throw new ServiceUnavailableError('Cannot process orders right now.');
  }
}

// ── FAIL CLOSED: authorization ───────────────────────────────────────
async function canAccess(user, resource) {
  try {
    return await policyService.check({ user, resource }, { timeoutMs: 1000 });
  } catch {
    return false;                     // deny, always
  }
}

// ── FAIL OPEN: an enrichment that is not a control ───────────────────
async function enrichWithCompanyData(email) {
  try {
    return await clearbit.lookup(email, { timeoutMs: 1500 });
  } catch {
    return null;                      // nice to have; not a gate
  }
}

// ── DEGRADE: a control with a safe conservative fallback ─────────────
async function checkRateLimit(key) {
  try {
    return await redisLimiter.consume(key);
  } catch {
    // Not open, not closed — a conservative local limit.
    metrics.increment('ratelimit.degraded');
    return localLimiter.consume(key, { limit: 10 });
  }
}

// Write the table into the code review checklist:
//
//   Dependency        On failure     Why
//   ───────────────────────────────────────────────────────────────────
//   fraud check       CLOSED (503)   bypassable control
//   authorization     CLOSED (deny)  bypassable control
//   rate limiter      DEGRADE        control, but availability matters
//   feature flags     last known     not a security control
//   enrichment        OPEN (null)    cosmetic
//   payments          CLOSED (503)   never assume success
//
// The failure mode is a design decision. Leaving it to a try/catch written
// during an incident is how bypasses get introduced.

Discussion

  • Be the first to comment on this lesson.