Detecting Attacks in API Traffic

The signals that separate an attack from a busy Tuesday, and how to alert on them without drowning.

Prevention fails eventually. Detection decides whether that becomes an incident you handled or a breach you announce months later.

The highest-signal indicators

SignalMeans
Spike in 403s from one actorenumeration or privilege probing
Sequential object idsBOLA scanning — legitimate clients do not walk an id space
SQL syntax errorsinjection probing; normal traffic produces almost none
Spike in 401s across many accountscredential stuffing
A new user agent at volumea new integration, or a scraper
Requests to unknown pathsscanning, or an API you forgot
Volume 20× an account's own baselinea compromised credential or a runaway script
Access at an unusual hour for that accountworth a look, not a block

Baseline per entity, not globally

A global threshold either misses your quiet accounts or constantly flags your busy ones. Compare an account against its own history.

Alert on what you will act on

An alert nobody investigates is worse than no alert, because it trains people to ignore the channel. Every alert needs a documented response — if the answer is "we would look at it and do nothing", make it a dashboard.

Detect the outcome, not only the attempt

Attempts are noisy and constant. Unusual data volume leaving your API is the signal that matters, and it catches attacks whose method you never anticipated.

Canaries

A record that no legitimate query returns, or an API key that is never used. Any access to either is unambiguous — no false positives, no tuning, and it fires on techniques you did not think of.

Example

Example · bash
# The single most valuable query in an API SIEM

# Actors generating many ownership denials — BOLA enumeration
event=authz.denied reason=not_owner
| stats count by actorId, span=5m
| where count > 20

# And the version that catches slow, patient scanning
event=authz.denied reason=not_owner
| stats dc(targetId) as distinct_targets by actorId, span=1h
| where distinct_targets > 50

# A patient attacker stays under any per-minute threshold. Distinct targets
# over an hour catches them; request rate does not.

When to use it

  • A canary invoice record is accessed, producing a zero-false-positive alert that reveals an ongoing enumeration.
  • An account exporting twenty times its own baseline volume is flagged, catching a compromised API key.
  • A spike in SQL syntax errors identifies injection probing hours before any payload succeeds.

More examples

Detection rules that fire on real attacks

Deduplicating alerts per actor per hour is what keeps the channel usable — without it, one enumeration produces hundreds of pages and the next real alert is ignored.

Example · javascript
// Each rule: what it catches, and what you do about it.
export const DETECTION_RULES = [
  {
    name: 'bola_enumeration',
    // Distinct targets over an hour — catches patient scanning that any
    // per-minute rate threshold would miss.
    query: `event=authz.denied reason=not_owner
            | stats dc(targetId) as targets by actorId span=1h
            | where targets > 50`,
    severity: 'high',
    response: 'Suspend the API key, notify the account owner, review what they DID access.',
  },
  {
    name: 'sequential_id_scanning',
    // Legitimate clients do not walk an id space.
    detect: async (window) => {
      const byActor = await groupTargetsByActor(window);
      return Object.entries(byActor)
        .filter(([, ids]) => consecutiveRun(ids.map(Number).sort((a, b) => a - b)) > 10)
        .map(([actorId]) => ({ actorId }));
    },
    severity: 'high',
    response: 'Block the actor, then check which requests SUCCEEDED.',
  },
  {
    name: 'credential_stuffing',
    // Many accounts, few IPs, high failure rate
    query: `event=auth.login.failure
            | stats dc(actorId) as accounts, count as attempts by ip span=10m
            | where accounts > 20 and attempts > 100`,
    severity: 'high',
    response: 'Rate limit the ASN, require CAPTCHA, notify targeted accounts.',
  },
  {
    name: 'injection_probing',
    // Normal traffic produces essentially zero SQL syntax errors.
    query: `error_code IN ("42601","42P01","22P02") | stats count by actorId span=5m
            | where count > 3`,
    severity: 'critical',
    response: 'Block immediately. Review the queries. Assume probing succeeded somewhere.',
  },
  {
    name: 'volume_anomaly',
    // Per-entity baseline, not a global threshold.
    detect: async () => {
      const anomalies = [];
      for (const actor of await activeActors('1h')) {
        const now = await requestCount(actor, '1h');
        const baseline = await medianHourlyCount(actor, '30d');
        if (baseline > 10 && now > baseline * 20) {
          anomalies.push({ actorId: actor, now, baseline, ratio: now / baseline });
        }
      }
      return anomalies;
    },
    severity: 'medium',
    response: 'Compare against their normal pattern. Contact the account owner.',
  },
  {
    name: 'data_egress_anomaly',
    // The OUTCOME, not the attempt. Catches methods you did not anticipate.
    query: `event=data.exported
            | stats sum(rowCount) as rows by actorId span=24h
            | where rows > 100000`,
    severity: 'high',
    response: 'Verify with the account owner before assuming it is legitimate.',
  },
];

// Every rule has a documented response. A rule without one becomes an alert
// nobody investigates, which trains the team to ignore the channel.

// Run them, and suppress duplicates so one incident is one page.
export async function runDetections() {
  for (const rule of DETECTION_RULES) {
    const hits = rule.detect ? await rule.detect() : await siem.query(rule.query);
    for (const hit of hits) {
      const key = `alert:${rule.name}:${hit.actorId}`;
      if (await redis.set(key, '1', 'EX', 3600, 'NX')) {      // 1 alert per hour
        await raiseAlert({ rule: rule.name, severity: rule.severity,
                           response: rule.response, detail: hit });
      }
    }
  }
}

Canaries: alerts with no false positives

Responding normally to a canary credential rather than rejecting it buys investigation time — the attacker keeps using a key that tells you exactly what they are doing.

Example · javascript
// A canary is a record or credential that NOTHING legitimate touches. Any
// access is, by construction, an incident. No tuning, no false positives, and
// it fires on techniques you never anticipated.

// ── 1. Canary records, seeded among real data ────────────────────────
export async function seedCanaries() {
  for (const tenant of await allTenants()) {
    await db('invoices').insert({
      id: canaryId(tenant.id),
      tenant_id: tenant.id,
      reference: 'INV-CANARY',
      customer_name: 'Internal Reconciliation',   // plausible, never queried
      total_cents: 133_700,
      is_canary: true,
    });
  }
}

// Every read path checks. If a canary is ever returned, something walked the
// table rather than querying for a specific record.
export function checkCanary(rows, context) {
  const hit = [].concat(rows).find((r) => r?.is_canary);
  if (hit) {
    audit.record({
      event: 'security.canary_accessed',
      actor: context.user, target: { type: 'canary_record', id: hit.id },
      context,
    });
    alertSecurityChannel(
      `🚨 CANARY ACCESSED: ${hit.id} by ${context.user?.id} from ${context.ip}`);
  }
  return rows;
}

// And exclude them from every legitimate response
const rows = await db('invoices').where({ user_id: id, is_canary: false });

// ── 2. Canary credentials ────────────────────────────────────────────
// An API key in a place only an attacker would look: an old config file, a
// staging environment variable, a comment in an internal document.
app.use(async (req, res, next) => {
  const key = req.get('x-api-key');
  if (key && CANARY_KEYS.has(key)) {
    await audit.record({
      event: 'security.canary_credential_used',
      context: { ip: req.ip, userAgent: req.get('user-agent'), requestId: req.id },
      metadata: { path: req.path, keyLocation: CANARY_KEYS.get(key) },
    });
    // This tells you WHERE they found it — which is often the more useful fact.
    await pageOncall(`Canary credential used (planted in: ${CANARY_KEYS.get(key)})`);

    // Respond normally for a while, so they do not know they are detected.
    return res.status(200).json({ data: [] });
  }
  next();
});

// ── 3. Canary tokens in documents ────────────────────────────────────
// A unique URL embedded in an internal document, a database dump, or a config
// backup. A request to it means that artefact left your control.
//   https://canarytokens.org — or host your own:
app.get('/assets/internal/:token.png', async (req, res) => {
  const canary = CANARY_TOKENS.get(req.params.token);
  if (canary) {
    await pageOncall(`Canary token fired: ${canary.description} — ` +
                     `this document is outside our control`);
  }
  res.type('png').send(TRANSPARENT_PIXEL);
});

// The value of canaries: no false positives, no threshold to tune, and they
// detect the technique you did not think of. The cost is an hour of setup.

Turning detection into a dashboard people read

The weekly review is where most of the value lives — real-time alerting catches loud attacks, and the review catches the quiet accumulation of unused keys and forgotten routes.

Example · javascript
// Detection is only useful if someone looks. Make it few, specific, and
// actionable — a wall of graphs is ignored within a week.

export const SECURITY_DASHBOARD = {
  // ── Top row: is something happening RIGHT NOW? ─────────────────────
  headline: [
    { title: 'Authorization denials (1h)',
      query: 'sum(rate(authz_denied[1h]))',
      alertAbove: 100, context: 'baseline is under 10' },
    { title: 'Failed logins (1h)',
      query: 'sum(rate(auth_login_failure[1h]))',
      alertAbove: 500 },
    { title: 'Unknown paths (1h)',
      query: 'sum(rate(api_unknown_path[1h]))',
      alertAbove: 50, context: 'scanning, or an API we forgot' },
    { title: 'Canary hits (24h)',
      query: 'sum(increase(canary_accessed[24h]))',
      alertAbove: 0, context: 'ANY value is an incident' },
  ],

  // ── Second row: who is doing it? ───────────────────────────────────
  actors: [
    { title: 'Top denial generators',
      query: 'topk(10, sum by (actor_id) (rate(authz_denied[1h])))' },
    { title: 'Top request volume vs their own baseline',
      query: 'topk(10, api_request_rate / api_request_baseline)' },
    { title: 'Top data export volume (24h)',
      query: 'topk(10, sum by (actor_id) (increase(rows_exported[24h])))' },
  ],

  // ── Third row: is anything degrading? ──────────────────────────────
  health: [
    { title: 'Rate limiter store errors', query: 'rate(ratelimit_store_error[5m])',
      context: 'if this is non-zero, limiting is degraded' },
    { title: 'Auth store errors', query: 'rate(auth_store_error[5m])' },
    { title: 'Partner schema violations', query: 'rate(partner_schema_violation[1h])',
      context: 'their breaking change, or their compromise' },
    { title: 'Certificate days remaining', query: 'min(cert_expiry_days)',
      alertBelow: 30 },
  ],
};

// ── Weekly review, which is what actually finds things ──────────────
export async function weeklySecurityReview() {
  return {
    newUserAgentsAtVolume: await newUserAgents({ minRequests: 1000, days: 7 }),
    accountsWithUnusualHours: await offHoursAccess({ days: 7 }),
    unknownPathsSustained: await sustainedUnknownPaths({ days: 7 }),
    apiKeysUnusedFor90Days: await staleApiKeys(90),        // revoke them
    routesWithNoTrafficFor30Days: await deadRoutes(30),    // delete them
    deniedActionsByReason: await denialBreakdown({ days: 7 }),
    adminActionsWithoutTicket: await adminActionsMissingTicketRef({ days: 7 }),
  };
}

// The last two lines of the weekly review generate more real risk reduction
// than most alerting: unused keys and dead routes are pure surface with no
// business value.

Discussion

  • Be the first to comment on this lesson.