Automated Security Testing
What tooling can find, what it cannot, and how to close the gap with tests you write once.
Automated tooling is excellent at pattern-matching and useless at understanding your business. Knowing which is which stops you trusting a clean scan.
What tools find
- SAST — dangerous patterns in code: injection sinks, unsafe regexes, hardcoded secrets. Noisy, and genuinely useful once tuned.
- SCA — known vulnerabilities in dependencies. High value, low effort, run it daily.
- DAST — probes a running application for injection, misconfiguration and known bad paths.
- Secret scanning — credentials in code and history. Enable push protection.
- Fuzzing — malformed input against parsers and decoders.
What no tool finds
Authorization. A scanner cannot know that invoice 1044 belongs to a different user, that this endpoint should be admin-only, or that this field should not be writable. Broken authorization is the dominant cause of API breaches, and it is precisely the category tooling is blind to.
That gap is closed by tests you write, generated from your route inventory so coverage keeps up with the codebase.
Make it a gate, not a report
A scan whose output is a dashboard is a scan nobody reads. Fail the build on high severity, with a documented exception process for the ones you accept.
Test in a real environment
Unit tests pass while configuration diverges. A short post-deploy smoke test against the actual environment catches the feature flag, proxy rule or environment variable that differs in production.
Example
# What each tool covers, honestly
# SAST → injection sinks, unsafe regex, hardcoded secrets
# SCA → known CVEs in dependencies
# DAST → misconfiguration, known-bad paths, reflected injection
# Secrets → credentials in code and git history
# Fuzzing → parser crashes, resource exhaustion
#
# NONE of them find:
# BOLA · BFLA · mass assignment · tenant isolation · business logic abuse
# excessive data exposure · missing resource limits
#
# Which is: the entire top half of the OWASP API Top 10.When to use it
- A generated authorization matrix catches a cross-tenant leak on a new bulk endpoint the day it merges.
- Daily dependency scanning surfaces a critical CVE hours after disclosure rather than at the next quarterly review.
- A post-deploy smoke test catches a staging feature flag that shipped to production and disabled rate limiting.
More examples
A pipeline that blocks rather than reports
Custom rules encoding your own past findings outperform generic rule packs — they know your codebase's conventions and produce far fewer false positives.
# .github/workflows/security.yml
name: security
on:
pull_request:
push: { branches: [main] }
schedule: [{ cron: '0 6 * * *' }] # daily: today's safe package is not
jobs:
static:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 } # full history for secret scanning
# 1. Secrets — in the diff AND in history
- uses: gitleaks/gitleaks-action@v2
env: { GITHUB_TOKEN: '${{ secrets.GITHUB_TOKEN }}' }
# 2. SAST
- uses: returntocorp/semgrep-action@v1
with:
config: >-
p/owasp-top-ten
p/javascript
p/secrets
.semgrep/custom.yml # ← your own rules matter most
# 3. Dependencies
- run: npm ci --ignore-scripts
- run: npm audit --audit-level=high
# 4. Your own greps for what a linter cannot express
- run: ./scripts/security-grep.sh
authorization:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci --ignore-scripts
# 5. THE ONE NO TOOL PROVIDES — generated from the route inventory
- run: npm run test:authz
dynamic:
runs-on: ubuntu-latest
steps:
- run: docker compose up -d && ./scripts/wait-for-healthy.sh
# 6. DAST against the running app, driven by the OpenAPI spec so it
# tests YOUR endpoints rather than guessing at paths.
- uses: zaproxy/action-api-scan@v0
with:
target: 'http://localhost:3000/openapi.json'
rules_file_name: '.zap/rules.tsv'
fail_action: true
# 7. Post-deploy reality check
- run: BASE=http://localhost:3000 ./scripts/authz-smoke.sh
- run: ./scripts/misconfig-scan.sh http://localhost:3000
# ── Custom Semgrep rules, which find more than the generic packs ─────
# .semgrep/custom.yml
rules:
- id: unscoped-find-by-id
languages: [javascript, typescript]
severity: ERROR
message: >-
Lookup by a request-supplied id with no ownership filter. Scope the
query to req.user.id or req.user.tenant.
patterns:
- pattern: $DB.$TABLE.findById(req.params.$ID)
- pattern-not-inside: |
$DB.$TABLE.findOne({..., user_id: ...})
- id: model-serialised-directly
languages: [javascript]
severity: WARNING
message: Use a Serializer view rather than returning a model.
pattern: res.json($MODEL)
- id: jwt-verify-without-options
languages: [javascript]
severity: ERROR
message: jwt.verify must pin algorithms and check issuer and audience.
pattern: jwt.verify($TOKEN, $KEY)Fuzzing the boundary
The no-state-change property catches handlers that write before validating — a class of bug that produces a correct-looking 400 and a corrupted database.
// Fuzzing finds what you did not think to test: crashes, hangs, and unhandled
// types in parsers and validators.
import fc from 'fast-check';
describe('input handling is robust', () => {
it('never crashes on arbitrary JSON', async () => {
await fc.assert(fc.asyncProperty(fc.jsonValue(), async (body) => {
const res = await request(app).post('/api/invoices')
.set('Authorization', `Bearer ${token}`).send(body);
// Any 4xx is fine. A 5xx means an unhandled input reached the code.
expect(res.status).toBeLessThan(500);
}), { numRuns: 1000 });
});
it('handles adversarial strings in every field', async () => {
const nasty = fc.oneof(
fc.string({ maxLength: 100_000 }), // very long
fc.constant(''), // empty
fc.constant('\0\0\0'), // null bytes
fc.constant('../../../etc/passwd'),
fc.constant("'; DROP TABLE users;--"),
fc.constant('<script>alert(1)</script>'),
fc.constant('{{7*7}}'), // template injection
fc.constant('${jndi:ldap://x}'),
fc.constant('\u202e\u0000'), // unicode direction + null
fc.constant('👨👩👧👦'.repeat(1000)), // grapheme clusters
fc.constant('a'.repeat(40) + '!'), // ReDoS trigger
);
await fc.assert(fc.asyncProperty(nasty, async (value) => {
const started = Date.now();
const res = await request(app).patch('/api/users/me')
.set('Authorization', `Bearer ${token}`).send({ name: value });
expect(res.status).toBeLessThan(500);
// A slow response on a short input is the ReDoS signature.
expect(Date.now() - started).toBeLessThan(1000);
}), { numRuns: 500 });
});
it('rejects deeply nested and oversized structures', async () => {
const deep = Array.from({ length: 10_000 })
.reduce((acc) => ({ a: acc }), { a: 1 });
const res = await request(app).post('/api/invoices')
.set('Authorization', `Bearer ${token}`).send(deep);
expect([400, 413]).toContain(res.status); // not a 500, not a hang
});
it('never changes state on a rejected request', async () => {
await fc.assert(fc.asyncProperty(fc.jsonValue(), async (body) => {
const before = await db('invoices').count('* as c').first();
const res = await request(app).post('/api/invoices')
.set('Authorization', `Bearer ${token}`).send(body);
const after = await db('invoices').count('* as c').first();
if (res.status >= 400) expect(after.c).toBe(before.c);
}), { numRuns: 300 });
});
});
// The last property is the most valuable and the least commonly tested:
// a rejected request must not have partially completed.The gap only you can close
Generating the suite from the inventory is the structural point: coverage becomes a property of the system rather than of whoever wrote the last test.
// Generate authorization coverage from the route inventory, so new routes are
// tested the day they merge rather than when someone remembers.
import routes from '../route-inventory.json' with { type: 'json' };
describe('security properties across every route', () => {
// ── 1. Authorization matrix (no tool can do this) ──────────────────
describe('object authorization', () => {
for (const route of routes.filter((r) => r.takesId)) {
it(`${route.method} ${route.path} → 404 for another user's object`, async () => {
const res = await call(route, { as: alice, targeting: bobsObject(route) });
expect(res.status).toBe(404);
});
it(`${route.method} ${route.path} → 404 for another tenant's object`, async () => {
const res = await call(route, { as: alice, targeting: globexObject(route) });
expect(res.status).toBe(404);
});
}
});
// ── 2. Function authorization ──────────────────────────────────────
describe('function authorization', () => {
for (const route of routes.filter((r) => r.adminOnly)) {
it(`${route.method} ${route.path} → 403 for a normal user`, async () => {
expect((await call(route, { as: alice })).status).toBe(403);
});
}
});
// ── 3. Mass assignment ─────────────────────────────────────────────
describe('property authorization', () => {
const FORBIDDEN_FIELDS = ['role', 'isAdmin', 'tenantId', 'userId',
'emailVerified', 'balance', 'createdAt'];
for (const route of routes.filter((r) => ['POST','PATCH','PUT'].includes(r.method))) {
it(`${route.method} ${route.path} ignores privileged fields`, async () => {
const before = await snapshotUser(alice.id);
await call(route, { as: alice,
body: Object.fromEntries(FORBIDDEN_FIELDS.map((f) => [f, 'INJECTED'])) });
expect(await snapshotUser(alice.id)).toEqual(before);
});
}
});
// ── 4. Output shaping ──────────────────────────────────────────────
describe('data exposure', () => {
for (const route of routes.filter((r) => r.method === 'GET')) {
it(`${route.method} ${route.path} exposes no sensitive fields`, async () => {
const res = await call(route, { as: alice });
if (res.status !== 200) return;
expect(findSensitiveFields(res.body)).toEqual([]);
});
}
});
// ── 5. Resource bounds ─────────────────────────────────────────────
describe('resource limits', () => {
for (const route of routes.filter((r) => r.returnsCollection)) {
it(`${route.method} ${route.path} caps the page size`, async () => {
const res = await call(route, { as: alice, query: { limit: 1_000_000 } });
const items = res.body.data ?? res.body;
expect(Array.isArray(items) ? items.length : 0).toBeLessThanOrEqual(100);
});
}
});
});
// This suite is generated, so it grows with the API automatically. That is the
// property that matters — hand-written security tests cover what someone
// remembered on the day, and the codebase moves faster than memory.
Discussion