Protecting Data

Encryption in transit and at rest, what a secrets manager actually buys you, and why deleting PII is a design problem, not a DELETE.

SyntaxGenerateDataKeyCommand({ KeyId, KeySpec: 'AES_256', EncryptionContext }) // => { Plaintext, CiphertextBlob }

"Is it encrypted?" is not one question. In transit, at rest, and reachable by your own application code are three different threat models, and an answer that says "HTTPS, and the database is encrypted" has covered the two easiest ones.

In transit

Public traffic is TLS, with 1.3 (RFC 8446, 2018) as the current version: a one round-trip handshake and no RSA key transport, so recorded traffic stays unreadable after a server key leaks. The decision worth stating is the internal hops. Either you terminate at the edge and trust the private network, or you re-encrypt service to service with mutual TLS. Both are defensible; being vague about which one you chose is not.

At rest

Volume and bucket encryption protects you against a stolen disk and nothing else — your application decrypts transparently, so a SQL injection or a leaked credential reads plaintext either way. When a field genuinely needs more, use envelope encryption: a per-record data key encrypts the field, a key management service encrypts the data key, and you store that wrapped key beside the row.

Secrets

A secrets manager exists so no long-lived credential sits in an image, a repository or a .env on a box. Processes fetch at start, cache briefly and refresh, so rotation is not a deploy. Better still is no stored secret at all: short-lived credentials from a workload identity.

Deleting it

An erasure request under GDPR Article 17 is a distributed problem. The row has copies in replicas, last night's backup, the search index, the warehouse, caches and logs. Design for it up front: keep personal data in one table everything else references by id, or crypto-shred — encrypt each subject under its own key and destroy the key.

Example

Example · javascript
// pii.js — envelope encryption for a column that must stay unreadable even to
// someone holding a database dump. KMS holds the key that protects the key.
const { KMSClient, GenerateDataKeyCommand, DecryptCommand } = require('@aws-sdk/client-kms'); // v3
const crypto = require('node:crypto');
const kms = new KMSClient({});

async function encryptField(plaintext, tenantId) {
  // One call returns both copies of the data key: Plaintext to use right now,
  // CiphertextBlob to persist. You never store the plaintext key.
  const { Plaintext, CiphertextBlob } = await kms.send(new GenerateDataKeyCommand({
    KeyId: process.env.KMS_KEY_ARN,
    KeySpec: 'AES_256',                      // a 32-byte key, the size AES-256 needs
    EncryptionContext: { tenant: tenantId }  // bound into the ciphertext; Decrypt must match
  }));

  const iv = crypto.randomBytes(12);         // 12 bytes is the recommended GCM IV length
  const cipher = crypto.createCipheriv('aes-256-gcm', Plaintext, iv);
  const body = Buffer.concat([cipher.update(plaintext, 'utf8'), cipher.final()]);
  const tag = cipher.getAuthTag();           // must be read after final(), never before
  Plaintext.fill(0);                         // stop holding the key in process memory

  // All four parts go in the row. The wrapped key is useless without KMS.
  return { wrapped_key: Buffer.from(CiphertextBlob), iv, tag, body };
}

async function decryptField(rec, tenantId) {
  const { Plaintext } = await kms.send(new DecryptCommand({
    CiphertextBlob: rec.wrapped_key,
    EncryptionContext: { tenant: tenantId }  // a mismatch fails the decrypt, by design
  }));
  const d = crypto.createDecipheriv('aes-256-gcm', Plaintext, rec.iv);
  d.setAuthTag(rec.tag);                     // set before final(), or final() throws
  try {
    return Buffer.concat([d.update(rec.body), d.final()]).toString('utf8');
  } finally {
    Plaintext.fill(0);
  }
}

// Cost note for the design discussion: one KMS call per record is fine for a
// profile read and ruinous for a bulk export. Cache the unwrapped key per
// request, or wrap one data key per tenant per day rather than per row.

When to use it

  • Answering "how do you protect customer data" without stopping at "we use TLS and encrypted disks".
  • Choosing which columns need field-level encryption rather than relying on volume encryption.
  • Rotating a database password without redeploying every service that connects.
  • Working out what a GDPR Article 17 erasure request has to touch beyond the primary table.

More examples

Reading secrets so that rotation is not a deploy

The point of a secrets manager is not that the string is hidden — it is that the string can change without a deploy, and that every read is audited. That only holds if the application re-reads. Reach for this shape whenever a credential is rotated on a schedule, and pair it with a reconnect-on-auth-error path.

Example · javascript
const { SecretsManagerClient, GetSecretValueCommand } = require('@aws-sdk/client-secrets-manager');
const sm = new SecretsManagerClient({});

function cachedSecret(secretId, ttlMs = 5 * 60 * 1000) {
  let value = null, fetchedAt = 0, inflight = null;
  return async function get() {
    if (value && Date.now() - fetchedAt < ttlMs) return value;

    // Shorthand: `inflight ??= expr` is exactly `inflight = inflight ?? expr`,
    // which collapses a thundering herd of concurrent refreshes into one call.
    inflight ??= sm.send(new GetSecretValueCommand({ SecretId: secretId })).then(
      r => { value = JSON.parse(r.SecretString); fetchedAt = Date.now(); inflight = null; return value; },
      e => { inflight = null; if (value) return value; throw e; } // serve stale rather than fail
    );
    return inflight;
  };
}

const dbCreds = cachedSecret(process.env.DB_SECRET_ARN);

async function connect() {
  // Read at connect time, not at boot. Rotation changes the password while the
  // process is alive; a credential captured once in a module-level const is the
  // reason your service dies four hours after a successful rotation.
  const { username, password } = await dbCreds();
  return pool.connect({ user: username, password });
}

Crypto-shredding: erasure where you cannot rewrite the data

This is the answer to the follow-up question "what about backups?". You cannot selectively edit an immutable backup, so you make the data in it meaningless instead. The design cost is that you must have decided to key data per subject before anyone asked to be deleted, which is why this belongs in the schema discussion rather than the compliance one.

Example · javascript
// An erasure request, handled in three parts because deleting the row is only one.
const { ScheduleKeyDeletionCommand } = require('@aws-sdk/client-kms');

async function eraseSubject(subjectId) {
  // 1. The copies you own and can write to.
  await db.none('DELETE FROM profiles WHERE subject_id = $1', [subjectId]);

  // 2. The per-subject data key. Once destroyed, every ciphertext copy of that
  //    subject's fields is undecryptable: last month's backup, the warehouse
  //    extract, the WORM archive. You cannot rewrite those; you do not need to.
  await kms.send(new ScheduleKeyDeletionCommand({
    KeyId: await keyArnFor(subjectId),
    PendingWindowInDays: 7   // 7 is the minimum, 30 the maximum — deletion is never instant
  }));

  // 3. The derived copies, by hand, because nothing fans the delete out for you.
  await Promise.all([
    redis.del(`profile:${subjectId}`),
    search.deleteDocument({ index: 'profiles', id: subjectId }),
    warehouse.enqueueTombstone(subjectId)  // last night's export already shipped
  ]);
}

Keeping PII out of logs at the serialiser

Logs are a data store with no schema, a long retention period and wide read access, which makes them the worst place for personal data. Apply this when a log pipeline ships to a vendor, when retention exceeds what consent covers, or the first time an erasure request arrives and you realise the only copy you cannot find is in log search.

Example · javascript
// Personal data escapes through logs far more often than through the database:
// one `log.info({ req })` and a header lands in a third-party log search index.
const pino = require('pino');

const log = pino({
  redact: {
    paths: [
      'req.headers.authorization',
      'req.headers.cookie',
      'body.password',
      'body.national_id',
      'user.email',
      '*.card_number'          // any parent object, but exactly one level deep:
                               // body.card_number yes, body.payment.card_number no
    ],
    censor: '[redacted]'
  }
});

// Redact centrally, not per call site: call sites are written by people who do
// not know what is in the object they are passing.
log.info({ tenant: req.user.tenant, route: '/checkout', ms: 142, status: 402 }, 'payment declined');

// What is left is still everything you need to debug: tenant, route, latency,
// outcome. If you need the identity, log the subject id and join it later.

Discussion

  • Be the first to comment on this lesson.