CAP, and Why PACELC Is More Useful
CAP constrains only what happens during a partition; PACELC adds the consistency-versus-latency choice you make the rest of the time.
PACELC: if (P) then (A or C) else (L or C)CAP is a statement about one narrow situation: when the network between replicas drops messages, a replicated system can either keep answering with possibly stale data, or refuse to answer until it is sure. Brewer posed it in 2000; Gilbert and Lynch proved a formal version in 2002.
The three things people wrongly take from it
- "Pick two of three." Partition tolerance is not a feature you decline. Packets get dropped whether or not you planned for it, so the only real choice is what the system does while that is happening.
- The C is ACID's C. It is not. CAP's C is linearizability: every read sees the latest completed write. A single-node database can be fully ACID and have nothing to say about CAP.
- The A is uptime. CAP availability means every request to a non-failed node gets a non-error response. A 99.99% target is a different claim entirely.
What PACELC adds
Abadi's formulation (IEEE Computer, February 2012) reads: if Partition, choose Availability or Consistency; Else, choose Latency or Consistency. The second half is the half you live in. With no partition anywhere, a replicated write still has to decide whether to wait for a remote acknowledgement before returning. Abadi classifies Dynamo, Cassandra and Riak as PA/EL, and fully ACID systems as PC/EC. Spanner sits on the consistency side of both letters and pays for it in commit-wait.
In an interview, do not recite CAP. Name the trade per flow: "the balance read is EC, the activity feed is EL."
Example
// @aws-sdk/client-dynamodb v3. ConsistentRead is the PACELC "E" knob made
// explicit: there is no partition here, you are just choosing latency or
// consistency on a per-call basis.
import { DynamoDBClient, GetItemCommand } from "@aws-sdk/client-dynamodb";
const ddb = new DynamoDBClient({ region: "us-east-1" });
// EC: route to the leader, pay the extra latency, and pay twice the read
// capacity. Correct when staleness is a bug rather than a cosmetic issue.
export function readBalanceForTransfer(accountId) {
return ddb.send(new GetItemCommand({
TableName: "accounts",
Key: { id: { S: accountId } },
ConsistentRead: true,
}));
}
// EL: any replica answers, so the read is faster and half the cost, and the
// item may be a few hundred milliseconds behind. This is the default.
export function readBalanceForDashboard(accountId) {
return ddb.send(new GetItemCommand({
TableName: "accounts",
Key: { id: { S: accountId } },
ConsistentRead: false,
}));
}
// Same table, same data, two different answers to "how stale may this be?".
// That is the decision PACELC is asking you to make out loud.When to use it
- Answering "what happens when the link between your two regions drops?" without hand-waving
- Deciding whether a cross-region read goes to the nearest replica or back to the leader
- Explaining why switching a replica to synchronous made the write path slower rather than safer
- Choosing a datastore for a flow where a stale read is a correctness bug, not a cosmetic one
More examples
The same trade, chosen per statement in Cassandra
Shows that the latency-versus-consistency choice does not have to be a deployment-wide setting. Reach for this when one service has both a money path and a display path against the same tables: pin the money path to LOCAL_QUORUM and leave the display path on ONE, instead of arguing about a single global default.
// cassandra-driver (DataStax Node.js driver). The consistency level is a
// per-query option, so one keyspace can serve both sides of the E trade.
const cassandra = require("cassandra-driver");
const { consistencies } = cassandra.types;
const client = new cassandra.Client({
contactPoints: ["10.0.1.11", "10.0.2.11"],
localDataCenter: "eu-west-1",
keyspace: "ledger",
});
// EC-ish: a majority of the LOCAL datacentre's replicas must answer. One
// round trip inside the region, no cross-region hop.
async function readEntryForAudit(id) {
const rs = await client.execute(
"SELECT id, amount, created_at FROM entries WHERE id = ?",
[id],
{ prepare: true, consistency: consistencies.localQuorum },
);
return rs.first();
}
// EL: the first replica to answer wins. Fastest possible read, and it may
// be missing a write that has not propagated yet.
async function readEntryForTimeline(id) {
const rs = await client.execute(
"SELECT id, amount, created_at FROM entries WHERE id = ?",
[id],
{ prepare: true, consistency: consistencies.one },
);
return rs.first();
}What the P branch looks like in code
The A-versus-C choice is usually discussed abstractly and then never written down. This is what it reduces to: a 503 or a degraded-but-accepted write, plus whatever reconciles the divergence later. Use this shape whenever an interviewer asks what happens during a partition, because it forces you to say who notices.
// A regional write path that has to behave during a partition. The decision
// is made once, in config, and stated in the response the caller sees.
const MODE = process.env.PARTITION_MODE; // "consistent" | "available"
async function recordVote(pollId, userId, choice) {
try {
// Requires an acknowledgement from the other region's replica.
return await writeWithRemoteAck(pollId, userId, choice, { timeoutMs: 800 });
} catch (err) {
if (!isUnreachable(err)) throw err;
if (MODE === "consistent") {
// CP: refuse the write rather than risk two regions disagreeing about
// the tally. The caller gets a retryable error, not a wrong answer.
const e = new Error("remote replica unreachable");
e.statusCode = 503;
e.retryAfterSeconds = 5;
throw e;
}
// AP: accept locally, mark it unmerged, and reconcile when the link
// returns. The tally is now approximate and the API must admit that.
await writeLocalOnly(pollId, userId, choice, { needsMerge: true });
return { accepted: true, authoritative: false };
}
}Putting the trade in the API contract
Makes the EL/EC choice a property of the request rather than of the deployment, and makes staleness visible to the client instead of folklore. Worth doing when the same endpoint serves a mobile dashboard and a settlement job, and you do not want to pay leader latency for every caller or lie to the one that needs the leader.
// Express. Instead of one global consistency setting, let the caller declare
// what it needs. The service can then satisfy each one honestly.
app.get("/v1/accounts/:id/balance", async (req, res) => {
// Default to the cheap, fast path; callers opt in to the expensive one.
const strong = req.get("X-Consistency") === "strong";
const row = strong
? await readFromLeader(req.params.id)
: await readFromNearestReplica(req.params.id);
// Tell the caller which branch it got, and how stale it may be. A client
// that cannot see this header cannot reason about what it is showing.
res.set("X-Consistency", strong ? "strong" : "eventual");
res.set("X-Staleness-Bound-Ms", strong ? "0" : "2000");
res.json({ balance: row.balance, asOf: row.updated_at });
});
Discussion