Externalising State with DynamoDB and ElastiCache
Authoritative state belongs in a store keyed by session and actor, not in the model's context, so the agent stays a stateless function that any worker can serve.
The final piece of the chapter is the one that makes the rest operable: the agent should hold no durable state. Every fact needed beyond the current request lives in a store. The agent becomes a stateless function of (incoming request, loaded state), and the turn handler looks the same every time - load state, build context, call the model, execute tools that read and write the store, persist, return.
Why the model must not be the store
Four reasons, and the exam cares about all of them.
- Scale. If the next turn can be served by any Lambda invocation or Fargate task, you can scale horizontally and deploy without draining sessions. In-process state pins a conversation to one worker.
- Durability. Lambda times out. Containers get replaced. A conversation whose only record was in memory is gone.
- Accuracy. A model is a lossy, non-authoritative store. It can paraphrase a figure, drop a digit, or be argued out of a value by a persuasive user. Anything you will act on - a balance, an entitlement, an approval - must be read back from the database at the moment you act.
- Auditability. "What did the agent know when it approved this?" is answerable from the durable record: the item history captured off DynamoDB Streams and landed in S3 or a log store. Note that the stream itself retains records for only 24 hours, so it is the change-capture feed, not the archive. It is not answerable at all from a context window that no longer exists.
Choosing the store
DynamoDB is the default system of record for agent state: single-digit-millisecond reads, on-demand capacity that suits spiky agent traffic, conditional writes for optimistic concurrency, a TTL attribute for automatic session expiry, and Streams for audit and downstream processing. Model it with a partition key per actor and a sort key per session or per turn, so one query loads everything a turn needs.
ElastiCache (Redis or Valkey) is for the hot working set: the transcript being appended to on every turn, per-user rate limits, and distributed locks that stop two concurrent requests for the same session from interleaving and corrupting the history. It is sub-millisecond but volatile - treat it as a cache in front of DynamoDB, never as the system of record.
S3 is for bulk: full transcripts for compliance, and large tool outputs. Rather than pasting a 200KB report into context, write it to S3 and give the model the key plus a short extract, with a tool that can read specific sections back.
Step Functions is the answer when the state is a workflow. If the process has defined stages, approvals and retries, let Step Functions own the execution state and make the agent one task within it. That inverts the usual design in a useful way: the agent stops being the thing that remembers where the process is.
The trade-offs
Two stores means consistency work, and the cache-versus-truth boundary must be explicit or you will serve stale sessions after a failover. DynamoDB TTL is not immediate deletion - items are typically removed within 48 hours of the expiry timestamp - so if a regulator requires prompt erasure, delete explicitly rather than relying on TTL. And do not put credentials or unnecessary personal data in the session table just because it is convenient; it is the most widely read table in the system.
Example
import time
import boto3
from botocore.exceptions import ClientError
ddb = boto3.resource("dynamodb").Table("agent-sessions")
# Table: PK=pk (actor#<id>), SK=sk (session#<id>), TTL attribute: expires_at
def load_state(actor_id, session_id):
item = ddb.get_item(
Key={"pk": f"actor#{actor_id}", "sk": f"session#{session_id}"},
ConsistentRead=True,
).get("Item")
return item or {"messages": [], "version": 0, "workflow_step": "start"}
def save_state(actor_id, session_id, state, idempotency_key):
"""Conditional write: fails if another turn wrote first, or if this
turn already landed (a retried Lambda invocation)."""
try:
ddb.put_item(
Item={
"pk": f"actor#{actor_id}",
"sk": f"session#{session_id}",
"messages": state["messages"],
"workflow_step": state["workflow_step"],
"last_turn_id": idempotency_key,
"version": state["version"] + 1,
"expires_at": int(time.time()) + 86_400, # TTL
},
ConditionExpression="attribute_not_exists(pk) OR "
"(version = :v AND last_turn_id <> :t)",
ExpressionAttributeValues={
":v": state["version"],
":t": idempotency_key,
},
)
except ClientError as e:
if e.response["Error"]["Code"] != "ConditionalCheckFailedException":
raise
# Concurrent turn or duplicate delivery: reload and re-evaluate.
def handle_turn(event, _context):
state = load_state(event["actor_id"], event["session_id"])
state["messages"].append(event["message"])
reply = call_agent(state) # stateless: state goes in, reply comes out
state["messages"].append(reply)
save_state(event["actor_id"], event["session_id"], state, event["turn_id"])
return replyWhen to use it
- A chat agent on API Gateway and Lambda where consecutive turns land on different invocations and must load the conversation from DynamoDB each time.
- An ordering agent that holds the cart in DynamoDB with conditional writes, so a user who says 'you agreed to give me 90% off' cannot talk the model into a price the database never recorded.
- A high-traffic agent using ElastiCache for the live transcript and distributed locks, with DynamoDB behind it as the durable system of record.
Discussion