When Not to Build an Agent
Most tasks labelled agentic are better served by a single call, a prompt chain or a state machine — here is how to tell, and what to build instead.
The most valuable agentic skill is declining to build an agent. An agent buys you one thing: run-time flexibility in the sequence of steps. You pay for it with non-determinism, extra latency, multiplied token cost and a system that is genuinely hard to test. If the flexibility is not needed, you have paid the price for nothing.
Four signals that say "do not build an agent"
- The steps are fixed. If you can draw the flowchart before the request arrives, encode the flowchart. A model asked to rediscover the same five steps on every run will occasionally get them wrong, and always cost more.
- It is one model call. Classification, extraction, summarisation, rewriting, and question answering over retrieved documents are single calls. RAG is not agentic — retrieve, then generate, then stop.
- Error tolerance is near zero. Ledger postings, medication dosing, regulatory filings and anything with a legal signature need determinism and an audit trail that a probabilistic planner cannot supply. Use the model for the judgement inside a step, and code for the sequencing between steps.
- You cannot check the work. If there is no automated validator and no human reviewer, an autonomous loop just produces confident output nobody can verify.
What to build instead
| Situation | Build this |
|---|---|
| One transformation of one input | A single Converse call with a tight prompt and a JSON schema |
| Answering from your own documents | Knowledge Bases for Amazon Bedrock and RetrieveAndGenerate |
| A fixed sequence of prompts and lookups | A prompt chain in code, or Amazon Bedrock Flows for a visual version |
| Branching, retries, parallelism, long-running steps | AWS Step Functions — deterministic, resumable, fully traced |
| Genuinely variable, goal-directed work | An agent — and only then |
The hybrid is usually the right answer
The best production designs are rarely all-agent or all-workflow. They put the deterministic skeleton in a state machine and drop agentic reasoning into the one or two states that need it. A document pipeline might extract, validate and route deterministically, and invoke an agent only in the "exception handling" branch where a human would otherwise have to improvise. You get replayable, observable orchestration for 90% of the flow and flexibility exactly where it earns its keep.
There is also a cost argument the exam likes. An agent re-sends its whole transcript on every turn, so a ten-step run can consume many times the tokens of a single call, plus a round trip per step. Where latency budgets are tight — an interactive checkout, a phone call — a fixed chain of two calls will beat a five-turn agent every time.
Example
{
"Comment": "Deterministic invoice pipeline; the agent is used only for exceptions",
"StartAt": "ExtractFields",
"States": {
"ExtractFields": {
"Type": "Task",
"Resource": "arn:aws:states:::bedrock:invokeModel",
"Parameters": {
"ModelId": "anthropic.claude-3-5-sonnet-20240620-v1:0",
"Body": {
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": 512,
"messages": [
{
"role": "user",
"content.$": "States.Format('Extract JSON fields from: {}', $.rawText)"
}
]
}
},
"Retry": [
{
"ErrorEquals": [
"States.ALL"
],
"MaxAttempts": 3,
"BackoffRate": 2
}
],
"ResultPath": "$.extraction",
"Next": "MatchesPurchaseOrder"
},
"MatchesPurchaseOrder": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.extraction.matched",
"BooleanEquals": true,
"Next": "PostToLedger"
}
],
"Default": "InvestigateWithAgent"
},
"PostToLedger": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "post-to-ledger",
"Payload.$": "$"
},
"End": true
},
"InvestigateWithAgent": {
"Comment": "InvokeAgent returns a streamed response, so Step Functions cannot call it through the AWS SDK integration. Wrap it in a Lambda function that drains the event stream and returns the final text.",
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "invoke-billing-agent",
"Payload": {
"agentId": "AGENT123456",
"agentAliasId": "TSTALIASID",
"sessionId.$": "$$.Execution.Name",
"inputText.$": "States.Format('Invoice {} did not match any PO. Investigate.', $.invoiceId)"
}
},
"End": true
}
}
}When to use it
- A payments team keeps settlement in Step Functions for its exactly-once guarantees and calls an agent only for unmatched transactions.
- A knowledge portal ships plain RAG rather than an agent, because every question is answered by one retrieval and one generation.
- A contact centre reverts an over-engineered agent to a two-call chain after discovering the agent added seconds of latency for the same result.
Discussion