Tracing an Invocation to Find Out Why the Agent Chose That Tool

Setting enableTrace on InvokeAgent streams back the model's rationale, the exact prompt it saw, the parameters it chose and the observation it got, which is the only reliable way to debug tool selection.

When a Bedrock agent answers wrongly, the interesting question is almost never "why was the wording bad". It is: which tool did it pick, with what arguments, what did the tool return, and what did the model see just before deciding. Tracing is how you get all four. Pass enableTrace=True to InvokeAgent and the response stream carries trace events interleaved with the answer chunks.

What comes back

Each trace event wraps one of several trace types:

preProcessingTrace — the input classification step. If the agent replies "I cannot help with that" for something obviously in scope, the pre-processing step is the usual culprit.
orchestrationTrace — the main loop, and where you will spend your time. It emits, per step: modelInvocationInput (the fully rendered prompt, including your instruction and every action group description as the model actually saw them), rationale (the model's stated reasoning for the step), invocationInput (the chosen action group and operation with resolved parameters, or the knowledge base query), and observation (the tool result, retrieved passages, or the final response).
guardrailTrace — which guardrail policy intervened, on input or output.
postProcessingTrace — the optional final rewrite.
failureTrace — a failureReason when a step blew up, typically a schema mismatch or a Lambda error.
routingClassifierTrace — with multi-agent collaboration, which collaborator the supervisor routed to.

How to read it

Work backwards from the wrong step. If invocationInput shows the wrong operation, the fix is in the operation descriptions: two tools whose descriptions overlap will be confused, and the cure is to write the trigger condition and the exclusion into each one ("use this for open claims only; for closed claims use…"). If it shows the right operation with the wrong parameters, the parameter descriptions are too thin, or the value was never in the conversation and the model invented it — enable AMAZON.UserInput so it can ask instead. If observation shows the tool returned a correct result and the answer still contradicts it, that is an instruction problem, not a tool problem. And if modelInvocationInput shows an old description, you forgot to PrepareAgent or you are invoking an alias pinned to an older version.

The operational catch

Trace data is delivered to the caller in the response stream and is not written anywhere for you. If you do not capture it, it is gone — you cannot go back and ask why yesterday's conversation went wrong. Log it yourself, keyed by sessionId, from the moment you have anything in production. Two cautions when you do: the trace contains the fully rendered prompts, so it includes prompt session attributes and any user text, and must be treated as sensitive; and it is verbose, so sample it or keep it on a short retention. Bedrock's model invocation logging is a separate, account-level setting that captures model inputs and outputs — useful, but it is not the agent's reasoning trace.

Finally, tracing is a debugging tool, not an evaluation strategy. Reading one trace tells you what happened once. Deciding whether a change improved the agent needs a fixed set of conversations, run against the test alias, with assertions on which tools were selected — traces are what those assertions read.

Example

import json, boto3

rt = boto3.client("bedrock-agent-runtime")

resp = rt.invoke_agent(
    agentId=AGENT_ID,
    agentAliasId="TSTALIASID",
    sessionId="debug-001",
    inputText="Cancel the claim I opened last Tuesday.",
    enableTrace=True,
)

for event in resp["completion"]:
    if "chunk" in event:
        print("ANSWER:", event["chunk"]["bytes"].decode())
        continue
    if "trace" not in event:
        continue

    t = event["trace"]["trace"]

    if "preProcessingTrace" in t:
        out = t["preProcessingTrace"].get("modelInvocationOutput", {})
        print("PREPROC:", out.get("parsedResponse"))

    if "orchestrationTrace" in t:
        o = t["orchestrationTrace"]
        if "modelInvocationInput" in o:
            # the fully rendered prompt - check your descriptions really are in here
            print("PROMPT BYTES:", len(o["modelInvocationInput"]["text"]))
        if "rationale" in o:
            print("WHY:", o["rationale"]["text"])
        if "invocationInput" in o:
            print("CHOSE:", json.dumps(o["invocationInput"], default=str)[:600])
        if "observation" in o:
            ob = o["observation"]
            if "actionGroupInvocationOutput" in ob:
                print("TOOL SAID:", ob["actionGroupInvocationOutput"]["text"][:600])
            if "knowledgeBaseLookupOutput" in ob:
                print("RETRIEVED:", len(ob["knowledgeBaseLookupOutput"]["retrievedReferences"]))

    if "guardrailTrace" in t:
        print("GUARDRAIL:", t["guardrailTrace"].get("action"))

    if "failureTrace" in t:
        print("FAILED:", t["failureTrace"]["failureReason"])

When to use it

Diagnosing an agent that calls the refund tool when the user only asked about delivery, by reading the rationale and the overlapping tool descriptions.
Proving that a change to an operation description actually reached the model, by inspecting modelInvocationInput on the test alias.
Building a regression suite that replays fixed conversations and asserts on the action groups named in invocationInput.

Discussion

Be the first to comment on this lesson.