Retrieve vs RetrieveAndGenerate, Citations and Grounding

Once a knowledge base is synced you query it through the bedrock-agent-runtime client, and there are two ways to do that. Choosing between them is choosing how much of the RAG loop you want to own.

Retrieve

Retrieve is search. You send a query and a retrieval configuration; you get back a list of chunks, each with its text, a relevance score, its source location and its metadata. No model is invoked, so you pay only for retrieval, and nothing is generated.

Use it when you want control over what happens next: you are building your own orchestration, you want to rerank or deduplicate results yourself, you want to feed the passages to a model outside Bedrock, or the passages are the answer — a search results page does not need prose.

RetrieveAndGenerate

RetrieveAndGenerate is the managed loop: retrieve, build the prompt, call a model, return the answer. You supply a knowledge base ID and a model ARN — commonly a cross-region inference profile — and you get back output.text plus a citations array. Pass a sessionId back on the next call and it maintains multi-turn context, including rewriting follow-up questions that depend on earlier turns.

It also accepts an EXTERNAL_SOURCES type, where you pass documents inline instead of naming a knowledge base — useful for "chat with this file I just uploaded" without ingesting anything.

Inside retrievalConfiguration.vectorSearchConfiguration you can set numberOfResults, a metadata filter, overrideSearchType of HYBRID or SEMANTIC where the store supports it, and a reranking configuration that runs a dedicated reranker model over the candidates before generation. Under generationConfiguration you can attach a guardrail and override the prompt template.

Citations and what they are worth

Each citation links a span of the generated text to the retrieved references that support it — the source URI, the chunk content and its metadata. That is the mechanism that turns a chatbot into something auditable: a user can click through to the paragraph the sentence came from.

Be precise about what a citation proves. It proves the passage was in the prompt when that sentence was generated. It does not prove the sentence faithfully represents the passage. Citations are a strong signal, not a correctness guarantee.

Grounding

For a stronger guarantee, attach a Bedrock guardrail with the contextual grounding filter. It scores the response against the retrieved passages on grounding (is this supported by the source?) and relevance (does this answer the question?), and blocks the response when either falls below your threshold. That is how you stop an ungrounded answer reaching a user rather than merely detecting it afterwards.

The trade-off

RetrieveAndGenerate is less code and fewer round trips, but the orchestration is a black box: you cannot inspect or modify the passages between retrieval and generation. Retrieve gives you that seam at the cost of writing the prompt, the model call, the citation mapping and the grounding check yourself. Start managed; move to Retrieve when you hit something the managed loop will not let you do.

import boto3

rt = boto3.client("bedrock-agent-runtime")

resp = rt.retrieve_and_generate(
    input={"text": "How many days of carer's leave do UK staff get?"},
    retrieveAndGenerateConfiguration={
        "type": "KNOWLEDGE_BASE",
        "knowledgeBaseConfiguration": {
            "knowledgeBaseId": "KB1234ABCD",
            "modelArn": (
                "arn:aws:bedrock:eu-west-1:111122223333:inference-profile/"
                "eu.anthropic.claude-sonnet-4-5-20250929-v1:0"
            ),
            "retrievalConfiguration": {
                "vectorSearchConfiguration": {
                    "numberOfResults": 8,
                    "overrideSearchType": "HYBRID",
                    "filter": {
                        "andAll": [
                            {"equals": {"key": "region", "value": "uk"}},
                            {"equals": {"key": "status", "value": "published"}},
                        ]
                    },
                }
            },
            "generationConfiguration": {
                "guardrailConfiguration": {
                    "guardrailId": "gr-abc123",
                    "guardrailVersion": "3",
                }
            },
        },
    },
)

print(resp["output"]["text"])
print("session:", resp["sessionId"])  # pass back for the next turn

for citation in resp["citations"]:
    span = citation["generatedResponsePart"]["textResponsePart"]["span"]
    for ref in citation["retrievedReferences"]:
        print(span, "->", ref["location"], ref["metadata"].get("effective_year"))