Attaching a Knowledge Base to an Agent, and RAG vs Fine-Tuning

Calling RetrieveAndGenerate yourself is a workflow: your code decides retrieval happens. Attaching a knowledge base to a Bedrock agent is different — the model decides, on each turn, whether this question needs the corpus, or an action group, or neither.

Associating the knowledge base

You call AssociateAgentKnowledgeBase against the agent's DRAFT version, supplying the knowledge base ID, a knowledgeBaseState of ENABLED or DISABLED, and — the important part — a description.

That description is the routing signal. It is injected into the agent's orchestration prompt, and it is all the model has to go on when deciding whether to search. Treat it exactly like a tool description: say what the corpus contains, what kinds of questions it answers, and explicitly what it does not cover so the agent reaches for the right action group instead. "Company documents" is a bad description. "Published UK HR handbook: leave, expenses, notice periods. Does not contain individual employee records — use the payroll action group for those" is a good one.

Two operational details bite people. First, an association is made against the draft, so nothing changes for callers until you run PrepareAgent and point an alias at the new version. Second, an agent can have several knowledge bases; if two descriptions overlap, routing becomes unreliable. Make the boundaries between them obvious.

Scoping retrieval per request

A single knowledge base usually serves many users who must not see each other's data. You do not need one knowledge base per tenant: on InvokeAgent you can pass sessionState.knowledgeBaseConfigurations with a retrieval configuration that overrides the defaults for this invocation, including a metadata filter. Your application — which knows who the caller is — injects the tenant or clearance filter, and the model cannot remove it because it never sees it. That is the correct place to enforce isolation. Never rely on a system prompt instruction such as "only answer about tenant X"; instructions are advisory, filters are not.

RAG or fine-tuning?

The two solve different problems and are frequently confused.

	RAG / knowledge base	Fine-tuning
Teaches	Facts, documents, current state	Behaviour: format, tone, task style, domain vocabulary
Update cost	Re-sync the data source	Prepare a labelled dataset and retrain
Freshness	Bounded by sync frequency	Frozen at training time
Citations	Yes, spans map to source chunks	None — the knowledge is in the weights
Access control	Per-request metadata filters	None — anything learned is available to everyone
Serving	Standard on-demand model invocation	Needs a dedicated deployment such as Provisioned Throughput, billed whether or not it is used

The access-control row is the one that ends most arguments. A fact baked into weights cannot be filtered, revoked or attributed. If a document must be visible to some users and not others, or must be correctable when it turns out to be wrong, it belongs in a knowledge base — full stop.

Fine-tuning earns its place when the behaviour is the problem: the model will not reliably produce your JSON shape, or it does not use your industry's terminology correctly, or you want a smaller, cheaper model to imitate a larger one on one narrow task. In practice the strong combination is a fine-tuned or well-prompted model for style plus a knowledge base for facts — they are complements, not alternatives.

Before reaching for either, exhaust prompt engineering and retrieval tuning. Most "the model does not know our domain" complaints are chunking and filtering problems wearing a costume.

import boto3

agent = boto3.client("bedrock-agent")
rt = boto3.client("bedrock-agent-runtime")

# 1. Associate the knowledge base with the DRAFT agent. The description is
#    the routing signal the model reads when deciding whether to search.
agent.associate_agent_knowledge_base(
    agentId="AGENT123ABC",
    agentVersion="DRAFT",
    knowledgeBaseId="KB1234ABCD",
    description=(
        "Published UK HR handbook. Use for questions about what the policy says: "
        "leave entitlement, expenses, notice periods, benefits. Does NOT contain "
        "individual employee data - use the payroll action group for balances "
        "and personal records."
    ),
    knowledgeBaseState="ENABLED",
)

# 2. PrepareAgent only rebuilds DRAFT. TSTALIASID always points at DRAFT, so
#    that alias picks the change up immediately; a production alias keeps
#    serving its pinned version until you publish a new version from DRAFT
#    (create_agent_alias snapshots DRAFT) and repoint the alias at it.
agent.prepare_agent(agentId="AGENT123ABC")

# 3. At invoke time, scope retrieval to this caller. The filter is applied
#    by the service, not suggested to the model.
response = rt.invoke_agent(
    agentId="AGENT123ABC",
    agentAliasId="TSTALIASID",
    sessionId="session-7f3c",
    inputText="How much notice do I have to give?",
    sessionState={
        "knowledgeBaseConfigurations": [
            {
                "knowledgeBaseId": "KB1234ABCD",
                "retrievalConfiguration": {
                    "vectorSearchConfiguration": {
                        "numberOfResults": 5,
                        "filter": {
                            "andAll": [
                                {"equals": {"key": "tenant_id", "value": "acme-uk"}},
                                {"equals": {"key": "status", "value": "published"}},
                            ]
                        },
                    }
                },
            }
        ]
    },
)

for event in response["completion"]:
    if "chunk" in event:
        print(event["chunk"]["bytes"].decode(), end="")