Chunking Strategies and Metadata Filtering

Retrieval never returns documents. It returns chunks. Whatever the boundaries cut through is lost, because the model only sees the passages you hand it. Chunking is therefore not a tuning knob you adjust later — it determines the ceiling on answer quality.

Why boundaries matter

Consider a table where the header row says "Region: UK" and a row 40 lines below says "Carer's leave: 5 days". Split between them and the retrieved chunk says "5 days" with no idea which region it applies to. The model has no way to recover the missing context, and it will answer anyway. Bad boundaries do not produce empty answers — they produce confidently wrong ones.

The same effect works in reverse. A chunk that is too large dilutes its own embedding: one vector has to represent five unrelated topics, so it matches everything weakly and nothing strongly.

The strategies Bedrock offers

• Default chunking — a sensible fixed size with overlap, respecting sentence boundaries. Fine for prose; a reasonable first attempt.
• Fixed-size chunking — you set the maximum tokens per chunk and an overlap percentage. Overlap exists precisely so a fact split across a boundary appears whole in at least one chunk. It costs storage in exchange for recall.
• Hierarchical chunking — small child chunks are embedded and searched, but the larger parent chunk is what gets returned to the model. You get the precision of a small vector with the context of a big passage. This is the best default for structured documents such as handbooks and manuals.
• Semantic chunking — boundaries are placed where the meaning shifts, detected by comparing embeddings of adjacent sentences. Better on unstructured prose that has no headings, at the cost of extra embedding calls during ingestion.
• No chunking — one chunk per file. Correct when you have already split content upstream into one record per file, and wrong otherwise.
• Custom transformation — a Lambda function you write, invoked during ingestion, for documents whose structure only you understand.

Metadata: the other half

Chunking decides what a passage contains. Metadata decides whether it is eligible to be returned at all. Alongside each source file in S3 you place a sidecar JSON file named after the source file plus .metadata.json, containing a metadataAttributes object. Those attributes are indexed with the chunk and can be filtered on at query time using operators such as equals, notEquals, greaterThan, in, listContains, startsWith, and the combinators andAll and orAll.

Metadata is what makes a single knowledge base usable by more than one audience. Tag by tenant, region, document status, effective date or classification, and filter at retrieval time. Filtering is also the cheap fix for staleness: keep superseded documents indexed but tag them status: archived and exclude them by default, rather than deleting and re-ingesting.

The trade-off and the failure mode

Every filter you apply narrows the candidate set before ranking. Over-filter and a perfectly good answer is excluded, and the model reports that it cannot find one — which looks identical to a corpus gap. When debugging "the answer is in there but it will not find it", remove the filter first and re-run before touching chunking.

The other trap is silent: metadata attributes that were not present at ingestion time do not exist on the chunk. Adding a sidecar file after the fact does nothing until that document is re-synced, and a filter on a missing attribute simply excludes the chunk.

{
  "_file": "s3://acme-policy-docs/published/handbook-uk-2026.pdf.metadata.json",
  "metadataAttributes": {
    "region": "uk",
    "doc_type": "handbook",
    "status": "published",
    "effective_year": 2026,
    "audience": ["all-staff", "managers"],
    "tenant_id": "acme-uk"
  }
}