Retrieve, Augment, Generate

The query-time flow that turns a question into a grounded answer.

At query time, RAG runs three steps: retrieve relevant chunks, augment the prompt with them, and generate the answer.

RAG pipeline: a query retrieves documents that augment the prompt before the LLM generates an answerQueryRetrievetop matchesVector databaseAugmentprompt + docsGenerate (LLM)grounded answer
The retrieved passages are inserted into the prompt so the model answers from them.

The augmented prompt tells the model: answer the question using only the context below.

Example

Example · json
{
  "messages": [{
    "role": "user",
    "content": "Answer using ONLY the context. If it is not there, say you don't know.\n\nCONTEXT:\n- Refunds are available within 14 days.\n- Shipping is free over $50.\n\nQUESTION: How long do I have to get a refund?"
  }]
}

When to use it

  • A customer-service bot runs the full RAG flow for every incoming question, retrieving from the product knowledge base before generating any reply.
  • An internal wiki assistant retrieves the three most relevant wiki pages for each employee question and passes them as context to the LLM.
  • A medical Q&A tool retrieves current guideline passages at query time, augments the prompt with them, then generates a grounded answer with citations.

More examples

Complete RAG pipeline in one script

End-to-end RAG: index documents, retrieve the best match for a query, augment the prompt, and generate a grounded answer.

Example · python
import numpy as np
from openai import OpenAI

client = OpenAI()

def embed(text):
    return np.array(
        client.embeddings.create(model='text-embedding-3-small', input=text).data[0].embedding
    )

cosine = lambda a, b: np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# 1. Index
docs = [
    'Free tier: 1GB storage, 100 API calls/day.',
    'Pro plan: 100GB storage, unlimited API calls, $29/month.',
    'Enterprise: custom storage, SLA, contact sales.',
]
doc_vecs = [embed(d) for d in docs]

# 2. Retrieve
query = 'What do I get on the Pro plan?'
q_vec = embed(query)
best = max(range(len(docs)), key=lambda i: cosine(q_vec, doc_vecs[i]))
context = docs[best]

# 3. Generate
r = client.chat.completions.create(
    model='gpt-4o-mini',
    messages=[{'role': 'user', 'content': f'Context: {context}\n\nQuestion: {query}'}]
)
print(r.choices[0].message.content)

Top-k retrieval for richer context

Retrieves the top-3 matching documents and concatenates them as context for a richer augmented prompt.

Example · python
import numpy as np
from openai import OpenAI

client = OpenAI()

def embed(text):
    return np.array(
        client.embeddings.create(model='text-embedding-3-small', input=text).data[0].embedding
    )

cosine = lambda a, b: np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

docs = [
    'Refunds are available within 30 days.',
    'Contact support at [email protected].',
    'Items must be in original packaging for returns.',
    'Shipping takes 5-7 business days.',
]
doc_vecs = [embed(d) for d in docs]
query  = 'How do I return a product?'
q_vec  = embed(query)
scores = [(i, cosine(q_vec, v)) for i, v in enumerate(doc_vecs)]
top3   = sorted(scores, key=lambda x: x[1], reverse=True)[:3]
context = '\n'.join(docs[i] for i, _ in top3)

r = client.chat.completions.create(
    model='gpt-4o-mini',
    messages=[{'role': 'user', 'content': f'Context:\n{context}\n\nQuestion: {query}'}]
)
print(r.choices[0].message.content)

System prompt RAG pattern

Puts the retrieved context in the system prompt and instructs the model to answer only from it.

Example · python
from openai import OpenAI

client = OpenAI()

retrieved_context = (
    'Policy excerpt: Orders over $50 ship free. '
    'Express shipping is available for $9.99. '
    'International shipping starts at $14.99.'
)

r = client.chat.completions.create(
    model='gpt-4o-mini',
    messages=[
        {'role': 'system', 'content':
            'You are a helpful assistant. Answer ONLY using the provided context. '
            'If the answer is not in the context, say so.\n\n'
            f'CONTEXT:\n{retrieved_context}'},
        {'role': 'user', 'content': 'How much is express shipping?'}
    ]
)
print(r.choices[0].message.content)

Discussion

  • Be the first to comment on this lesson.