Evaluating & Testing LLM Features
Treat prompts like code: build a golden set, run offline evals, and catch regressions before your users do.
The single habit that separates teams who ship reliable AI from teams who firefight it: they measure. You would not deploy a function with no tests; an LLM feature deserves the same discipline.
Build a golden set
Collect 20–200 real inputs paired with known-good outputs (or acceptance criteria). Include the easy cases, the tricky edge cases, and the ones that have already burned you. This is your regression suite. Every prompt change, model swap, or temperature tweak runs against it.
Offline evals: three ways to grade
- Exact / structural match — best for extraction and classification. Did the JSON have the right fields and values?
- Assertions — "the answer must contain the order number", "must be under 40 words", "must not mention a competitor". Cheap, fast, and catch most regressions.
- LLM-as-judge — for open-ended output where there is no single right answer, a second model grades against a rubric. Powerful, but validate the judge against human ratings first.
Track a scorecard over time
Run the suite on every change and record pass rate, cost, and latency. A prompt tweak that fixes one case but drops your overall score three points should never reach production — and without a golden set you would never have known.
Example
// A tiny offline eval harness — the shape scales to hundreds of cases.
const goldenSet = [
{ input: "Card charged twice", expect: { team: "Billing" } },
{ input: "App crashes on login", expect: { team: "Engineering" } },
{ input: "How do I export data?", expect: { team: "Support" } },
];
async function runEval(promptVersion) {
let pass = 0;
for (const { input, expect } of goldenSet) {
const out = await classifyTicket(input, promptVersion);
if (out.team === expect.team) pass++;
else console.log(`FAIL: ${input} -> ${out.team}, wanted ${expect.team}`);
}
const score = pass / goldenSet.length;
console.log(`v${promptVersion}: ${(score * 100).toFixed(1)}% (${pass}/${goldenSet.length})`);
return score;
}
// Gate deploys on it: fail CI if the score drops below the last known-good.
// if (await runEval("v3") < BASELINE) process.exit(1);When to use it
- A team runs their golden test set of 100 prompt/expected pairs in CI so a prompt regression is caught before merging any change.
- A product scores new model versions against the current baseline on 50 real user queries before deciding whether to upgrade.
- A QA engineer uses an LLM-as-judge to score 500 chatbot responses for helpfulness, skipping expensive human annotation.
More examples
Golden test set with pass rate
Runs a golden test set and reports a pass rate — the baseline metric to track across prompt iterations.
from openai import OpenAI
client = OpenAI()
GOLDEN = [
{'q': 'Capital of France?', 'a': 'Paris'},
{'q': 'Capital of Japan?', 'a': 'Tokyo'},
{'q': 'Capital of Brazil?', 'a': 'Brasilia'},
]
passed = 0
for item in GOLDEN:
r = client.chat.completions.create(
model='gpt-4o-mini',
messages=[{'role':'user','content':item['q'] + ' One word.'}]
)
if item['a'].lower() in r.choices[0].message.content.lower():
passed += 1
print(f'Pass rate: {passed}/{len(GOLDEN)} = {100*passed//len(GOLDEN)}%')LLM judge scores helpfulness 1-5
Uses a second LLM call to rate helpfulness numerically, enabling automated quality tracking across many responses.
import json
from openai import OpenAI
client = OpenAI()
def helpfulness_score(question, answer):
r = client.chat.completions.create(
model='gpt-4o-mini',
messages=[{'role':'user','content':
f'Q: {question}\nA: {answer}\n\n'
'Rate helpfulness 1-5. JSON: {"score": int, "reason": str}'
}],
response_format={'type':'json_object'}
)
return json.loads(r.choices[0].message.content)
result = helpfulness_score(
'How do I open a file in Python?',
'Use open("filename", "r") to open a file for reading.'
)
print(f"Score: {result['score']}/5 — {result['reason']}")Regression test between two prompt versions
Runs both prompt versions on the same input so length and wording differences are immediately visible.
from openai import OpenAI
client = OpenAI()
PROMPTS = {
'v1': 'Summarise the following text.',
'v2': 'Summarise the following text in one sentence, for a non-expert.'
}
TEST_TEXT = 'Photosynthesis is the process by which plants use sunlight, water, and CO2 to produce glucose and oxygen.'
for version, sys_prompt in PROMPTS.items():
r = client.chat.completions.create(
model='gpt-4o-mini',
messages=[
{'role':'system','content':sys_prompt},
{'role':'user','content':TEST_TEXT}
]
)
out = r.choices[0].message.content
print(f'{version} ({len(out)} chars): {out[:100]}')
Discussion