Iterating on Prompts
Treat prompts like code: test them, spot failures, and refine — do not expect perfection first try.
Prompts are rarely perfect on the first attempt. The professional approach is to iterate: run the prompt on real inputs, look at where it falls short, and adjust.
A simple loop
- Write a first prompt and run it on several real examples.
- Note the failures — wrong format, missed cases, wrong tone.
- Add an instruction or example that fixes that specific failure.
- Repeat until the output is reliably good.
Test with variety
Try tricky and unusual inputs, not just the easy ones. A prompt that handles edge cases is one you can trust in production.
Example
// A tiny test harness for a prompt
const cases = [
{ input: "Great value!", expect: "positive" },
{ input: "Meh, it's fine.", expect: "neutral" },
{ input: "Total waste of money", expect: "negative" },
];
for (const c of cases) {
const label = await classify(c.input); // wraps your Claude call
console.log(c.input, "=>", label, label === c.expect ? "OK" : "FIX");
}When to use it
- Run 10 sample inputs through your prompt and score outputs before shipping a content-generation feature.
- Keep a changelog of prompt versions so you can roll back to the previous version when a deployment degrades quality.
- Measure pass rate against a fixed test set after every prompt change to catch regressions before users do.
More examples
Run a prompt against a test set
Runs the prompt against three labelled cases and counts how many the current prompt version passes.
const testCases = [
{ input: "great product, fast shipping", expected: "positive" },
{ input: "broke after one day", expected: "negative" },
{ input: "fine, nothing special", expected: "neutral" },
];
let passed = 0;
for (const tc of testCases) {
const msg = await client.messages.create({
model: "claude-haiku-4-5",
max_tokens: 8,
messages: [{ role: "user", content: `Classify: ${tc.input}` }],
});
if (msg.content[0].text.trim().toLowerCase() === tc.expected) passed++;
}
console.log(`${passed}/${testCases.length} passed`);Version your prompts in code
Stores prompt versions as named constants so you can switch between them and compare scores.
const prompts = {
v1: "Classify this review as positive, negative, or neutral:",
v2: "Read the review and respond with exactly one word — positive, negative, or neutral:",
v3: "Classify the sentiment. Respond with exactly one lowercase word (positive/negative/neutral):",
};
const ACTIVE = prompts.v3; // change here to roll backLog failures for prompt debugging
Collects every failing case into an array and prints it as a table so you can see exactly where the prompt breaks.
const failures = [];
for (const tc of testCases) {
const msg = await client.messages.create({
model: "claude-haiku-4-5",
max_tokens: 8,
messages: [{ role: "user", content: `${ACTIVE}\n${tc.input}` }],
});
const got = msg.content[0].text.trim().toLowerCase();
if (got !== tc.expected) failures.push({ input: tc.input, expected: tc.expected, got });
}
console.table(failures);
Discussion