Reliable Structured Output
Get JSON your code can trust every time by combining schema-constrained output with real validation.
Asking a model to "return JSON" and then calling JSON.parse on the result works in the demo and fails in production. A senior approach treats model output as untrusted input and builds two layers of defense.
Layer 1: constrain the generation
Most modern APIs let you enforce structure at generation time — a JSON mode or a schema parameter that guarantees syntactically valid JSON matching the shape you describe. Use it. It removes an entire class of "the model added a chatty sentence before the JSON" bugs. Pair it with a low temperature for consistency.
Layer 2: validate against a schema
Valid JSON is not the same as correct data. The model can still return an empty string where you expected an email, a number as a string, or a status outside your allowed set. Validate the parsed object against a schema (JSON Schema, Zod, Pydantic — whatever your stack uses) before trusting a single field.
What to do on failure
- Retry once or twice — a low-temperature retry often succeeds.
- Repair — feed the validation error back to the model and ask it to fix its output.
- Fall back — return a safe default and log the raw output so you can improve the prompt.
Never let unvalidated model output flow straight into a database write, an API call, or a UI. The validator is the boundary between "the model suggested" and "the system did."
Example
import { z } from "zod";
// One schema, reused everywhere.
const Contact = z.object({
name: z.string().min(1),
email: z.string().email(),
plan: z.enum(["free", "pro", "enterprise"]),
});
async function extractContact(text) {
for (let attempt = 0; attempt < 2; attempt++) {
const res = await callLLM({
model: "small-fast-model",
temperature: 0,
// Ask the API to CONSTRAIN output to the schema (JSON mode / schema param).
responseFormat: { type: "json_schema", schema: zodToJsonSchema(Contact) },
messages: [{ role: "user", content: `Extract the contact:\n${text}` }],
});
// Layer 2: valid JSON is not correct data — validate before trusting it.
const parsed = Contact.safeParse(JSON.parse(res.text));
if (parsed.success) return parsed.data;
// else: retry once; on final failure, log res.text and return a safe default.
}
throw new Error("Could not extract a valid contact");
}When to use it
- A data extraction pipeline uses JSON mode so every invoice-parsing response is machine-readable without fragile regex post-processing.
- A form-filling agent validates each API response against a Pydantic model before inserting it into the database, catching schema violations early.
- A content API returns structured output so the front end can render title, body, and tags from a single AI call without string parsing.
More examples
JSON mode guarantees valid JSON
Enables JSON mode so the API guarantees valid JSON output — parsing never raises a JSONDecodeError.
import json
from openai import OpenAI
client = OpenAI()
r = client.chat.completions.create(
model='gpt-4o-mini',
messages=[
{'role':'system','content':'You are a JSON API. Always return valid JSON.'},
{'role':'user','content':'Extract name and email from: "Contact Alice at [email protected]"'}
],
response_format={'type': 'json_object'}
)
data = json.loads(r.choices[0].message.content)
print(data['name'], data['email'])Validate output with Pydantic
Parses the JSON into a Pydantic model so missing or wrong-typed fields raise a clear ValidationError.
import json
from pydantic import BaseModel, ValidationError
from openai import OpenAI
class ContactInfo(BaseModel):
name: str
email: str
phone: str | None = None
client = OpenAI()
r = client.chat.completions.create(
model='gpt-4o-mini',
messages=[
{'role':'system','content':'Return JSON only.'},
{'role':'user','content':'Extract: "John Doe, [email protected], +1-555-0100"'}
],
response_format={'type':'json_object'}
)
try:
contact = ContactInfo(**json.loads(r.choices[0].message.content))
print(contact)
except ValidationError as e:
print('Schema error:', e)Retry on invalid JSON
Retries the call up to N times if the output cannot be parsed, providing a safety net against occasional malformed JSON.
import json
from openai import OpenAI
client = OpenAI()
def reliable_json_call(prompt, retries=3):
for attempt in range(retries):
r = client.chat.completions.create(
model='gpt-4o-mini',
messages=[
{'role':'system','content':'Return only valid JSON, nothing else.'},
{'role':'user','content':prompt}
],
response_format={'type':'json_object'}
)
try:
return json.loads(r.choices[0].message.content)
except json.JSONDecodeError:
if attempt == retries - 1:
raise
return {}
result = reliable_json_call('Give me a JSON object with key "status" set to "ok".')
print(result)
Discussion