Temperature & Sampling
Temperature controls how random or focused the model's word choices are.
Temperature is a number (usually 0 to 2) that controls how the model picks among likely next tokens.
- Low (0 - 0.3) — focused and repeatable. The model almost always takes the most likely token. Best for facts, code, and extraction.
- Medium (0.7 - 1.0) — balanced. Good default for chat and general writing.
- High (1.2+) — creative and varied, but more likely to wander or make mistakes.
Related: top_p
top_p (nucleus sampling) is another knob: it keeps only the smallest set of tokens whose probabilities add up to p, then samples from those. Tune temperature or top_p, not both.
Example
{
"model": "llm-medium",
"messages": [{"role": "user", "content": "Write a tagline for a coffee shop."}],
"temperature": 0.9,
"top_p": 1.0
}When to use it
- A legal document drafter sets temperature=0 to get deterministic, consistent clause wording across every run.
- A creative ad-copy tool sets temperature=1.2 to produce varied, surprising taglines for A/B testing.
- A customer-service bot uses temperature=0.3 to stay on-topic while still avoiding robotic repetition.
More examples
Deterministic output at temperature 0
Runs the same prompt three times with temperature=0 to confirm identical, deterministic output.
from openai import OpenAI
client = OpenAI()
for _ in range(3):
r = client.chat.completions.create(
model='gpt-4o-mini',
messages=[{'role': 'user', 'content': 'What is 2+2?'}],
temperature=0
)
print(r.choices[0].message.content)Varied creative output at high temperature
Runs the same creative prompt three times at high temperature to show different outputs each time.
from openai import OpenAI
client = OpenAI()
for _ in range(3):
r = client.chat.completions.create(
model='gpt-4o-mini',
messages=[{'role': 'user', 'content': 'Suggest a one-word colour to paint a spaceship.'}],
temperature=1.4
)
print(r.choices[0].message.content)Nucleus sampling with top_p
Combines temperature with top_p nucleus sampling to balance creativity with coherence.
from openai import OpenAI
client = OpenAI()
r = client.chat.completions.create(
model='gpt-4o-mini',
messages=[{'role': 'user', 'content': 'Write a tagline for a coffee brand.'}],
temperature=0.7,
top_p=0.9 # only sample from top 90% probability mass
)
print(r.choices[0].message.content)
Discussion