Request Parameters
Common knobs: max tokens, temperature, top_p, stop sequences.
Beyond model and messages, a handful of parameters shape the output.
| Parameter | What it does |
|---|---|
| max_tokens | Caps the length of the reply. Protects against runaway cost. |
| temperature | Randomness of word choice (0 = focused, high = creative). |
| top_p | Nucleus sampling; an alternative to temperature. |
| stop | Strings that force generation to end when produced. |
Example
{
"model": "llm-medium",
"messages": [{"role": "user", "content": "List 3 fruits, one per line."}],
"temperature": 0.2,
"max_tokens": 60,
"stop": ["\n\n"]
}When to use it
- A customer chatbot sets max_tokens=150 and temperature=0.4 to keep replies concise and consistent across thousands of daily interactions.
- A creative writing tool sets top_p=0.95 and temperature=1.1 to produce varied, imaginative story continuations.
- A code-completion feature uses stop=["\n\n"] so the model halts as soon as it finishes the current function block.
More examples
Set temperature and max_tokens
Shows the two most-used parameters: temperature for randomness and max_tokens to limit output length.
from openai import OpenAI
client = OpenAI()
r = client.chat.completions.create(
model='gpt-4o-mini',
messages=[{'role': 'user', 'content': 'Describe a sunset.'}],
temperature=0.8,
max_tokens=80
)
print(r.choices[0].message.content)Use a stop sequence
Demonstrates a stop sequence — generation halts when the model emits the specified token pattern.
from openai import OpenAI
client = OpenAI()
r = client.chat.completions.create(
model='gpt-4o-mini',
messages=[{'role': 'user', 'content': 'List colours one per line:'}],
stop=['\n\n'] # stop after the first blank line
)
print(r.choices[0].message.content)
print('Finish reason:', r.choices[0].finish_reason)Frequency and presence penalties
Uses frequency_penalty and presence_penalty to discourage repetition and encourage topic diversity.
from openai import OpenAI
client = OpenAI()
r = client.chat.completions.create(
model='gpt-4o-mini',
messages=[{'role': 'user', 'content': 'Write a paragraph about the sky.'}],
frequency_penalty=0.6, # reduce repeating the same words
presence_penalty=0.3 # encourage introducing new topics
)
print(r.choices[0].message.content)
Discussion