Tokens, Context and Cost
What a token is, how a context window fills, counting before you send, and what a feature costs per thousand calls.
client.messages.count_tokens(model="claude-opus-5-5", messages=messages).input_tokensA token is a chunk of text the model treats as one unit, usually a short word or a word fragment. You are billed per token in both directions, and the context window is a ceiling on the total: system prompt plus every message in the history plus tool definitions plus the space reserved for the reply.
Counting before you send
Estimates from character counts are fine for a guard rail and wrong for a budget. The accurate number comes from the provider: client.messages.count_tokens(model=..., messages=..., system=...) returns input_tokens for exactly the request you are about to make. It is model-specific, so pass the same model id you will call, and it is a separate request, so do not run it on the hot path for every call.
Truncating the right end
When a conversation outgrows its budget, drop from the oldest end. The system prompt stays, the newest user turn stays, and the turns in between go first; dropping the newest turn is the one mistake that makes the model answer the wrong question. Keep user and assistant turns paired when you drop them, or the history reads as two users talking.
Working out the cost
Prices are quoted per million tokens, input and output separately, and output is several times the price of input. Cost per call is (in_tokens * in_price + out_tokens * out_price) / 1_000_000. Multiply by the call volume before you commit to a feature: a long system prompt resent on every turn of a chat is usually the largest line on the bill.
What trips people up
max_tokens is reserved out of the window, not added to it. Setting it to the window size leaves no room for the prompt and the request is rejected.
Example
When to use it
- Rejecting an oversized upload before it is sent to the model
- Deciding how many retrieved chunks fit in one prompt
- Putting a per-call price on a feature during planning
- Trimming a long chat history without breaking the conversation
More examples
The authoritative count
count_tokens prices the exact request you are about to send, including the system prompt and tool definitions. It is the right tool for an admission check at the edge of a job, and for measuring whether a prompt edit made the system prompt 200 tokens heavier on every call.
SYSTEM = open("prompts/triage.md").read()
count = client.messages.count_tokens(
model="claude-opus-5-5", # counts are model-specific
system=SYSTEM,
messages=[{"role": "user", "content": ticket_body}],
)
print(count.input_tokens)
# Use it to refuse oversized work before paying for it, not on every call.
if count.input_tokens > 150_000:
raise ValueError("document too large; chunk it first")Budgeting the window explicitly
How a retrieval prompt decides how much context it can afford. Everything is subtracted from one window: the reply reservation, the system prompt, the conversation, framing overhead. Filling greedily from the best chunk down means a tight budget loses the least relevant material first.
WINDOW = 1_000_000 # your model's context window - look it up, they differ by an
# order of magnitude and change between releases
MAX_REPLY = 4_000 # reserved out of the window, not added to it
def room_for_context(system_tokens, history_tokens):
"""Tokens left for retrieved documents after everything else is placed."""
overhead = 500 # tool definitions and message framing
return WINDOW - MAX_REPLY - system_tokens - history_tokens - overhead
budget = room_for_context(1_200, 8_400)
chunks = []
for chunk in ranked_chunks: # best first
if chunk.tokens > budget:
break # stop at the first one that will not fit
chunks.append(chunk)
budget -= chunk.tokensWhat a chat feature actually costs
The quadratic growth people are surprised by: because the API is stateless, turn ten pays for turns one through nine again. This is the model to run before launch, and the argument for prompt caching, shorter system prompts, or summarising old turns instead of resending them.
PRICE = {"in": 4.00, "out": 20.00} # USD per million tokens
def chat_cost(system_tokens, turns, user_tokens, reply_tokens):
"""A stateless API resends the whole history, so input grows per turn."""
total_in = total_out = 0
history = system_tokens
for _ in range(turns):
history += user_tokens
total_in += history # the full prompt, every turn
total_out += reply_tokens
history += reply_tokens
return (total_in * PRICE["in"] + total_out * PRICE["out"]) / 1_000_000
print(f"${chat_cost(1500, 10, 60, 250):.4f} per 10-turn conversation")
print(f"${chat_cost(1500, 10, 60, 250) * 10_000:,.0f} per 10k conversations")
Discussion