Why Python Runs AI
What the Python AI ecosystem gives an application developer, and where using a model ends and training one begins.
client = anthropic.Anthropic() # credentials resolved from the environmentPython runs AI work for two unrelated reasons, and it helps to keep them apart. The training stack (PyTorch, JAX, NumPy) is C, C++ and CUDA with a Python surface, because researchers wanted a scripting language on top of fast kernels. The application stack is just HTTP: a JSON request, a JSON response, and the vendor's SDK wrapping both. You are almost certainly working in the second one.
What you actually need
As an application developer, the parts of the ecosystem you touch are small. One SDK (pip install anthropic), json for payloads, asyncio for fan-out, logging for the audit trail, and the error handling you would write for any flaky network dependency. You do not need NumPy to call a model. Most of the skill is ordinary backend engineering: timeouts, retries, idempotency, cost accounting, and keeping non-deterministic output away from code paths that must be deterministic.
Using a model is not training one
Calling an API means you own the prompt, the parsing and the failure modes. Fine-tuning or training means you own data collection, GPUs, evaluation and drift. The second is a different job with different costs, and almost no product needs it before it has exhausted prompting and retrieval.
What trips people up
The model is an unreliable remote service that charges per call and sometimes returns prose where you expected JSON. Treat it as an integration boundary, not as a library function, and wrap it behind one module of your own so the rest of the codebase never imports the SDK directly.
Example
When to use it
- Deciding whether a feature needs a model call at all or just a regular expression
- Explaining to a team why the AI work is mostly ordinary integration engineering
- Choosing between prompting, retrieval and fine-tuning before any code is written
- Isolating a vendor SDK behind one module so the provider can be swapped
More examples
The same call through the SDK
The real call, for comparison with the hand-built dict. The SDK returns objects rather than dicts, so blocks are reached with attributes (b.type, b.text) instead of keys. Reach for this once your credentials are in place; everything in the runnable example above is the same request with the network removed.
import anthropic
client = anthropic.Anthropic() # reads credentials from the environment
response = client.messages.create(
model="claude-opus-5-5",
max_tokens=1024,
system="Answer in one sentence.",
messages=[{"role": "user", "content": "What is a token?"}],
)
print(next(b.text for b in response.content if b.type == "text"))One seam for the whole provider
A Protocol plus two implementations is the cheapest way to keep the SDK out of your business logic. Tests inject StubCompleter and run offline in milliseconds; production injects AnthropicCompleter. Do this on day one, because retrofitting it after the SDK has spread through twenty modules is a refactor nobody schedules.
# app/llm.py - the only module that imports the SDK
from typing import Protocol
class Completer(Protocol):
def complete(self, system: str, user: str) -> str: ...
class AnthropicCompleter:
def __init__(self, client, model="claude-opus-5-5"):
self._client, self._model = client, model
def complete(self, system: str, user: str) -> str:
reply = self._client.messages.create(
model=self._model, max_tokens=1024, system=system,
messages=[{"role": "user", "content": user}],
)
return "".join(b.text for b in reply.content if b.type == "text")
class StubCompleter:
"""Used by every test that is not the one smoke test."""
def complete(self, system: str, user: str) -> str:
return "stubbed: " + user
Discussion