Streaming UX Through a Server Proxy

Stream tokens to the browser for a responsive feel — but always through your own server, so the API key never leaves the backend.

Users expect chat and long answers to type themselves onto the page. Streaming delivers that, but the browser must never talk to the Anthropic API directly — doing so would ship your API key to every visitor. The pattern is a thin server proxy.

The pipeline

The browser calls your endpoint (no key involved).
Your server calls Claude with streaming on, using the key from the environment.
As chunks arrive, your server writes them straight to the browser response.
The browser reads the response body stream and appends each chunk to the DOM.

Details that separate a demo from production

Flush as you go. Write each delta immediately; do not buffer the whole reply and defeat the point.
Handle disconnects. If the user navigates away, abort the upstream stream so you are not billed for tokens nobody will see.
Stream for big outputs anyway. Even when no human watches, streaming avoids request-timeout failures on large max_tokens values.

Example

// server.js — Express proxy. The key lives here and only here.
app.post("/api/chat", async (req, res) => {
  res.setHeader("Content-Type", "text/plain; charset=utf-8");

  const controller = new AbortController();
  req.on("close", () => controller.abort());   // client left → stop upstream

  const stream = client.messages.stream(
    {
      model: "claude-sonnet-5",
      max_tokens: 2048,
      messages: [{ role: "user", content: req.body.message }],
    },
    { signal: controller.signal },
  );

  stream.on("text", (delta) => res.write(delta));   // forward each chunk live
  try {
    await stream.finalMessage();
  } catch (e) {
    if (e.name !== "AbortError") throw e;
  }
  res.end();
});

// browser: read the streamed body and append to the page
// const reader = (await fetch('/api/chat', { method: 'POST', body })).body.getReader();
// for (;;) { const { value, done } = await reader.read(); if (done) break; append(decode(value)); }

When to use it

Stream Claude's reply through a Next.js route handler directly to the browser so the chat feels instant.
Show a live word count that increments as tokens stream in to keep the user engaged during long generation.
Implement a 'Stop generating' button that aborts the fetch stream mid-response, cutting unnecessary token spend.

More examples

Next.js streaming route handler

Wraps the Claude stream in a Web Streams ReadableStream so Next.js sends tokens to the browser as they arrive.

// src/app/api/stream/route.ts
import { NextRequest } from "next/server";
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

export async function POST(req: NextRequest) {
  const { messages } = await req.json();
  const stream = await client.messages.stream({
    model: "claude-haiku-4-5",
    max_tokens: 1024,
    messages,
  });

  return new Response(
    new ReadableStream({
      async start(controller) {
        for await (const chunk of stream) {
          if (chunk.type === "content_block_delta") {
            controller.enqueue(new TextEncoder().encode(chunk.delta.text));
          }
        }
        controller.close();
      },
    }),
    { headers: { "Content-Type": "text/plain; charset=utf-8" } }
  );
}

Browser reads stream and updates UI

Reads the streaming response chunk by chunk and appends each decoded text fragment to the output element.

async function streamChat(messages, outputEl) {
  const res = await fetch("/api/stream", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ messages }),
  });
  const reader = res.body.getReader();
  const dec = new TextDecoder();
  outputEl.textContent = "";
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    outputEl.textContent += dec.decode(value);
  }
}

Abort stream on user cancel

Attaches an AbortController to the fetch so a Stop button cancels mid-stream and appends a [stopped] marker.

const controller = new AbortController();
document.getElementById("stop").addEventListener("click", () => controller.abort());

const res = await fetch("/api/stream", {
  method: "POST",
  body: JSON.stringify({ messages }),
  headers: { "Content-Type": "application/json" },
  signal: controller.signal,
});

try {
  const reader = res.body.getReader();
  // ...read loop...
} catch (err) {
  if (err.name === "AbortError") outputEl.textContent += " [stopped]";
}

Discussion

Be the first to comment on this lesson.