# Streaming (/guides/streaming)
Set `stream: true` and Zumik returns a server-sent event stream of OpenAI-shaped chunks. Tokens
arrive as the provider produces them - on every provider, including Anthropic and Gemini - with a
clean terminator and (when you ask for it) a trailing usage chunk with the exact token counts you
were billed for.
## Stream a chat completion [#stream-a-chat-completion]
```python title="Python"
stream = client.chat.completions.create(
model="code.fast",
messages=[{"role": "user", "content": "Explain the diff."}],
stream=True,
stream_options={"include_usage": True},
)
for chunk in stream:
if chunk.choices:
print(chunk.choices[0].delta.content or "", end="")
elif chunk.usage:
print("\ncached:", chunk.usage.prompt_tokens_details.cached_tokens)
```
```bash title="cURL"
curl -N https://api.zumik.ai/v1/chat/completions \
-H "Authorization: Bearer $ZUMIK_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "code.fast",
"messages": [{"role": "user", "content": "Explain the diff."}],
"stream": true,
"stream_options": {"include_usage": true}
}'
```
A streamed response sets `Content-Type: text/event-stream` and `Cache-Control: no-cache`, and the body
is a sequence of `data:` lines.
## The chunk shape [#the-chunk-shape]
Each event is `data: ` followed by a JSON object whose `object` is `chat.completion.chunk`. The frames
arrive in this order:
Opens the message. The delta carries only the role.
```json
{"id":"chatcmpl-…","object":"chat.completion.chunk","created":1750000000,"model":"code.fast",
"choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}
```
One or more frames carrying text in `delta.content`. Tool calls stream the same way:
`delta.tool_calls` carries the call's `id`, `type`, and function name first, then argument
string fragments, all keyed by `index` so your client can accumulate them.
```json
{"id":"chatcmpl-…","object":"chat.completion.chunk","created":1750000000,"model":"code.fast",
"choices":[{"index":0,"delta":{"content":"The patch "},"finish_reason":null}]}
```
An empty delta with a `finish_reason` (`stop`, `length`, `tool_calls`, or `content_filter`).
```json
{"id":"chatcmpl-…","object":"chat.completion.chunk","created":1750000000,"model":"code.fast",
"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
```
Emitted only when `stream_options.include_usage` is `true`. `choices` is empty and `usage` carries
the final counts, including cached input.
```json
{"id":"chatcmpl-…","object":"chat.completion.chunk","created":1750000000,"model":"code.fast",
"choices":[],
"usage":{"prompt_tokens":1240,"completion_tokens":180,"total_tokens":1420,
"prompt_tokens_details":{"cached_tokens":1024}}}
```
The stream ends with the literal sentinel.
```text
data: [DONE]
```
## stream\_options.include\_usage [#stream_optionsinclude_usage]
By default a streamed response carries no usage object - matching OpenAI. Set
`stream_options.include_usage: true` to receive one trailing chunk after the finish frame whose
`usage` block reports `prompt_tokens`, `completion_tokens`, `total_tokens`, and
`prompt_tokens_details.cached_tokens`. The cached count is how you confirm
[prompt-cache capture](/guides/prompt-caching) on a streamed call.
## Token-by-token relay [#token-by-token-relay]
Streaming is a true relay by default: Zumik opens the provider's own streaming endpoint and pipes
chunks through as they arrive, so time to first token is the provider's, not the full generation
time. OpenAI, xAI, and Fireworks already speak the OpenAI chunk dialect and relay verbatim.
Anthropic (Messages API) and Gemini (`streamGenerateContent`) stream in their own SSE dialects, and
Zumik translates them frame by frame into the same `chat.completion.chunk` shape - text deltas,
tool-call deltas, finish reasons, and all - so your client never sees the difference.
Billing stays exact. Before the stream opens, the request is pre-checked against your budget for
the worst case (input plus the requested `max_tokens` at the published rate), so a near-empty
balance is rejected as a normal error rather than cut off mid-stream. The actual charge lands once
the stream closes, computed from the provider's own reported usage - the same numbers the trailing
usage chunk shows you.
Two paths still fall back to buffered delivery (the full completion fetched, charged, then
re-emitted as chunks): a provider route with no usable streaming key, and deployments that set
`ZUMIK_STREAM_RELAY=0` to opt out of the relay entirely.
Budget and rate-limit rejections surface before the first byte, as a normal error response - never
as a half-streamed body that gets cut off. The
[retry rules](/guides/idempotency-and-retries) still apply: do not auto-replay a generation after
observable streamed output unless the path supports resumability.
## On /v1/responses and /v2/responses [#on-v1responses-and-v2responses]
The OpenAI-compatible [`/v1/responses`](/api-v1/responses) supports `stream: true` with the
Responses SSE event protocol (`response.created` through `response.completed`, including
`response.output_text.delta` and function-call argument deltas) — see
[its streaming section](/api-v1/responses#streaming) for the event sequence. Under the hood it
resolves and charges the generation, then emits the events; for the lowest time-to-first-token
use `/v1/chat/completions` with `stream: true`, which relays provider tokens live. The native
[`/v2/responses`](/api-v2/responses) stays buffered and adds `execution_profile` and a formal
[`qos_outcome`](/guides/qos-outcomes) to the response object.
Read `cached_tokens` from the usage chunk to see how much of the prefix was reused.
Safe retries around a stream that disconnects.