Streaming

Set "stream": true and the response arrives as server-sent events instead of one body.

curl -N https://api.cognitivers.com/v1/chat/completions \
  -H "Authorization: Bearer $COGNITIVERS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cog-fast",
    "stream": true,
    "messages": [{ "role": "user", "content": "Write a short brief." }]
  }'

data: {"choices":[{"delta":{"role":"assistant"}}]}

data: {"choices":[{"delta":{"content":"The"}}]}

data: {"choices":[{"delta":{"content":" brief"}}]}

data: {"choices":[{"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Use curl -N when testing by hand. Without it curl buffers the output and shows you nothing until the request ends, which looks exactly like a stall.

Handling the stream

Every current OpenAI client handles this already, so prefer the client over your own parser. If you are writing one, four rules cover it:

1. Ignore comment lines. A line starting with : is a keep-alive, not data.
2. Split on blank lines. One event is one or more data: lines followed by an empty line.
3. Stop at [DONE]. It is a sentinel, not JSON.
4. Concatenate deltas. Each chunk carries a fragment, and finish_reason marks the last one.

Retries

A stream can end early for reasons that have nothing to do with the model: a dropped connection, a client-side read timeout, a proxy restart. A retry is safe because requests are stateless, but only the last complete answer counts: do not append a retry to a partial answer you have already shown or stored, because the two generations are different continuations of the same prompt and the seam shows.

The pattern that behaves well: hold the answer in memory, mark it committed when finish_reason arrives, and retry the whole request on failure rather than resuming it.

Long reads

If the input is very large, the model can spend significant time before the first token. During that window the stream stays open with keep-alive lines. Send those requests to https://stream.cognitivers.com/v1; see Long context.

Usage in a stream

Ask for usage in the final chunk rather than counting tokens yourself. Counting deltas on your side drifts from what is billed, and the difference shows up as an argument at invoice time.