Long context

Both chat models take 262,144 tokens of context. A prompt that size is a different kind of request from a chat message: the model reads for a long time before it writes anything, and the things that break during that read are not the model's fault.

Use the streaming host

curl https://stream.cognitivers.com/v1/chat/completions \
  -H "Authorization: Bearer $COGNITIVERS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cog-fast",
    "stream": true,
    "messages": [{ "role": "user", "content": "<200K tokens of logs>" }]
  }'

Same key, same body, same response shape as api.cognitivers.com. The difference is how the connection is held while the prompt is being read.

Keep-alives are not noise

While the model reads, nothing is generated yet, so a connection can look idle to everything between you and us: a load balancer, a proxy, a corporate network appliance, a client library with a read timeout. The API sends periodic comment lines during that window, and every conforming server-sent-event client ignores them.

Two practical consequences:

• If you wrote your own SSE parser, skip lines that begin with a colon. Treating one as data is the most common self-inflicted failure here.
• If you are behind a proxy of your own, do not buffer the response. A proxy that waits for a complete body before relaying it will time out on a long read.

Choose the model by what it reads

cog-fast is the long-context model by design. For a very large input it is the one that reads cheapest and fastest, and it serves the whole 262,144-token window on every plan.

cog-pro reads less: 64K on pay-as-you-go, 128K on Builder, the full window on Pro and above. Its input ceiling is a commercial term rather than a hardware limit, and it is in Models and prices.

What to do when a request is too big

1. Send it to the streaming host if you are not already.
2. Reuse the prefix. If you resend the same long system prompt or document on every turn, keep it byte-identical at the start of the message list. A cached prefix reads at a tenth of the input rate.
3. Split by meaning, not by count. Two requests over two coherent halves of a document usually beat one request over the whole thing, and they fail independently.
4. Summarise once, then reuse. If a corpus will be queried repeatedly, embed it with cog-embed and retrieve passages per request instead of resending the corpus.

Verifying that a long request works

Before building a pipeline on the assumption, send the real shape once, from the environment that will run it, through whatever proxy sits in between. A 200K-token request that works from a laptop and fails from a container behind an egress proxy is a proxy configuration problem, and it is much cheaper to find that out now than during a batch run.