InferenceCapabilities

Streaming

How to receive the model's reply in parts while it is being written, instead of waiting for the whole thing.

With streaming, the reply arrives in parts, as in a chat: the first words appear right away rather than after the model has finished the whole text. This is convenient when a person reads the reply and waiting for all of it is unpleasant. To turn on streaming, add "stream": true to the request.

Request

curl https://api.yenisei.ru/v1/chat/completions \
  -H "Authorization: Bearer $YENISEI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "yenisei-100",
    "messages": [{"role": "user", "content": "Tell me about the Yenisei River"}],
    "stream": true
  }'

# client is set up as on the "First request" page
stream = client.chat.completions.create(
    model="yenisei-100",
    messages=[{"role": "user", "content": "Tell me about the Yenisei River"}],
    stream=True,
)
for chunk in stream:
    part = chunk.choices[0].delta.content
    if part:
        print(part, end="", flush=True)

Stream format

The response arrives as Server-Sent Events: each data: {…} line contains the next piece of text in choices[].delta.content, and the last line is data: [DONE].

Dropped connection

While the stream is running, the connection stays open. If it drops midway, the part of the reply you have already received has been charged to your credit balance. To continue the conversation, send a new request and add the text you received to messages.

Back

Models

Tool calling