A million tokens of context: how not to drown in it, using Nova Premier as an example

With a 1M-token window like Nova Premier from Amazon, the temptation is to dump the entire corpus in and start asking questions. That is almost never a good idea. A large context is switched on for a specific task, not by default: it has a price and its own traps.

Nova Premier is not in the Yenisei catalog. Our models with a window of around a million tokens are Yenisei Max (yenisei-max), DeepSeek V4 Pro (deepseek-pro), Kimi K3 (kimi-k3), GLM 5.3 (glm-5.3), Llama 4 Scout (llama-4-scout). Everything below applies to any of them.

What a million really is

A million tokens is roughly 700 to 750 thousand words, that is, a hefty book or a couple hundred PDF pages. The model can hold that in one request, but there are three caveats:

• You pay for every input token. Fill the window to the brim on every call and you get a bill proportional to that volume, multiplied by the number of requests.
• Attention is not perfect. Even good models find a fact buried in the middle of a huge canvas less reliably than the same fact at the beginning or the end.
• Latency grows. The larger the input, the longer the wait for the first token.

When a large window is justified

• One-off analysis of a whole document where the connections are spread across the entire text: a contract with cross-references between clauses, a long technical report, a module's codebase.
• Analysis with images: text plus diagrams and screenshots in one context. This needs a model with image input, such as Yenisei Max or Kimi K3.
• "Find every mention of X and consolidate" tasks, where it is unclear in advance where the relevant parts are.

When RAG is better

If you have a knowledge base that receives hundreds of similar questions, stuffing a million tokens into every request is wasteful. It is cheaper and more accurate to build RAG: split the corpus into chunks, index them with embeddings (BGE-M3, bge-m3), and pull only the relevant pieces into the context. Then the model receives not a million tokens but five to ten thousand that matter.

from openai import OpenAI
client = OpenAI(
    base_url="https://api.yenisei.ru/v1",
    api_key="YENISEI_KEY",
)
resp = client.chat.completions.create(
    model="deepseek-pro",
    messages=[
        {"role": "system", "content": "Answer strictly from the attached fragments. If the answer is not there, say so."},
        {"role": "user", "content": f"Question: {question}\n\nFragments:\n{retrieved_chunks}"},
    ],
)
print(resp.choices[0].message.content)

Practical techniques

• Important things at the edges. Put the instruction and the key question at the beginning and repeat them at the end. Ballast goes in the middle.
• Ask for source references. "State which fragment the fact comes from" sharply reduces fabrication.
• Do not repeat what does not change. If the system prompt and the reference material are the same between requests, consider whether they need to be included in full on every call.
• Measure. Run your task at 50K and at 500K of context and compare the quality. Often the difference is smaller than the difference in the bill.

Bottom line

A million-token window is for tasks where the context physically cannot be cut into pieces without losing meaning. Everything else is solved cheaper and more reliably with the combination "embeddings, careful retrieval, and a smaller model". Keep the big window in reserve rather than switching it on out of habit.