Cost optimization: how to count tokens and save

The LLM bill almost never grows where you expect. People optimize the model while the money leaks into bloated context and needless calls. Let's look at how to count and where to cut. A reminder: 1 credit = 1 ₽, so the whole calculation is straight in rubles.

What makes up the price

You pay for tokens: input (prompt, history, context) and output (the answer, and for reasoning models the thinking as well). Output is usually several times more expensive than input. Start from facts, not guesses:

u = resp.usage
print(u.prompt_tokens, u.completion_tokens)

Log usage for every call with a scenario tag (bot, autocomplete, report). Without that breakdown you do not know what to optimize.

Where the money leaks

• Bloated context. In RAG, people put in 30 chunks "just in case". Eight precise ones give the same answer at a quarter of the cost.
• The whole dialog history every time. A long chat sends the entire exchange with every request. Trim it to the last N turns plus a summary of the older part.
• A flagship on a simple task. Classifying a ticket on deepseek-pro instead of qwen-flash is an order of magnitude overpayment for the same result.
• Reasoning where it is not needed. high generates thousands of billable thinking tokens for nothing.
• Long answers. No max_tokens limit, so the model writes a wall of text and you pay for every token.

Matching the model to the task is the main lever

The price difference between Flash and Pro models is usually 5 to 20x. Distributing tasks across models correctly saves more than any micro-optimization of the prompt:

• Mass inference (classification, extraction, bots): Flash.
• Complex logic, important texts, refactoring: Pro.
• Embeddings: a separate cheap model (bge-m3); do not spend a chat model on them.

Caching

Two levels. Your own cache keyed by prompt hash: for repeating requests (the same questions to a bot, the same prefixes in autocomplete), return the saved answer without calling the model at all. And prompt caching on the API side, where supported: a long unchanging system prompt is cached and billed cheaper on repeat calls, so keep the static part at the beginning.

Trimming context

def trim_history(messages, keep=8):
    system = [m for m in messages if m["role"] == "system"]
    rest = [m for m in messages if m["role"] != "system"]
    return system + rest[-keep:]

Simple trimming to the last turns covers most chats. For long ones, every N turns compress the cut-off part into a summary with a cheap model and keep it in the system prompt.

Limit the output

max_tokens is the most underrated lever. Output tokens are expensive; set a reasonable ceiling for the task (300 for a bot, 1500 for a report) and ask the model to be brief in the prompt. "Answer in one paragraph" saves real money.

Count on real load

"On average" estimates lie. Take a day of logs from the working system with usage, compute the cost by scenario, and find the top 3 expense items. Almost always 2 or 3 scenarios produce 80% of the bill: optimize those and leave the rest alone. 1 credit = 1 ₽, so the logs show rubles directly, with no conversion.