Reasoning models: when to turn on reasoning_effort

Reasoning models build an internal chain of thought before answering. This noticeably raises quality on tasks with logic, math, planning, and complex code. The price is latency (the model "thinks" for seconds) and cost (reasoning tokens are billed too).

The reasoning_effort parameter

The level of effort is adjustable. Usually low | medium | high are available:

resp = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[{"role": "user", "content": "Work out why this algorithm is O(n^2) and propose an O(n log n) version."}],
    reasoning_effort="high",
)

• low: brief thinking, faster and cheaper. For tasks that need a little thought but not a long chain.
• medium: the balance, a sensible default for complex requests.
• high: maximum effort: olympiad math, untangling convoluted logic, multi-step planning. Expensive and slow.

When reasoning pays off

• Math and multi-step calculations.
• Debugging a non-obvious bug where you have to trace the data flow.
• Planning: breaking a large task into steps with dependencies.
• Constraint problems where options have to be enumerated without violating the conditions.

When it is unnecessary

• Field extraction, classification, translation. There is nothing to think about; reasoning only makes the answer slower and more expensive. Take an ordinary model with temperature=0.
• Content generation, support chat. A templated answer gains nothing from deliberation.
• RAG answers from ready context. The model is retelling what was found, not solving a problem.
• Simple tool calls. Calling a function by name needs no reasoning.

Putting high on a support bot is like hiring a professor to answer "where is my order". Expensive and slow, with no gain.

Cost: count the reasoning tokens

The main budget trap: reasoning is generated as output tokens and billed even though you do not see it. On high, the model may "think through" several thousand tokens before a short answer. So the bill grows not in proportion to the answer's length but in proportion to the depth of thinking.

print(resp.usage.completion_tokens_details.reasoning_tokens)

Log reasoning_tokens and see what you are paying for. 1 credit = 1 ₽; at volume, the difference between medium and high hits the bill hard.

A practical strategy

• In your application, default to an ordinary model without reasoning.
• Turn reasoning on selectively, for a specific type of request, not globally.
• Start with medium and move up to high only if medium genuinely falls short on your tasks; verify on a sample, not on a hunch.
• Set a timeout and show the user a "thinking" indicator: several seconds of silence without explanation look like a hang.

Reasoning is needed for a narrow range of tasks. Most requests in a typical product do not need it, and high left on "just in case" is simply paying extra for latency.