Extended thinking: architecting for Claude's reasoning mode
Extended thinking is a mode some Claude models support in which the model works through a problem step by step before producing its final answer. It's easy to treat as a flag to turn on for "better" answers, which leads to the most common mistake: turning it on everywhere and calling it a quality upgrade. It's an architecture decision with a latency and cost profile, and the engineering problem is deciding exactly which requests benefit enough to justify the cost.
1. What extended thinking changes
In a direct-answer interaction, the model reads a prompt and produces a response; any intermediate reasoning happens implicitly. With extended thinking, the model first produces an explicit reasoning phase (working through the problem, considering possibilities, checking intermediate steps against each other) and then produces its final response. The reasoning is a distinct, visible phase before the answer.
This matters most for problems with multi-step logical structure, where reaching the right answer depends on working through intermediate steps in order and an early mistake propagates into a wrong result. Examples: a complex debugging task, where the cause must be narrowed through a chain of "if this were the cause, we'd also see that"; a multi-constraint planning problem, where a solution that satisfies one requirement may violate another; a proof-shaped question, where each step must follow from the last. For these, explicit reasoning helps because the difficulty lives in the reasoning chain.
It does nothing for a much larger class of request: a simple lookup, a straightforward rewrite, a short generation, a question with a single direct answer. Extracting a phone number from a paragraph, drafting a one-line status update, or answering a factual question the model already knows doesn't get more accurate with extra reasoning steps, because there's no multi-step structure to improve. The architectural test is specific: does the task have intermediate structure that working through step by step measurably improves the odds of getting right?
2. The latency and cost tradeoff this introduces
Extended thinking adds time and tokens compared with a direct answer, and neither is negligible at production volumes. The reasoning phase runs before the final response starts, so a user or downstream pipeline step waits longer. The reasoning content consumes tokens on every request it's applied to.
Neither cost rules extended thinking out. Both mean you should choose where to apply it. The mistake is defaulting it on for every request, whether it's the multi-step kind from section 1 or a simple case. That system pays the full latency and token cost everywhere while only a fraction of requests need deeper reasoning. The rest get slower and more expensive with no accuracy gain.
The rule: apply extended thinking selectively, to the subset of requests that benefit from deeper reasoning. Spend the latency and token budget where it buys something and skip it everywhere else.
This is also a measurement problem. Whether a request benefits isn't obvious from the request; check it against outcomes. Does routing a slice of traffic through extended thinking improve the metric that matters (fewer follow-up corrections, a higher pass rate on a downstream check, fewer escalations to a human) enough to justify its latency and token cost on that slice? Turning it on for a category and never checking for the accuracy gain is the same as shipping a change and never checking whether it helped. Make the trade only where the evidence says it pays.
3. The routing problem: deciding what qualifies
Selective use creates a design problem a global setting never had: deciding, before calling the model, whether a request is complex enough. This is a classification problem, and there are three structurally different ways to solve it, each with its own cost, accuracy, and complexity.
A cheap upfront classifier. Run a fast, inexpensive check (a small model call, a rule over request features, or a simpler heuristic model) that predicts whether the request is the multi-step kind, then route accordingly. It adds a little latency and cost to every request in exchange for a per-request decision. It's only as accurate as the classifier: a weak one either sends easy requests to extended thinking (the waste this section exists to avoid) or misses hard ones and sends them down the fast path, where they underperform.
A fixed rule based on request type. If the system already categorizes requests by type (a support ticket tagged "billing dispute" versus "password reset," a coding-agent step tagged "diagnose failure" versus "run a lint check"), that category is often a reasonable proxy for complexity and needs no separate classifier. It's cheaper to build and reason about than a learned classifier, but only as good as the correlation between category and complexity. A category that's usually simple but occasionally holds a hard case will under-serve that case every time.
A two-pass, escalate-on-failure approach. Try a fast direct answer first. Escalate to extended thinking only if that attempt fails an explicit check or the model reports low confidence. This avoids paying for extended thinking on requests that turn out to be easy, including ones a classifier would have wrongly flagged. The cost: escalated requests pay the first pass's latency too, so the worst case is slower than going straight to extended thinking. It also needs a reliable way to detect that the first attempt failed or is uncertain, which is its own hard problem.
None of the three wins in general. A classifier trades a small universal cost for a direct decision; a fixed rule trades accuracy for simplicity; a two-pass approach trades worst-case latency for fewer unnecessary extended-thinking calls. The right choice depends on how expensive a wrong routing decision is in each direction for your system, and how well request types already track complexity.
| Approach | Fits best when | Main cost |
|---|---|---|
| Cheap upfront classifier | High request volume, complexity hard to infer from metadata alone | Adds latency/cost to every request; only as good as the classifier |
| Fixed rule by request type | Request categories already correlate well with complexity | Under-serves the occasional hard case hiding in an easy category |
| Two-pass, escalate on failure | Most requests are easy and a fast first attempt is cheap | Worst case pays first-pass cost plus escalation; needs reliable failure/low-confidence detection |
The three fail differently when they route wrong. A classifier that under-flags complexity sends a hard request down the fast path, and it comes back confidently wrong with no sign anything was skipped, the same silent failure as an ordinary tool-use mistake. A fixed rule that miscategorizes fails the same silent way, for every request in the miscategorized slice. A two-pass approach fails visibly: its worst case is slow, not silently wrong. In a system still being tuned, that's often the more recoverable failure, though it's the more expensive one at the tail.
4. Extended thinking inside agentic loops
The same selectivity applies inside an agent loop, where a single setting for the whole session is the easy default. A multi-step agent (one that plans, calls tools, reads results, and decides what to do next) isn't uniformly hard across its steps. Some steps need multi-step reasoning: recovering from an unexpected tool failure, working out which of several plausible root causes explains an error, choosing between two conflicting pieces of evidence. Others are routine: making a tool call whose arguments are already determined, checking whether a status field equals a known value, formatting a result for the next step.
Apply extended thinking to the reasoning-heavy steps only. Turning it on for the whole loop pays for deep reasoning on every routine tool call, most of which gain nothing. Turning it on for the steps that need it (the diagnosis step, not the status check) captures the accuracy benefit without paying for it everywhere. This is section 3's principle at the granularity of loop steps instead of requests.
5. The transparency angle, and its limit
A surfaced reasoning trace gives a human reviewer something to check besides a bare final answer. It helps debugging: when a conclusion looks wrong, the trace shows where the reasoning went off track (a wrong early assumption, evidence overweighted, a step that skipped a constraint) instead of leaving the reviewer to guess. For diagnostic or planning work, it turns "the model got this wrong" into "the model got this wrong at this step, for this reason," a much better place to start a fix.
A coherent, well-structured trace is still not verification that the conclusion is correct. AI evaluation methods makes the same point about fluent wrong answers: fluency and structure show the model produced something that reads well, nothing more. A trace can walk through several plausible steps to a wrong conclusion, and a confident, organized trace is more persuasive on the way to being wrong than a bare wrong answer, because it looks like work was shown. Use the trace as a debugging aid and a starting point for the reviewer's own checking. Don't let it replace that checking.
6. Worked example: a good candidate and a bad one
A good candidate. A scheduling system assigns shifts across a team. Each person has different availability windows, some shifts require a certification only some people hold, nobody can work two overlapping shifts, and weekly hours per person must stay within labor-rule limits. Satisfying all of these at once is the interacting-constraints structure that benefits from explicit reasoning: check a candidate assignment against each constraint, backtrack on a violation, and decide which trade-off to make when two constraints conflict. The extra reasoning time goes to a problem where working step by step is what produces a correct, constraint-satisfying answer.
A bad candidate. A support system answers "what's your return policy for electronics?" from a known policy document. It's a direct lookup: the model has or retrieves the right text and returns it. Extended thinking adds latency and token cost and gets nothing back, because there's no reasoning chain to improve. This is the case section 2's rule exists to catch.
A quick test for whether a request is a good candidate for extended thinking:
1. Does getting a correct answer genuinely depend on working through
several interacting steps in order (not just retrieving or
generating one thing)?
2. Could an early mistake in that chain plausibly produce a wrong final
answer if it went unchecked?
3. Is the request a simple lookup, a direct rewrite, or a short-form
generation task with a single well-defined answer?
Extended thinking is a good fit when 1 and 2 are true. If 3 is true
instead, the extra reasoning step likely adds latency without adding
accuracy — route it down a direct-answer path instead.
Extended thinking is a capability for reasoning-shaped problems. Spend your design time on the routing logic, per request or per agent-loop step, that sends it only where the problem's structure calls for it.