When the model gets 80% cheaper, your architecture is wrong in a new way

Price per million tokens in 2026 is roughly a fifth of what it was two years ago — somewhere between 70 and 85% lower depending on tier. Over a longer arc the numbers are stranger: from GPT-3 at $60 per million in 2020 to sub-dollar pricing today is a reduction of several hundred times.

The interesting consequence is not that AI got cheaper. It is that a large amount of architecture was built around it being expensive, and that architecture is still running.

The compromises that no longer earn their keep

If you built an AI feature in 2023 or 2024, you almost certainly made some of these trades. They were correct then.

You truncated context aggressively, sending a summary instead of the document, because the document cost too much. You used a weaker model for a task that needed a better one, and wrote post-processing to patch the difference. You cached responses far past the point where they were fresh. You made a feature opt-in behind a button because running it on every request was unaffordable. You batched things overnight that users wanted immediately.

Each was a reasonable response to a real constraint. The constraint moved by an order of magnitude. Most of the compromises did not move at all, because nobody revisits a working feature to ask whether its original justification still holds.

That is where the value is now: not in adding AI features, but in removing the workarounds around the ones you have.

The re-examination worth doing

Take your existing AI features and, for each, ask what you would build if tokens cost a tenth of what they did when you designed it.

Frequently the answer is: send the whole document rather than the summary, because the summarisation step was itself a cost-saving measure that lost information and added a failure mode. Or: use the better model on the path where quality actually matters, and delete the post-processing that existed to patch a weaker one. Or: run it inline instead of behind a button, because the reason it was opt-in was cost and the reason it was unpopular was friction.

This is a nicer kind of engineering work than most. You are deleting code, not adding it.

Two cautions

The floor is not zero, and the decline is slowing. Credible projections put further reductions at 3–5x annually through 2027 and then tapering. Meanwhile the physical constraints on data centre capacity are real and pushing the other way. Designing on the assumption that inference approaches free is a bet, not a trend line.

Falling unit price and falling bills are different things. Cheaper tokens tend to produce more tokens: longer contexts, more calls per interaction, agentic loops that make dozens of requests where a single prompt used to do. Plenty of teams have watched per-unit cost drop and total spend rise. The unit is not the bill.

That is why per-request instrumentation matters more now, not less. If you cannot state the token cost of a single user interaction, you cannot tell an efficiency gain from a volume increase, and both look identical on a monthly invoice.

What this changes strategically

For smaller teams the honest read is that this is good news, with a caveat.

The good news is that the cost advantage large companies had in AI features is mostly gone. When inference was expensive, running a sophisticated model over every request was a capability that came with a balance sheet. At current prices it is available to anyone, and what separates products is the same thing that always did: understanding the problem and designing the interaction well.

The caveat is that this is true for your competitors too. Cheap inference is not a moat. It is a floor everyone now stands on, and the differentiator moved back to product judgement — which is harder to buy and harder to copy.

The teams getting the most from this are not the ones adding AI to more surfaces. They are the ones going back to features built under the old prices and asking which of the compromises were about cost, and are therefore now just damage.