DeepSeek’s recent decision to drastically reduce prices on its V4-Pro model by 75% should have been good news for enterprise AI vendors and developers. Instead, many are finding that cheaper models don’t automatically translate into healthier margins.
The reason is simple: as inference costs plummet, agent systems voraciously consume tokens faster than prices fall. For the last two decades, the software economy was dictated by the same rule. Infrastructure became cheaper every year, while applications became more capable. It was initially hypothesized that AI followed the same pattern. As frontier models improved and token prices fell, many assumed inference would become a negligible operational expense. That assumption has begun to crumble exponentially.
A chatbot typically converts a user’s question into a model call. an agent turns it into a decision chain of planning, retrieving, using tools, verifying, summarizing and tracking. The user sees a response. The seller pays for the loop. That’s the 100x problem: the same user-visible request can cost much more to serve as an agent workflow than as a chatbot or a recovery augmented generation (RAG) response. For longer workflows, the multiplier is higher. The drop in model prices helps, but does not repair a product architecture that turns one ad into dozens of billable operations.
The magnitude of what is now at stake is clear in the way model vendors themselves price relationships with developers. OpenAI’s proposed program to give each Y Combinator startup $2 million in API credits (an amount that would have funded an entire seed round in any previous tech cycle, and when the same cohort made do with a few thousand dollars in AWS credits) is less a recruiting benefit than an admission of what it now costs to run an AI-native company during its first year of product. For established companies that modernize agents in existing product lines, the absolute numbers are even higher.
What is token amplification?
In a single-turn chatbot, one user message produces approximately one model call. The ratio between inputs and turnover is approximately 1:5.
in a multi-step agent implemented in customer service, sales operations, finance, legal review and engineering, that ratio typically reaches 1:700 or higher. Each iteration of the loop carries forward the cumulative conversation, tool results, and reasoning traces. Each step adds; nothing falls off.
TO "simple" agent query like “What did our biggest customer ask about last week? It typically plays seven price trades before returning a response:
-
User notice (~50 tokens)
-
Tool definitions and system messages (~3000 tokens, repeated on each call)
-
Recovery (~5000 context tokens)
-
Model Call #1: Tool Selection (8000 Inputs/200 Outputs)
-
Running tools (~4000 tokens returned)
-
Model Call #2 – Summary (12,000 inputs/400 outputs)
-
Model Call #3: Follow Decision (12,400 inputs/100 outputs)
In one sentence, approximately 35,000 entry tokens are invoiced. Somewhere between $0.10 and $0.40 per query in a frontier model. Multiply that by one million inquiries per month (the volume at stake for any B2B business function) and the line item will be in six figures.
Why this breaks the existing AI business model
The dominant price story for Enterprise AI has been Seat-based SaaS: Payment per user per month, delivery of agent capacity, capture margin. That model assumes a reasonably limited cost per user.
Token amplification breaks the assumption. A power user running 50 agent invocations per day with a $40 per seat plan may cost more in inference than the plan charges. Token amplification breaks the traditional SaaS pricing model. When a power user’s daily agent activity costs more in inference than their monthly subscription fee, providers’ gross margins become negative, a paradox that worsens as customers deepen their agent adoption, the same usage curve that providers are selling to their boards of directors. Several vendors are now privately reporting negative gross margins for heavy users, mirroring recent cloud spending reports from the Bessemer ‘Supernova’ cohort, where the correlation between AI agent adoption and gross margin contraction has gone from a theoretical risk to a major P&L headwind.
Visible symptoms have begun to leak into public coverage. Bloomberg this week documented a widening gap between Salesforce’s Agentforce marketing demos and the capabilities actually shipped to customers. This is the kind of gap that opens up, unsurprisingly, when the promised functionality is technically possible but not economical to serve at the price implied by the seating plan. Salesforce is the most observed case, not the only one.
"For my team, the cost of computing goes far beyond employee costs." — Bryan Catanzaro, VP of Applied Deep Learning, Nvidia
The strategic implication is not "AI is expensive." The dominant business model assumed by most AI-native business plans does not survive contact with agent workloads.
A simple example
Consider an enterprise software provider that charges $40 per user per month for an AI-enabled help desk assistant. In conclusion, a traditional chatbot could cost only a few cents per user per day, leaving healthy gross margins.
Now replace that chatbot with a fully agentic workflow capable of investigating tickets, querying internal systems, composing responses, validating results, and escalating exceptions. If a heavy user executes 50 to 100 agent requests per day, inference consumption can increase by an order of magnitude. What was once an insignificant infrastructure cost becomes a material operating expense.
This creates an unusual dynamic: the customers who receive the greatest value from the product are often the customers who generate the greatest inference costs. In extreme cases, providers may find that their most engaged users bring in the least profits. The result is a growing realization across enterprise software that agent adoption and margin expansion are no longer automatically aligned.
Agent orchestration is the new moat
The technical answers are known and convergent. They are not new, but they are essential for survival.
-
Cost-conscious routing: This technique involves a small classifier model that decides which level (equivalents to Haiku, Sonnet, Opus) handles each query. Well-tuned routers reduce inference costs by about 60% without any degradation in quality
-
Fast caching: anthropicOpenAI and Google are now offering 75-90% discounts on cached prefixes.
-
Context discipline: You can truncate tool outputs, prune reasoning traces, and limit tool depth to prevent your agent from falling down a rabbit hole.
-
Speculative decoding– For self-hosted deployments, this technique guarantees 2 to 3 times effective performance on the same GPUs.
"Organizations using orchestration-based governance report greater productivity gains: a holistic orchestration layer is associated with a six times greater productivity impact than compliance-only approaches." — IBM
Companies that build this layer well are starting to look less like microservices operators and more like financial trading systems: Each route decision has a price, each route with its own profits and losses, each tenant with a measured budget.
What Business Leaders Should Really Do
FOur movements separate the companies that will still have margin in 24 months from those that will not:
-
Make inference cost a first-class metric. Track by feature, by tenant, and by query class in the same way you tracked cloud cost starting in the mid-2010s.
-
Budget like a media buyer. Set maximum cost limits per thousand queries per feature. Cover them. Alert about excesses. Engineering will not enforce this alone.
-
Treat the router as core infrastructure, not an optimization. It is the new load balancer.
-
Quarterly audit notices. A system notice of 4,000 tokens that grew organically over six months is a six-figure bill in slow motion. Most teams have never read their own production messages from start to finish.
-
Negotiate volume commitments in advance. Frontier model providers now offer Reserved Instance style prepaid commitments at substantial discounts. The list price is the worst price any company will ever pay.
The next 24 months
The structural change underlying agent AI is not that it is expensive. As today’s DeepSeek price cut underscores, frontier inference unit costs are falling roughly 3x per year, and the curve is not slowing.
The change is that amplification is outweighing price cuts. Reducing costs per token by 75% does not help a company whose agents generate 700 times more tokens per user query than their pricing model assumed. For the first time since the cloud era began, architectural decisions are once again becoming real-time financial decisions. A quick redesign is a marginal event. A mislinked agent loop is an interrupt with a credit card attached.
The companies that survive the next 24 months of AI infrastructure pricing won’t be the ones using the cheapest model. They will be those whose agents are intelligent. and Know what it costs to think.
That’s the 100X problem. And it’s coming faster than price cuts can hide it.
Maitreyi Chatterjee is a senior software engineer at a large technology company.
Devansh Agarwal works as a machine learning engineer at a leading technology company.
