Serving a model costs real money per token, and most AI companies price below that cost because the alternative is losing the customer. ICONIQ's 2026 survey puts enterprise AI gross margins at 45 per cent in 2025, rising to an expected 53 per cent this year.
A software company at 45 per cent gross margin is not a software company. It is a services business with a subscription wrapper.
Every lab prices output at five times input
| Model | Input, per MTok | Output, per MTok | Ratio |
|---|---|---|---|
| Claude Opus 5.5 | $4 | $20 | 5.0 |
| Claude Haiku 5.5 | $0.10 | $0.50 | 5.0 |
| GPT-6 Astra | $10 | $50 | 5.0 |
| Gemini 4 Carbon | $2 | $10 | 5.0 |
The ratio is the same across four labs and a hundredfold range of prices. That is not collusion; it is the cost structure. Generating a token means a forward pass per token, while reading one means a forward pass over the prompt once, and the industry has settled on charging five times for the difference.
Where the money goes
Compute, mostly. Inference is 23 per cent of enterprise AI budgets, which makes it the largest line item after people.
The compute market is repricing underneath. Microsoft has been winding down its Oracle GPU capacity, a sign that the contracts signed in the shortage are being renegotiated now that the shortage is over.
Margins improve for a boring reason
53 per cent next year is not a breakthrough. It is the same thing that happened to cloud hosting: better utilisation, cheaper chips, and customers who stop experimenting and start paying for a steady workload.
The labs that reach 70 per cent will be the ones whose inference runs on their own silicon. The rest will keep paying someone else's margin.