Renting inference by the token is the cheapest way to start and the most expensive way to scale. The crossover is a utilisation question, not a price question.
| Option | What you pay | When it wins |
|---|---|---|
| Token API | Per token, nothing when idle | Under a few billion tokens a month |
| Batch API | About half, for results within 24 hours | Anything not interactive |
| Own GPU | Per hour, whether busy or not | Steady load you can keep above half |
The idle problem
A GPU you rent costs the same at 3am as at noon. A token API costs nothing at 3am. If your traffic has a peak and a trough, the API is cheaper by construction, because you are renting someone else's utilisation.
That is the whole argument, and it is why almost everyone should start with the API.
The batch discount is the free one
Anthropic's batch API takes 50 per cent off in exchange for a 24-hour turnaround. If a job does not need an answer now — a nightly re-classification, a backfill, a report — that is a straight halving with no engineering.
When owning the GPU wins
When the load is steady and the model is open-weight. gpt-oss-120b runs on a single 80GB GPU, and at 4-bit it needs about 60 GB, so one machine serves it.
The arithmetic is then: GPU-hour price divided by tokens served per hour. Below a few cents per million tokens, owning beats renting — and you get a fixed bill instead of a variable one.
The number that moves
Token prices fall. Hardware prices do not. Every cut from a lab lowers the crossover point, so a case for owning the GPU made last year needs re-making this year.
Check the current list price before you buy a machine to beat it.