The real running cost of an AI agent: it is not the build, it is the tokens

A company evaluating an AI agent usually asks about the build cost first — the one-time investment to design, connect and test it. The number that actually determines whether the agent stays worth running is a different one: the per-request cost of the model calls behind it, paid every single time the agent runs, for as long as it stays in production. Underestimating that recurring cost, or overestimating it based on outdated pricing, both lead to the wrong decision.

A cost that has been dropping faster than most budgets assume

The 2025 AI Index Report from Stanford HAI documents the scale of the drop: the cost of querying a model performing at the level of GPT-3.5 on a standard benchmark fell from about $20 per million tokens in November 2022 to roughly $0.07 per million tokens by October 2024 — a more than 280-fold reduction in under two years. A cost estimate built on last year’s pricing, in either direction, is likely already wrong.

What still drives the real bill

Falling per-token prices do not mean the total bill trends to zero on their own — the total is driven as much by design choices as by the underlying price: how much context gets sent with every request, how many steps an agent takes to complete one task, and whether a cheaper, smaller model is used for routine steps while a stronger one is reserved for the ones that actually need it.

• →Measure per-request cost, not per-conversation cost, before estimating monthly spend — a multi-step agent can call a model many times for one user interaction.
• →Trim what gets sent as context on every call — unnecessary history or a bloated system prompt is paid for on every single request, at scale.
• →Reserve the strongest, most expensive model for the steps that actually need it, and route routine steps to a smaller, cheaper one — most agent tasks do not require the top-tier model at every step.