An AI consultant told Axios that one of their clients spent half a billion dollars on Claude in a single month after nobody set a usage limit. Treat the figure with the skepticism a secondhand anecdote deserves — but the cause is mundane and worth taking seriously: there was no cap. It is not isolated. Microsoft canceled most of its internal Claude Code licenses partly over cost, and enterprise AI plans, as Axios notes, are not the all-you-can-eat deals they look like. The half-billion is the lurid extreme; the common case is quieter, and the failure behind both is identical — no cap, and nowhere to put one.
So treat the bill as what it is: not a pricing problem to be solved with a cheaper model or a better rate, but the first thing ungoverned autonomy does that a finance team can see.
The unit of spend changed
For a decade, software cost meant seats: a subscription times a head count, a line item you could predict a year out. The cost was bounded because the thing consuming it was a person. Agents broke that. A single user action fans out — the agent plans, retrieves, calls a tool, reflects, retries, and re-reads its whole context on every turn. One action goes in; an unbounded number of calls come out.
In one afternoon, I pointed eight Claude Code instances at a single repository. They were coordinating through a shared log with no orchestrator above them hardened a tool I'd been building, writing 7.2 million tokens of output between them. (The same run logged over a billion cache-read tokens - but this it is double-counted per-turn re-reading, and cached tokens are the cheapest there are.) They do not pause, do not get bored, and do not notice they are expensive. The thing spending the money cannot be told to stop, because it is not a someone — it is a process, and a process has no judgment about its own budget.
On my subscription, this was not a problem, which is probably the most dangerous way to meet it: free leaves no mark. Price it the way a metered account would, at roughly €25 per million tokens, and that afternoon runs a couple of hundred euros — not millions, not a number anyone escalates. But run agents across a handful of small projects a few afternoons a week, and those unalarming sessions compound into thousands a month, collected quietly, no single charge large enough to trip a wire.
The behavior in that run was governed impeccably, by a shared set of rules the agents had all read. The spend was governed by nothing. Those are different axes, and only one shows up on an invoice.
A dashboard tells you what already burned
The standard fix for runaway cloud spend is FinOps: tag every request, roll it up by team, show each group what it burned, bill it back. Visibility helps; it is not control. A dashboard tells you what already burned — a coroner's report, not a circuit breaker. By the time a month of small, reasonable-looking sessions adds up to a number worth questioning, the tokens are spent.
Governing autonomous spend means moving the decision to the moment of the call: refusing it before the money leaves, not accounting for it after. There is exactly one place in the stack that sees every call before it leaves — the gateway, a proxy between your applications and every model provider. The whole field has converged on one shape: a single OpenAI-compatible endpoint behind which any provider can be swapped.
That convergence is a signal. When a dozen independent products land on the identical chokepoint, the chokepoint is not a fashion — it is where control has to live. The gateway matters for that, not for its feature list: it is the only layer that can say no before the spend happens. Everything else either makes the call or counts it afterward.
What a cap requires
Accept that the spender cannot stop itself, and the controls that matter fall out. They all act before the spend, at the granularity the spend now happens.
Cap tokens, not requests. This is the one most teams get wrong. Requests are not fungible — a single call with a 128,000-token context can cost as much as dozens ordinary ones. A limit that counts requests waves through the one that empties your account. Meter what you are billed for, and block at the budget line.
Hand out virtual keys, not real ones. An agent should never hold a provider secret. Give it a rotating proxy key bound to a budget, an expiry, and a list of models it may call, and keep the real credential in a vault. A leash, not the keys to the building.
Make routing a policy, not a preference. Left alone, an agent reaches for the strongest model on every step, including the trivial ones. A gateway can send cheap work to cheap or local models and save the frontier for what earns it — the point is that the choice leaves the agent's hands.
Inspect at the boundary. Detect and redact names, addresses, and identifiers before a prompt leaves, and keep audit logs that outlive whatever rules you answer to. A filter at the gateway constrains the bytes that go out. A policy document constrains a human — and in an agent pipeline, there is no human at the keyboard to constrain.
None of this is exotic. It is the same instinct that makes you run an agent as a non-privileged user instead of root: you scope what it can reach, because you have decided not to trust its judgment. The gateway is that instinct at the network boundary instead of the operating system.
Gateways that cap, and gateways that only watch
The market reinvents itself every few months, but the products sort cleanly on the only question that matters: does it cap spend before it happens, or just report it afterward? The newest go further than counting — agentgateway, the Linux Foundation's Rust data plane, parses the OpenAI-format request body to pull out model names, token counts, and tool calls, and applies policy before the call goes out. The cap is turning into a first-class feature.
| Gateway | Hosting | Best for | Caps spend before it happens? | Watch for |
|---|---|---|---|---|
| LiteLLM | Self-hosted, open source | Teams that want full control and will run the plumbing | Yes — hierarchical token budgets (RBAC/SSO are enterprise) | You operate Redis and Postgres yourself |
| OpenRouter | Managed SaaS | Fastest on-ramp to hundreds of models behind one key | Yes — per-key credit and spend limits | Traffic flows through a third party |
| Portkey | Managed / self-hostable | Governance-first shops needing SOC2/HIPAA and budgets | Yes — virtual keys with budget caps (paid tiers) | Analytics and access control lean on its SaaS layer |
| Cloudflare AI Gateway | Managed, edge | Teams already on Cloudflare wanting caching and DLP | Partly — request-rate limits and cost analytics, not token budgets | Spend control is coarser than per-token |
| Vercel AI Gateway | Managed | Web stacks on Vercel; zero-data-retention routing | Partly — spend tracking and rate limits, DX-first | Part of the Vercel platform, billed with it |
| Helicone | Managed, open-source core | Observability — seeing every call in detail | No — monitoring only | In maintenance mode since the March 2026 Mintlify acquisition |
| Kong AI Gateway | Self-hosted / enterprise | Existing Kong users treating LLM traffic like any API | Yes — but token limits are enterprise-only | AI features sit behind the enterprise tier |
| Bifrost(Maxim AI) | Self-hosted, open source (Go) | High-throughput setups needing speed and governance | Yes — hierarchical virtual-key budgets | Newer; the "50x faster than LiteLLM" figure is Maxim AI's own |
| Eden AI | Managed | LLMs plus OCR, speech, and vision through one API | Limited — cost optimization via routing | Breadth beyond LLMs, less depth on token-level caps |
| Agentgateway / kgateway | Self-hosted, open source (Linux Foundation) | Routing tool calls and agents, not just models | Yes — budget and spend controls, plus MCP and A2A | Kubernetes-oriented; aimed at the agent layer |
| LLM Gateway | Managed or self-hosted | A simple unified API across many providers | Limited — cost tracking and routing | Lighter on enterprise governance |
This is a starting map, not a ranking, and the volatile parts — tier gates, active development, exact limits — move fast, so check the current state first. Read the cap column before the rest: a gateway that only watches spend is a nicer dashboard, not a circuit breaker. Helicone is the cautionary case — a well-built observability gateway that dropped into maintenance mode after Mintlify acquired it. Capable today is not the same as maintained tomorrow.
The threshold is throughput, not headcount
You do not always need one. The line is not company size — it is how much of your call volume comes from software rather than people. One assistant integration and a few humans pressing a button is fine on a per-seat plan; a solo developer can even run a swarm on a personal subscription and never see a bill, which is exactly why it does not generalize. The moment a company needs central budgets, attribution, and redaction across teams, it is on metered billing, where every one of those re-reads is counted and charged.
And the gateway is not free. The open-source options cost nothing to license but expect you to run the datastores they lean on; the managed ones lift that load and hand you a dependency instead. Governing cost has a cost — pretending otherwise is how the second surprise arrives.
Where the next fight is
Once enforcement lives at the gateway, it stops just counting what your agents do. It decides what they may spend, and through its filters, what they may say. That is real power in one place — which makes the next question unavoidable: who writes the policy the agents run under, and is it something a machine enforces, or a paragraph in a wiki everyone has agreed to ignore?
The frontier is already past that. The hard problems now are agents that delegate to other agents and call external tools. The protocols for that layer — MCP for agent-to-tool, A2A for agent-to-agent, both now under the Linux Foundation — standardize how agents reach tools and each other, not who pays for what they do.
The gateway is becoming where a company decides what its autonomous systems may and may not do. That makes it the most valuable thing in the stack to get right — and the most valuable to compromise. Centralized control is the point. It is also the next fight.
Unlock the Future of Business with AI
Dive into our immersive workshops and equip your team with the tools and knowledge to lead in the AI era.