18 Nov 2026
6 Min Read
Your AI agents spend at machine speed. Your team reacts at human speed.
An AI agent stuck in a loop at $4 a minute costs $240 an hour. Leave it running overnight and that's nearly $3,000 before anyone opens a laptop. Nothing about it looks broken -- every request succeeds, every dashboard is green, and the bill arrives next month.
That is the new operating gap for any company putting agents into production. Agents spend your money at machine speed. Your team reacts at human speed. The distance between those two speeds is where surprise bills, unhappy customers and long blame filled incident calls come from.
Closing that gap takes a system that watches live events, knows what normal looks like and acts on proof within minutes. DeltaStream is the foundation for building systems like that. To show what's possible, we built one for our own AI agents, called OpenStream. This post walks through the problem, the building blocks and how the same approach applies well beyond AI spend.
Three quiet failures, three kinds of cost
Every team running agents ends up asking the same three questions. Each one maps to a cost that grows the longer nobody notices.
| The question | What happens | What it costs |
|---|---|---|
|
Is an agent running away with the budget? |
A loop, a bad prompt or a retry storm multiplies spend |
Dollars: found on the invoice, weeks later |
|
Are users waiting while every dashboard is green? |
A model provider quietly fails over; requests still succeed, just 10–15× slower |
Customers: they feel every second, and some leave |
|
When something breaks, is it us or our provider? |
Errors spike and nobody can say where they start |
Trust: hours lost to back-and-forth while the problem keeps running |
None of these is a dramatic outage. That is exactly why they are expensive: they don't page anyone. The longer they run, the more they cost.
Why the usual tools miss it
Most companies already have three lines of defense. Each one breaks down for agents.
- The invoice is accurate, but it arrives after the money is gone.
- Dashboards watch success rates and errors. A runaway agent and a slow provider both look healthy on those charts.
- Fixed spending limits are the same for every agent. $5 a minute is normal for a busy agent and alarming for a small one, so one number is either too tight or too loose. Teams end up with noisy alerts or no alerts.
The deeper issue is that these problems don't show up in any single request. One call to a model looks fine. The pattern only appears across thousands of requests over minutes and hours. Catching it means watching every request as it happens and comparing it with what normal looks like for that specific agent. Weekly reports and nightly batch jobs are too late for that.
What it takes, and what DeltaStream provides
Any system that catches problems as they happen needs the same few building blocks. DeltaStream provides them, so a team can focus on what to watch for instead of the plumbing.
| Building block | What DeltaStream does | How OpenStream uses it |
|---|---|---|
|
See every event as it happens |
Connects to live event streams such as Kafka and processes each event the moment it arrives |
Reads a record of every model call: cost, tokens, model, provider |
|
Know what normal looks like |
Keeps a running picture per customer, product, account or agent, over windows like the last 5 minutes and the last hour |
Learns each agent’s own normal spend rate |
|
Spot the pattern, not the single event |
Always-on queries that compare now with normal and track whether a problem persists |
12 detectors for spend spikes, hidden slowdowns and provider failures |
|
Produce evidence you can act on |
Results are precise and repeatable, so fixed rules can act on them safely |
Switches an agent off only on sustained, fresh proof |
|
Build it from intent |
AI writes the streaming logic from plain-English descriptions |
Every detector was written by AI from plain-English descriptions |
The last row matters most for leaders. The scarce skill used to be writing streaming code. With DeltaStream, the scarce skill is knowing your business well enough to say what should never go unnoticed.
One example: OpenStream
OpenStream is what those building blocks look like when pointed at AI spend. It watches every agent's spend as it happens and learns what normal looks like for each one, so nobody has to set a limit per agent. DeltaStream does the continuous watching; OpenStream is a small service that turns DeltaStream's findings into alerts and actions.
Here is what happens when an agent misbehaves:
- Every request is seen. Our model gateway (OpenRouter) sends a record of each call: the model, tokens, cost and which provider handled it.
- Each agent is judged against its own normal. DeltaStream compares the last minute with that agent's last five minutes and last hour. Four times normal is a warning; ten times is severe. Small dollar floors keep tiny agents from paging anyone.
- People hear about it in minutes, in plain terms. An alert lands in Slack in dollars, next to that agent's normal rate.
- One click explains it. An AI agent investigates: which model, how much, has this happened before. It answers in plain English, in the thread.
- If it keeps getting worse, it can be switched off. Only for agents you've opted in, and only after the checks in the next section pass.
In our test, a deliberately runaway agent was caught and switched off in a few minutes, with no human involved. Turning it back on took one request and one click from a person.
Why it's safe to let it act
Automation that can shut things off is only useful if leaders trust it. We built OpenStream around one rule: AI explains, rules act.
- The AI never pulls the trigger. The investigating agent is read-only. It can look at the evidence and explain it, but it has no access to switch anything off.
- Action needs proof, not a hunch. An agent is switched off only if it has stayed severe for over a minute, the evidence is fresh, and the system has confirmed it has the right key. Afterward it reads the key back to confirm, and every action is logged.
- It's opt-in, agent by agent. You choose which agents may be switched off automatically. Everything else only alerts.
- Missing data is never treated as safe, and it never triggers a shutdown either. If the system can't prove something, it doesn't act on it.
- Turning an agent back on is always a human decision. Someone asks for recovery in the thread and confirms. The incident closes itself after 15 quiet minutes.
This pattern carries over to anything you build on DeltaStream. The streaming layer produces evidence the system can prove; fixed rules decide what happens next; AI explains and humans stay in charge of the big decisions. Leaders get machine-speed response and still know exactly when the system will act, and on what evidence.
Nobody coded this by hand
I didn't write OpenStream's detection logic. I described what I wanted in plain English:
Compare every agent with its own normal. Warn at 4×, severe at 10×, and only act if it lasts a minute.
Tell me when silent retries are costing users time.
If several agents fail on the same provider, tell me it's upstream, not my agent.
AI turned those sentences into 12 detectors that DeltaStream runs on every request, around the clock. I didn't hand-write one of them.
Going from idea to 12 running detectors didn't require a streaming specialist. It required knowing what to watch for, and DeltaStream did the rest.
What you could build next
OpenStream is one example. The same building blocks fit any business that runs on live events. You describe what should never go unnoticed, AI builds the detectors, DeltaStream runs them live, and clear rules decide when to act.
| Business | Describe it in plain English | What the system does |
|---|---|---|
|
Payments |
“Flag any merchant whose declines jump well above its own normal.” |
Alerts the risk team in minutes, with the evidence |
|
E-commerce orders |
“Tell me when checkouts in a region stop completing.” |
Pages on-call before the revenue dip shows in a report |
|
Inventory |
“Warn when a product is selling far faster than its usual rate.” |
Triggers a reorder before it stocks out |
|
Fleets |
“Tell me when a vehicle goes off its normal route or stops too long.” |
Notifies dispatch while there’s still time to act |
Three questions worth asking your team this week:
- If one of our agents started looping at 2 a.m., how long until someone knew, and what would it cost by then?
- Would we notice if a provider slowed down but nothing technically failed?
- When something breaks, how long does it take us to know whether it's our code or our vendor?
If the honest answers are "the next morning," "probably not" and "hours," you have the same gap we had. And the questions only change by industry: swap "agent" for merchant, region, product or vehicle.
See it running: a 3-minute walkthrough of OpenStream catching a runaway agent, explaining it and switching it off. Have a system like this in mind for your business? See what you can build on DeltaStream at deltastream.io.