Why We Built AgentPulse
AI agents fail quietly and bill loudly. This is the story of how a single runaway token bill turned into an alerting-first observability tool for agent fleets.
The bill arrived on a Monday. One of our background agents — a document-summarization worker that normally cost a few dollars a day — had spent the weekend retrying a malformed prompt in a loop. Every retry was a full context window. By the time anyone looked at the dashboard, it had burned through a month's budget in 36 hours. The agent never crashed. It never threw an error we caught. It just quietly did the wrong thing, thousands of times, at full token price.
What stung wasn't the money. It was that we had a dashboard the whole time. The data was right there — run volume had spiked, average cost per run had tripled, the success rate had quietly cratered. Nobody was looking, because nobody watches a dashboard on a Saturday.
Dashboards are where failures go to be ignored
Most agent observability tools are dashboard-first. They give you beautiful charts of latency and token usage, and they assume someone is staring at them. But agents fail at 2am, on weekends, in the long tail of edge cases you didn't test. The question that matters isn't "what do the charts look like right now" — it's "tell me the moment something is wrong."
So we built AgentPulse alerting-first. The dashboard is still there, and it's good. But the product is organized around rules that watch your metrics for you and fire to Slack, a webhook, or email the instant a threshold is breached. Error rate over 5%? Cost per hour over your budget? Latency doubling? You hear about it while it's still cheap to fix.
Cost belongs to products, not models
The second thing we learned from that weekend: your model invoice tells you what you spent, not where it went. A single line item for "LLM API" is useless when you're running a dozen agents across three products. We made cost-by-product the lead metric — a breakdown of exactly which product and which agent is burning budget, so a runaway loop shows up as a bar that doesn't belong instead of a surprise at the end of the month.
No SDK, on purpose
We didn't want adoption to require rewriting agent code or pinning a vendor library. AgentPulse ingests runs over a single authenticated HTTP call — POST /v1/runs with a Bearer secret. Whatever language your agent is written in, whatever framework, you send one request after each run with status, duration, steps, tokens, and cost. That's the whole integration. It works the same whether your agent is a Python script, a Node worker, or a no-code workflow.
Hosted, so there's nothing to babysit
AgentPulse is a hosted, managed service. You create an org, register an agent, and start sending runs — there's no infrastructure to stand up, no database to operate, and no upgrades to chase. We run it so you can spend your attention on your agents instead of on the thing watching your agents.
We built AgentPulse because we'd already paid the tuition. If you run agents in production and the only way you find out one went sideways is the invoice, this is for you.