Why we poll instead of proxy

Most cost-monitoring tools are proxies. They sit between your application and the LLM provider, forwarding every request and logging it on the way through. That gives them per-request visibility — but it also gives them your prompts, your completions, and a new failure mode in your request path.

ModelBrake takes a different approach.

How polling works

You give us a read-only admin credential — the one that can query spend but cannot call models. We poll the provider's usage API on a schedule:

• OpenAI — every 15 minutes via the admin usage API
• Anthropic — every 15 minutes via the admin API
• AWS Bedrock — every 6 hours via Cost Explorer (data availability constraint)
• Vercel AI Gateway — every 15 minutes via the billing API

Each poll records the current spend snapshot. We compare today's accumulating spend against the median of your trailing days and fire an anomaly alert when it crosses 3× that median.

median trailing daily spend: $120
today at 2 PM:               $380  →  3.2× → anomaly alert

What we trade off

The honest tradeoff is a detection gap. Spend that occurs between polls is not visible until the next poll completes. For OpenAI, Anthropic, and Vercel, that gap is at most 15 minutes. For Bedrock, it can be up to 6 hours (plus Cost Explorer's own data lag).

For most teams, 15 minutes is fast enough to catch a four-figure spike before it becomes a five-figure invoice. A runaway loop burning $1,000/hour accumulates $250 in 15 minutes — that is detectable. By contrast, provider dashboards update hourly or daily, and billing alerts fire on monthly totals, not velocity.

What we never see

Because we never sit in the request path:

• Your prompts and completions never touch our infrastructure
• Your application API keys stay in your stack
• There is no added latency on any model call
• There is no new failure mode — if ModelBrake goes down, your LLM calls keep working

The read-only credential we use cannot call models, create keys, or access prompt content. It can only read aggregated usage and billing data.

When a proxy is the right choice

A proxy is the right tool when you need:

• Per-request logging with prompt/completion content
• Prompt inspection for PII scanning or content policy enforcement
• Real-time rate limiting that rejects requests immediately when a budget is exhausted

If you need any of those, use a proxy. ModelBrake is not trying to replace that use case. We are solving a different problem: catching cost anomalies without changing your infrastructure.

The credential model

ModelBrake requires exactly one credential to monitor: a read-only admin key. This key can read usage data but cannot:

• Call any model endpoint
• Create or delete application keys
• Access prompt or completion content
• Modify billing settings

If you opt in to automatic key revocation (an optional feature), you provide a separate write-scoped credential deliberately. The monitoring credential and the revocation credential are different keys with different permission scopes.

Read the security page for details on how credentials are encrypted and isolated.