AI API & Agent
Spend Firewall.

No. Your provider API key (e.g. sk-ant-...) is passed through the proxy header on each request and immediately discarded. LLMCap only stores your LLMCap proxy key, hashed with bcrypt. We never log your provider keys.

Yes — streaming is supported from day one. LLMCap passes SSE chunks through in real time. If the budget is exceeded mid-stream, the connection is closed and a final 429 event is sent. The token that triggered the cap is not charged.

The next incoming request is rejected with HTTP 429 before it reaches the provider. The token is never consumed, so you are never billed for it. Your app receives the same 429 response structure providers use for rate limiting, so existing error handling works as-is.

Yes — the Provider Secret Vault lets you store your Anthropic, OpenAI, Google, Mistral, Cohere, and Bedrock credentials in LLMCap, encrypted at rest with AES-256-GCM (96-bit random nonce per entry). Once saved, your code only uses a LLMCap proxy key — the provider key never appears in your codebase or environment variables. Secrets are never returned after initial save and never written to logs.

Yes — the Policy Engine lets you create priority-ordered allow/block rules scoped to models, providers, organizations, or projects. Rules are evaluated top-down with deny-wins semantics. For example: allow gpt-4o-mini, block claude-opus-*, allow everything else. Model prefix matching handles version suffixes automatically (gpt-4o blocks gpt-4o-mini but not gpt-4o).

You configure one or more HTTPS webhook endpoints in the dashboard. When an anomaly is detected (spend spike, budget at 80%/95%, cache drop, or error rate spike), LLMCap POSTs a signed JSON payload to your endpoint within seconds. Each payload includes an HMAC-SHA256 signature in the X-LLMCap-Signature header so you can verify it's genuine. Works with PagerDuty, Slack, Teams, or any custom endpoint.

Yes — AWS Bedrock is fully supported including Claude, Llama, Titan, Mistral, and Cohere models via Bedrock. Pass your AWS credentials in request headers alongside your LLMCap key. LLMCap re-signs requests using SigV4 and forwards them to the Bedrock Runtime. Credentials are never stored. This is exactly what fills the gap that AWS Budgets leaves open: AWS Budgets has 8-12 hour reporting lag and sometimes doesn't trigger at all — LLMCap blocks before the charge happens.

Self-hosting is on the roadmap. The proxy is open source (FastAPI + Redis). For now, the managed service at proxy.llmcap.io is the recommended path — it's already deployed with <35ms latency worldwide.

No. LLMCap logs only the metadata needed to enforce your cap and power your dashboard: provider, model, token counts, cost, status, latency, and app version. We never store prompt text, system instructions, user messages, or assistant responses. Your conversations stay strictly between your application and the LLM provider — we only count the tokens that pass through, we don't read them. You can confirm exactly what we record by opening the Request Log in your dashboard.

Yes. Every connection uses TLS 1.3. Your application connects to proxy.llmcap.io over HTTPS — there is no unencrypted path available. LLMCap's outbound connection to each provider (Anthropic, OpenAI, Google, etc.) is also HTTPS. Provider API keys you send in request headers are never written to disk or any log. Keys stored in the Provider Secret Vault are encrypted at rest with AES-256-GCM with a 96-bit random nonce per entry.

LLM calls will fail with a connection error until the proxy recovers. We target 99.9% uptime via Railway with health monitoring and automatic restarts. For applications requiring higher availability, we recommend wrapping LLMCap calls in a circuit breaker with a configurable direct-to-provider fallback that you can activate if needed. A self-hosted high-availability deployment option is on the roadmap.

No. Requests without X-LLMCap-Agent-Id or X-LLMCap-Run-Id headers have no agent or run budget applied. They are tracked against your workspace and API key budgets as normal. You can run mixed traffic (some tagged, some not) in the same workspace.

They are enforced together — the first exhausted limit blocks the request. If your workspace has a $50/day budget, an agent has a $5/day cap, and a run has a $1.00 total cap, any of the three reaching its limit returns a 429 before the provider is called. Set a high workspace limit for overall protection and tighter agent and run limits for individual workflows — all checked atomically before each request.

When the spending cap is hit during a stream, LLMCap sends a terminal SSE event — data: {"error": "budget_exceeded", "code": 429} — then closes the connection cleanly. Partial tokens received before the cutoff are not billed; the request that triggered the cap pays zero. Your existing SSE error handler receives this event the same way it receives any provider-side error, so no special handling is required.

Multiple layers limit the blast radius. (1) IP Allowlist — even with valid credentials, requests from IPs outside your allowlist are rejected at the gate before any token is consumed. (2) Provider Secret Vault keys are AES-256-GCM encrypted at rest; an attacker with database access cannot extract them without the separate encryption key. (3) Spending caps are enforced at the proxy — no rogue actor can spend beyond your configured cap. (4) The audit log is append-only and tamper-proof at the database trigger level — every action is recorded and cannot be erased. Assign team members the minimum role they need (RBAC), and enable IP allowlisting on production keys to further reduce exposure.