LLM gateway · on-premises or your own cloud
Meter every LLM call.
Charge it back.
Stop what shouldn't leave.
Your teams are already sending company data to model APIs. Tokenomix sits in front of that traffic as a proxy, attributes every call to a real person and budget, and screens the prompt before it leaves. One gateway covers the vendor APIs and the models you run yourself.
Audit log
illustrative
| time | dept / identity | model | tokens | cost | verdict | reason |
|---|---|---|---|---|---|---|
| 14:22:07 | marketing / a.okafor | gpt-4o-mini | 1,204 | $0.0031 | ALLOW | — |
| 14:22:09 | support / helpdesk-bot | claude-sonnet-4 | 3,918 | $0.0164 | REDACT | pii: email, phone_uk |
| 14:22:11 | engineering / ci-runner | gpt-4o | 842 | $0.0092 | BLOCK | secrets: aws_access_key |
| 14:22:14 | finance / m.reyes | llama-3.3-70b | 2,077 | $0.0000 | ALLOW | routed local: cost_ceiling |
| 14:22:16 | research / k.lindqvist | gpt-4o | 5,310 | $0.0421 | WARN | quota: 92% of monthly cost |
| 14:22:18 | unknown / no key | — | 0 | $0.0000 | BLOCK | identity: key required |
- 7 + 1
- pattern detectors, plus a fine-tuned classifier
- ~140 ms
- the classifier adds this to a request
- 5
- metrics you can set a hard limit on
- 0
- calls that leave the network before screening
01 — The gap
Shadow AI isn't a policy problem. It's a plumbing problem.
Most organizations found out they had an AI problem from the invoice. One corporate card, forty teams, and a line item nobody can break down. Ask who spent it and the honest answer is a shrug.
The security side is worse, because there's no invoice at all. A support agent pastes a customer record into a chat window. An engineer drops a stack trace with a live key in it. A contractor's script loops ten thousand rows of a personnel file through a model in another jurisdiction. None of it shows up anywhere, because nothing was watching the wire.
You can write a policy about this. People will read it once. What actually changes behavior is a gateway every call has to pass through, and a number next to each team's name at the end of the month.
02 — The request path
Five stages, in this order, on every call.
The order is the design. Quotas are checked before screening, so a request that's already over budget never reaches the GPU and never costs you anything to reject.
-
01
Identity
The validated API key resolves to a person and a department.
Never a client-supplied header. A caller can claim to be anyone; a key can only be the thing it is.
-
02
Quotas
Tokens, requests and spend checked against every limit that applies.
Runs before screening, so a request that is over budget costs no GPU time. Most restrictive limit wins.
-
03
Screening
Seven pattern detectors plus a fine-tuned classifier.
Jailbreaks, PII, secrets, prompt injection, toxicity, blocked topics and your own regex.
-
04
Policy
Block, warn, redact, log or allow. Strictest match wins.
Department rules layer on top of central rules and can only ever tighten them.
-
05
Audit
Verdict, tokens, cost and the deciding rule, on every call.
The row is written whether the request was allowed or stopped, which is what makes chargeback possible.
Add ~140 ms and one hop. Point your SDK at the gateway's base URL and the rest of your code doesn't change.
One gateway in front of both halves of your estate.
Most organizations end up running models in two places and governing neither properly. Tokenomix fronts them together, so a policy, a budget and an audit trail mean the same thing whichever side of the fence a request lands on.
Hosted vendor APIs
- OpenAI
- Anthropic
- Azure OpenAI
- Cohere
Keys stay in the gateway rather than in forty application configs, and the provider's own auth scheme is handled for you.
Models you run yourself
- vLLM
- Ollama
- Anything OpenAI-compatible
- On-prem or in your own VPC
- Air-gapped
The same screening and the same chargeback apply. Self-hosted traffic is usually the part nobody is measuring at all.
03 — In the product
Numbers a finance director will accept, evidence an auditor will.
One console for both jobs. Spend rolls up by department, user, application and model. Every verdict keeps the rule that produced it, so an audit answer takes a search box rather than a project.
04 — Before you ask
What it doesn't do.
Cost limits can't be perfectly exact
Request counts are exact. Token and dollar totals aren't knowable until the model has answered, so the call that crosses your monthly limit completes, and the next one is refused. If you need a hard ceiling to the cent, no proxy can give you one.
Screening is not a guarantee
Pattern detectors catch what they've been given patterns for. The classifier catches a good deal of what they miss, including paraphrased jailbreaks. Neither is a proof. Treat the verdict as a strong signal and keep the audit log, which is the part that holds up later.
One GPU, roughly eight requests a second
That's what one card gives you. Above it you add GPUs, or run pattern-only screening for the traffic that doesn't need semantics. We'd rather tell you the number now than have you find it in week three.
It's a gateway, not a DLP suite
Tokenomix sees traffic addressed to model APIs. Someone typing into a consumer chat app on their phone is a different problem with a different answer, and we won't pretend otherwise.
05 — Where it runs
Your racks, your keys, your logs.
All of it runs on your hardware. Provider credentials are encrypted at rest with a key you hold. Prompts and verdicts are written to your database and never to ours, because there isn't one.
That matters most for the organizations with the strongest reason to want this product. A bank, a hospital group or a defense contractor can't fix a data governance problem by routing every prompt through a startup's SaaS tenant.
06 — Questions
The ones we get asked first.
Does Tokenomix run on our own infrastructure?
Yes, and that is the normal deployment. The gateway, the database, the classifier and the dashboard all run on hardware you control, in your data center or your own cloud account. There is no phone-home and no vendor tenant holding your prompts.
Do we have to change our applications?
Change one base URL. Tokenomix speaks the OpenAI-compatible API, so most SDKs need a single line changed and nothing else. Streaming responses work the same way they did before.
What happens if the classifier is unavailable?
Screening falls back to the seven pattern detectors, which need no GPU. Requests keep flowing and keep being checked, with the semantic layer missing until the classifier is back.
Can a department turn screening off for itself?
Only for the rules central IT marked as bypassable. Policies marked non-bypassable are evaluated regardless, and a central setting can withdraw the capability across the whole organization without editing a single department record.
How is spend attributed to a team?
Every call is attributed to the validated API key, which resolves to a user, a department and an application. Chargeback groups spend by any of those, splits input and output tokens, and exports to CSV.
Next step
Bring a month of your own invoices.
Forty-five minutes, screen shared, your questions. We'll show you the gateway against live traffic and tell you plainly whether it fits what you're dealing with.