Deployment
It runs where your data already is.
There is no Tokenomix cloud, no vendor tenant and nothing to phone home to. You run it, you hold the encryption key, and every prompt and verdict lands in your own database.
01 — Why it matters
A SaaS answer doesn't fit this problem.
The organizations with the most to gain from controlling LLM traffic are the ones least able to solve it by sending that traffic somewhere else. A hospital group cannot fix a data governance problem by routing every prompt through a startup's tenant, and a bank's third-party risk team will reach the same conclusion more slowly and more expensively.
So the product was built the other way round. Self-hosted is not a premium tier here. It's the only tier, and it removes an entire vendor assessment from your path.
02 — What you run
Four moving parts, one of them optional.
- The gateway
- The data path. Your applications point at it, it resolves identity, enforces quotas, screens the request and forwards it. This is the only part that has to be fast.
- The console
- Policies, quotas, providers, users, chargeback and the audit log. Stateless, so you can run several behind a load balancer and lose one without losing the service.
- Your record of it
- Every request, verdict and dollar is written to a database you run, in your own network. We have no copy, because there is nowhere for a copy to go.
- The classifier
- Optional, and the only part that wants a GPU. Volta through Rubin is supported, so it runs on whatever generation you already have. Leave it out entirely and screening falls back to the pattern detectors, on ordinary hardware.
03 — Operating it
The parts your platform team will ask about.
- Where it runs
- Your data center or your own cloud account. Both are the same deployment as far as the software is concerned; what matters is that the keys, the logs and the database belong to you.
- Air-gapped installs
- Nothing phones home. If your models are self-hosted as well, the whole system works with no route to the internet at all.
- Credentials
- Provider API keys are encrypted at rest with a key you hold. Personal keys are stored as a prefix and a hash, so the raw secret exists exactly once, at the moment it is created.
- Upgrades
- An upgrade is rehearsed against a copy of your own production data on an isolated second instance before it touches the live one. Testing a migration against an empty schema proves nothing.
- Passwords
- Twelve characters minimum, no composition rules, and a check against common padded words. Changing a password invalidates every session issued before it, including ones the user has forgotten about.
- High availability
- The application tier is stateless today, which is a prerequisite for production systems.
04 — Questions
Before the architecture review.
What hardware do we need?
For pattern-only screening, an ordinary VM. For the classifier, one GPU. Everything from Volta through Turing, Ampere, Ada, Hopper and Blackwell to Rubin is supported, so whatever generation your estate has already standardized on will work, and if you are running something we have not certified yet we will certify it for you. A single card carries roughly eight requests a second at about 140 ms each. Above that you add cards, or send the traffic that does not need semantic screening to the pattern detectors.
Can we run it in our own cloud account instead?
Yes. On-premises and your own VPC are the same deployment as far as the software is concerned. What matters is that the database, the keys and the logs are in an account you control.
How do applications point at it?
Change the base URL to the gateway and pass a Tokenomix API key. The endpoints are OpenAI-compatible, so most SDKs need one line changed. Streaming works as before.
What happens if the gateway goes down?
Traffic to model APIs stops, which is the honest answer and worth planning for. Run more than one instance behind a load balancer: the application tier is stateless and the quota counters are shared, so the instances agree with each other.