Platform · Routing
Some prompts should never reach a vendor.
The gateway can decide which provider and model serves a request instead of passing through whatever the client asked for. Because screening has already run, that decision can depend on what is in the prompt.
01 — The idea
Egress control that reads the payload.
A firewall rule can say this host may talk to that host. It cannot say this particular paragraph may not leave the building. Once a detector has looked at the prompt, the gateway can.
The practical version is dull and useful: anything containing customer PII or a credential goes to the model on your own GPUs, everything else goes wherever it is cheapest, and both paths produce the same audit row. Most organizations already believe this is their policy. This is the part that makes it true.
- Constraints narrow
- Every constraint removes candidates and can never add one. They compose by intersection, which is where the guarantee lives: a department policy is structurally incapable of widening what central IT allows.
- Preferences choose
- Exactly one winner, evaluated in priority order, first match taken. A preference can never bring back a target that a constraint removed.
- Runs after screening
- Constraints can read what the detectors found. That is what makes "anything carrying PII stays on a local model" a rule the gateway can enforce rather than a sentence in a policy document.
- Fails toward local
- If constraints eliminate every candidate, the request falls back to the configured local target. Never to the cheapest one, and never off-site. An external fallback is refused outright.
- A missing detector is not an excuse
- A constraint conditioned on a detector that never ran is enforced, not skipped. Otherwise turning screening off would quietly open an egress hole, which is exactly the kind of interaction that ships as a breach.
- Shadow mode first
- By default the decision is computed and written to the log next to the model the client asked for, while the request is still served as it always was. You get weeks of evidence about what routing would do before it does anything.
02 — In the console
Targets, constraints, preferences, switch.
03 — Questions
Routing, in practice.
Why not just use the policy engine for this?
The policy engine resolves conflicts by taking the maximum over a total order, from allow up to block. Routing targets have no such order. "Cheapest" and "fastest" are not comparable, and two rules naming different providers is a conflict to be settled by priority, not something to maximize over. Two different problems, two evaluators.
Is routing on by default?
No. It runs in shadow mode, which computes the decision and records it without acting on it. Enforcement is a stored setting that takes precedence over the environment, so turning it on is a deliberate act in the console rather than a config file somebody forgot about.
What can a rule route on?
Detector findings, department, user, the requested model and the endpoint. The common shape is a constraint that keeps sensitive content on self-hosted models, plus a preference that sends everything else to whichever provider is cheapest for that workload.