Routing and cost gateways
Developer LLM gateways route requests among model providers and track model usage and cost. They give application teams one gateway for reaching the models they use.
SUPERWISE® Sentinel
What an LLM gateway is, and why one should remove personal data
A plain-English guide to routing gateways, data-protection gateways, and the role an AI gateway can play before a request reaches ChatGPT or Claude.
What is an LLM gateway?
An LLM gateway is a control point between an application or user and the language models they call. It receives requests before the model does, then applies the job it was built for, such as routing traffic, tracking cost, or protecting data, before forwarding the request and recording what happened.
Some teams call this an LLM firewall: a gateway between their applications and the model provider that checks every request before it leaves. Sentinel replaces PII and secrets in the formats it supports, can block a request, and records metadata only. Prompts never pass through SUPERWISE's cloud.
Two kinds of AI gateway
Both categories sit in the path to a language model. They are built to solve different jobs.
Data-protection gateways
A data-protection gateway checks what a request contains before forwarding it. Sentinel replaces email addresses, phone numbers, Social Security numbers in the 123-45-6789 format, credit card numbers, IBANs, IP and MAC addresses, full-length OpenAI keys, and AWS secret keys when labeled. Bank account numbers, tax IDs and EINs, national IDs, VINs, dates of birth, and medical record numbers are replaced only in the labeled formats the gateway recognizes. It does not catch names, street addresses, passport numbers, bank routing numbers, driver's licenses, Okta tokens, unlabeled numbers, short test keys, or values under a column header in a pasted table. For each request Sentinel records metadata only: counts, which rule fired, and latency. Prompts and responses are never sent to SUPERWISE.
Where Sentinel fits
Four values become {{REDACTED}}
Sentinel replaces email addresses, phone numbers, Social Security numbers in the 123-45-6789 format, credit card numbers, IBANs, IP and MAC addresses, full-length OpenAI keys, and AWS secret keys when labeled. Bank account numbers, tax IDs and EINs, national IDs, VINs, dates of birth, and medical record numbers are replaced only in the labeled formats the gateway recognizes. It does not catch names, street addresses, passport numbers, bank routing numbers, driver's licenses, Okta tokens, unlabeled numbers, short test keys, or values under a column header in a pasted table. Each value it catches becomes {{REDACTED}}. An admin can change the replacement text, or block the request instead.
Email addresses
{{REDACTED}}
Phone numbers
{{REDACTED}}
Labeled account numbers
{{REDACTED}}
API keys
{{REDACTED}}
Choose where to go next
Review the product, choose a deployment path, or install Sentinel for Windows.
Compare this LLM gateway with LiteLLM and Microsoft Presidio.
Follow the PII redaction guide to define, test, and record the request path. Then read how runtime enforcement works and use the AI privacy and security glossary for the underlying terms.
LLM gateway questions
Plain answers about how LLM gateways work
An LLM gateway is a control point between an application or user and the language models they call. It receives requests before the model does, then applies the job it was built for, such as routing traffic, tracking cost, or protecting data, before forwarding the request and recording what happened.
An API gateway manages traffic between clients and APIs. An LLM gateway sits in the request path to language models and applies controls built for AI traffic, such as model routing, cost tracking, or data protection.
An LLM firewall is an LLM gateway that checks traffic against security and data-protection rules. Guardrails check the prompt on its way out. The model's response comes back unchanged. Sentinel can replace supported PII and secrets or block the request, and records metadata only.
SUPERWISE Sentinel works with supported traffic to ChatGPT and Claude. Sentinel replaces listed identifiers before a prompt leaves, including some values only in recognized formats or with recognized labels. It does not catch names, street addresses, passport numbers, bank routing numbers, driver's licenses, Okta tokens, unlabeled numbers, short test keys, or table-header values. Each value it catches becomes {{REDACTED}}. An admin can change the replacement text, or block the request instead. For each request Sentinel records metadata only: counts, which rule fired, and latency. Prompts and responses are never sent to SUPERWISE.
SUPERWISE Sentinel can run on a computer or in your own cloud. The deployment guide covers the Windows app, command-line gateway, and cloud gateway paths.