A support agent pastes a customer’s message into an internal assistant and the message has a card number in it. Something on the path finds the digits before the request leaves the building, puts a different sixteen digits in their place, and forwards it. The model answers the question that was actually asked. The agent notices nothing, and a line is written down naming which rule fired, on which span, under which version of the policy.
All of that happened inside one request, in the window between a client sending it and a provider receiving it. It could not have been arranged earlier or afterwards.
Afterwards, asked at the end of the quarter whether any prompt containing a card number had reached a model, the same platform engineer spent three days grepping an S3 bucket of access logs and still could not say. The logs had request IDs, status codes, latencies and token counts. They had no bodies, because storing bodies had failed the same data-handling policy the question came from. The gateway had forwarded everything without looking at any of it.
A request carries a credential, a model name, a body, and sometimes a tool call inside the body. For as long as one process holds the connection, it can refuse the request, rewrite one span of it, count what it draws from a budget, and write down what it did and under which rule. We call those four decide, redact, meter and record, and use no other words for them. Once the provider has the bytes, the remaining questions can only be answered from records.
Who is on the path
Six kinds of product, and each arrived carrying its first buyer’s concerns.
The marketplace (OpenRouter, Requesty, Vercel’s AI Gateway) occupies the provider’s position rather than a hop in front of it. It sells choice and one bill, and it earns when a developer buys credits.
The proxy (LiteLLM, Portkey, Helicone) sells one API for every provider, virtual keys and a spend counter, to a developer or an ML platform team. It sits in the middle of the path, and to offer one API it rewrites your request into its own shape on the way through.
The API gateway you already run (Kong, Apigee, and the Envoy family: Envoy AI Gateway, kgateway, agentgateway) terminates everything already. It sells “AI is another protocol” to the platform team, and does AI as a plugin on a route.
The cloud’s own service (Bedrock Guardrails, Azure API Management, Model Armor) sells integration inside one cloud’s identity and billing, to whoever owns the account.
The detector (Presidio, Lakera, Prisma AIRS, Guardrails AI, NeMo Guardrails) sells detection to the security team. It is reached by a call from whatever holds the connection, and it answers about the string it was handed.
The governance platform (Credo AI, Holistic AI, IBM watsonx.governance, OneTrust) sells the policy document and the evidence binder to risk and compliance. It sits before the call and after it.
Each of the six answers the buyer that pays for it. The same two questions apply to all of them. While this thing holds the connection, which of the four can it do, and whose record is it afterwards?
Positions, not features
The feature tables are converging. Proxies are adding single sign-on, role-based access and MCP gateways. API gateways are adding prompt guards and token metering. The cloud services are adding each other’s models. A table that says one of them lacks a feature is wrong within a quarter, and we are not going to publish one.
Position does not converge: where on the path a product sits, and who pays it for what. A marketplace will not stop being the counterparty. A gateway will not stop thinking in routes. A governance platform will not build a data plane. Those are the facts a deployment can be planned around.
We built pistra for the cell none of them occupies: one process that holds the connection, does all four, and keeps the record on your side of it. Each post from here takes one position and says who should use it instead.
If nobody has asked you for evidence
A log is fine. Request IDs, status codes, latencies and token counts answer the question most teams get, which is who spent what. The four words start to earn their keep at the first question about what a prompt contained, or whether a rule ran, and by then the quarter being asked about has already been forwarded.
The series
- Buying models through a marketplace. Whose record it is, when the party keeping it is paid on the purchase.
- Proxies and the unified API. What the provider actually received, and the rewrite in between.
- Response caching. Why caching belongs with the provider, and when to run one anyway.
- API gateways with AI plugins. Keep it. Where its vocabulary ends.
- The cloud’s own AI services. Custody again, from inside a tenancy.
- Calling a detector. Inputs, not decisions.
- Governance platforms. What the binder can hold, and where it comes from.
- Comparing the six positions. Three questions to ask any vendor, and the empty cell you get back.
How these posts are kept
A post describes the day it was written. The names above are positions as of that day. Anything that can change, a price, a plugin list, how a vendor's product works, sits in a box like this one with a date and a link, and the boxes are the only text edited after publication. The map post is re-issued rather than edited, with the previous version linked.