Suspend a key, provider or rule

It is 03:00 and a provider has to stop serving. Editing the configuration is the wrong move twice over. If an operator or a Terraform provider maintains that source, its reconciler will restore what it believes and revert you in ten minutes, correctly and silently. If you write into a source of your own instead, the composition refuses you, because you would be redefining an object somebody else owns. Both are right, and both leave the door locked.

Suspensions are the other door. A suspension is not a configuration change and never becomes one.

curl -sS -X PUT https://gw.example/admin/v1/suspensions/provider:openai \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"reason":"INC-4471 leaking prompt text into provider logs","duration":"4h"}'

The provider stops serving on every node before that call returns.

$TOKEN is a JWT from the issuer this gateway trusts, there is no admin secret to hold. pistra login caches one and pistra admin -url presents it for you. The curl form is here because it is what a script or an SDK sends. See Sign in from the command line.

What you can suspend

The target is kind:name, and the kinds are a closed set:

Target	Effect
provider:<name>	Requests the model resolves to it are refused with 503; as a failover candidate it is skipped
profile:<name>	Every key on the profile is refused with 403
key:<name>	That key is refused with 403
mcp:<name>	Calls to that MCP server are refused with 503
a2a:<name>	Calls to that peer agent are refused with 503
principal:<issuer>/<subject>	A caller an identity provider vouches for is refused with 403, whatever token they present

A key or a profile is the reversible half of a pair. Revoking a key is the other half, and it is permanent, right down to the name, which is never reusable afterwards. Reach for a suspension while you are still finding out, and for a revocation once you know: see Mint, rotate and revoke a virtual key.

Every one is a subtraction, some configured thing stops being available. Nothing here adds a provider, raises a limit or turns a guardrail up. A mechanism that could would be a configuration edit with a lease on it, and the documents would stop describing the deployment. If your emergency is that you need something new, write your own source. A provider under a name nobody else uses collides with nothing, so no permission and no negotiation is needed. See Manage the configuration through the API.

A suspended provider is not silently rerouted to its failover list. Somebody who takes a provider out because it is mishandling data has not thereby asked for the same prompts to go to a different third party. Moving that traffic is a decision, and it belongs in the configuration where it can be read.

The lease is the point

Every suspension expires, and there is no way to make one that does not. Say duration (“4h”, “30m”) or expires_at (RFC 3339), one is required. How long something stays out is the decision the person breaking the glass is making, and a default would be somebody else making it for them in advance.

When the lease lapses, the deployment returns to what its sources say, by itself. There is no rebuild, no sweep, nothing for anybody to remember to undo, and nothing left behind for the next person to wonder about. That is the difference between this and a configuration change. A degraded state with no defined exit is how a four-hour incident becomes a six-month mystery.

Extending is deliberate. Send the same request again. There is no precondition and re-suspending replaces what was there, so two responders reaching for the same switch is not a race anybody has to resolve. The longer lease wins.

To end one early:

curl -sS -X DELETE https://gw.example/admin/v1/suspensions/provider:openai \
  -H "Authorization: Bearer $TOKEN"

Letting it lapse is the ordinary ending. This is for when the incident closed sooner.

Seeing what is out

Use the following example:

curl -sS https://gw.example/admin/v1/suspensions -H "Authorization: Bearer $TOKEN"

Each entry carries who did it, their stated reason, and when the lease ends. in_force: false is a lease that has already lapsed and has not yet been dropped, it stopped applying the moment it expired. unknown: true means the suspension names something the configuration does not define. It subtracts nothing, and it is usually a typo or a source that was deleted while the thing was out.

GET /admin/v1/config carries a suspended count beside file_status, so “why is this deployment not doing what its configuration says” has one place to look rather than two.

The refusal a client gets says both halves:

503  provider openai is suspended until 2026-08-23T07:00:00Z:
     INC-4471 leaking prompt text into provider logs

Say who may reach for the switch

Suspensions are their own Cedar resource class, separate from config, because they are not configuration. That separation is the point. The on-call engineer needs to take a provider out of service and does not need to be able to rewrite the deployment.

The resource instance is the whole kind:name target, so a policy can scope by kind through the id:

@id("oncall-may-suspend-providers")
permit(
  principal in Role::"oncall",
  action in Action::"write",
  resource in Resource::"suspensions"
) when { resource.id like "provider:*" };

@id("everyone-sees-what-is-out")
permit(principal, action in Action::"read", resource in Resource::"suspensions");

resource.id is the bare target, provider:openai, while the entity itself is Resource::"suspensions/provider:openai". So scope by kind with like on the id, as above, and name one thing exactly with the full entity:

@id("only-this-one")
permit(
  principal in Role::"billing",
  action == Action::"suspend",
  resource == Resource::"suspensions/provider:expensive-vendor"
);

What this does not cover

Suspensions are one half of break-glass. The other half is getting into the admin API when the identity provider is the thing that broke. That is a different problem with a different answer, admin.local_socket, a second listener that answers to the filesystem rather than to an issuer. See Reach the admin API without your identity provider.

Neither half covers a cluster that has lost quorum. A suspension is a replicated fact, so writing one needs raft like any other write. Getting raft back is a third thing again, offline and on a stopped node: see Recover a cluster that has lost quorum.