The commit horizon

How a streamed response is inspected without being buffered.

The problem

Bytes already sent cannot be recalled. A card number that arrives split across four SSE frames is not visible in any one of them, and by the time the fourth arrives the first three are on the client’s screen.

The obvious answers are both bad. Buffer the whole response and streaming is gone, the feature users notice. Scan each frame independently and every match that crosses a frame boundary is missed, which is most of them, because providers emit a few tokens at a time.

The horizon is computed

A streamed response with its last twenty bytes held: a completed card number in the released part, a second one still arriving inside the hold, and the over-long match that would have to exist for the release to be unsafe

A match can only cross a point in the stream if it starts within maxSpan bytes of that point. So hold exactly maxSpan, and everything older is provably safe.

maxSpan is not a guess. It is computed exactly by walking the compiled RE2 syntax tree of the enabled recognizers, fold-orbit aware for rune widths. Measured:

enabled recognizers	hold
CREDIT_CARD	20 bytes
US_SSN	11 bytes
IBAN_CODE	43 bytes
all of them	unbounded

“Unbounded” is honest rather than defeatist. Entities like EMAIL_ADDRESS and URL contain unbounded repeats, so no finite bound exists. Pretending otherwise would silently shorten the horizon. Policy then chooses between a counted risk (on_hold_overflow: emit) and paying the latency (buffer).

Entity restriction is the streaming-latency lever, not just the throughput lever. This is the opposite of the unary path, where restricting recognizers barely moves the number because the prefilter decides in one pass that there is nothing to look at.

A rule that sounded right and was wrong

The refused rule: releasing at every non-word run would have cut through both spaces inside a US_SSN written 123 45 6789

An earlier design released at any non-word run, reasoning that recognizers are \b-anchored so a match cannot span a space.

That is unsound. Patterns contain internal delimiters: an SSN written 123 45 6789, dates, IBANs with spaces. A match can span a space, and a delimiter rule would have cut straight through one.

The principled refinement is a second generated re2c block whose YYFILL call means “a live partial match reaches here”, which would let the hold shrink adaptively. It is not built. Note that the existing region prefilter cannot answer the question. Truncated matches never reach an accepting state, so “no region here” would read as safe when it is the opposite.

The rest of the mechanism

• Per-channel windows. choices.0.content and an Anthropic content block index are separate growing segments, each marked no-cache so the delta cache is not polluted with half-sentences.
• Tool-call arguments are held until complete. A fragment cannot be validated, and a rewritten fragment unbalances the document.
• Frames are released whole and byte-identical unless an edit landed in them. Edits are queued in channel coordinates and materialized once at release, because applying them one at a time compounds offsets. That was a bug in the first draft of this design.
• Rescans are incremental, over [scanned − hold, end], deduped by (ref, start, end, entity), which keeps the cost linear.
• Partial redaction is counted separately for the case where a span’s start was already sent.

Cost: 4.6 µs per frame, about two thirds of which is the pattern DFA. The delay is exported as pistra_guardrail_stream_held_bytes and deliberately kept out of pistra_overhead_seconds. That metric measures work the gateway does, not delay it chooses.