Run the front door on VMs

Three virtual machines behind a load balancer, forming one raft cluster, with no Kubernetes anywhere. Nothing in the data plane talks to a Kubernetes API, so this is the same binary and the same config the chart installs. You write the cluster block yourself, and the load balancer in front is now yours to configure.

For the Kubernetes version see Deploy the front door with Helm.

Install

Each release publishes a .deb and an .rpm per architecture, and a tar.gz for anything else. The package installs the binary at /usr/bin/pistra, the systemd unit, the pistra service account and an empty /etc/pistra. It starts nothing, because there is no config yet.

$ sudo dpkg -i pistra_1.0.0_linux_amd64.deb      # or: rpm -i ...

Verify what you downloaded first. One signature covers every artifact, because it is over the checksum file that lists them:

$ cosign verify-blob pistra_1.0.0_checksums.txt \
    --signature pistra_1.0.0_checksums.txt.sig \
    --certificate pistra_1.0.0_checksums.txt.pem \
    --certificate-identity-regexp '^https://github.com/' \
    --certificate-oidc-issuer https://token.actions.githubusercontent.com
$ sha256sum -c pistra_1.0.0_checksums.txt --ignore-missing

From the tarball instead, the unit is in it and the rest is manual:

$ tar xzf pistra_1.0.0_linux_amd64.tar.gz
$ sudo install -m 0755 pistra /usr/bin/pistra
$ sudo install -m 0644 systemd/pistra.service /etc/systemd/system/
$ sudo useradd --system --no-create-home --shell /usr/sbin/nologin pistra
$ sudo install -d -m 0750 -o root -g pistra /etc/pistra

pistra version prints the build. That is the first thing to paste into a bug report.

The binary is built without cgo but is not static. The region prefilter the pii detector runs on, and the optional ML detector tier, load shared libraries through dlopen at runtime. Any ordinary Linux userland is fine. A FROM scratch container is not. The image ships on distroless rather than distroless static.

The prefilter is Vectorscan, and the binary expects it on the host. The .deb and .rpm depend on libvectorscan5 and vectorscan respectively. From the tarball, install it yourself (apt install libvectorscan5, dnf install vectorscan). A host that cannot have it says prefilter: pure on each pii detector and gets the built-in pass, which is slower on ordinary text and unbounded on adversarial text. A host that has neither the library nor that line does not start, and the error names the setting.

The config

Two files per host. The deployment document, /etc/pistra/deployment.yaml, is the same bytes on every host, since it is what the hosts agree on and the first one to start seeds the cluster from it:

auth:
  virtual_keys: true
providers:
  - preset: openai
    api_key: ${OPENAI_API_KEY}

The node file, /etc/pistra/pistra.yaml, differs per host in the cluster block. This is the part the chart writes for you from the StatefulSet’s ordinals, and the part where a mistake is quiet.

listen: :8484
deployment: /etc/pistra/deployment.yaml

cluster:
  node_id: pistra-1              # pistra-2, pistra-3 on the others
  data_dir: /var/lib/pistra/raft
  secret: ${PISTRA_CLUSTER_SECRET}
  raft_addr: 0.0.0.0:7000
  forward_addr: 0.0.0.0:7001
  advertise: pistra-1.internal:7000
  forward_advertise: pistra-1.internal:7001
  peers:
    - { id: pistra-1, raft_addr: pistra-1.internal:7000, forward_addr: pistra-1.internal:7001 }
    - { id: pistra-2, raft_addr: pistra-2.internal:7000, forward_addr: pistra-2.internal:7001 }
    - { id: pistra-3, raft_addr: pistra-3.internal:7000, forward_addr: pistra-3.internal:7001 }

Three rules cover most of what goes wrong:

• peers is the same list on all three hosts, and includes the host itself. It seeds the first configuration and nothing after. Once the cluster exists its membership lives in the raft log, and editing this list later cannot change it.
• peers is exclusive with join and bootstrap. A fixed fleet uses peers. A node entering a cluster that is already running uses join. bootstrap starts a cluster of one. Setting peers alongside either is refused at startup, because the mistake it prevents is expensive to unpick. That mistake is a node that meant to join and instead started a second cluster sharing a secret with the first.
• Bind wide, advertise by name. raft_addr is what this process listens on. advertise is what the other two dial and what goes into the raft log. Advertising 0.0.0.0 records an address nobody can reach.

Keep the number of voters odd. Three tolerates one failure. Four tolerates one failure and needs three votes to do it.

Credentials go in an environment file, not the config:

$ sudo install -m 0640 -o root -g pistra /dev/null /etc/pistra/env
$ printf 'OPENAI_API_KEY=sk-...\nPISTRA_CLUSTER_SECRET=%s\n' \
    "$(openssl rand -hex 32)" | sudo tee /etc/pistra/env >/dev/null

The cluster secret is the same string on all three hosts. It authenticates write forwarding and membership changes between them, and each host derives from it the certificate authority under which the three speak TLS to each other on ports 7000 and 7001. No certificate is issued or distributed for those ports.

The systemd unit

The package installs it at /usr/lib/systemd/system/pistra.service. The tarball ships the same file under systemd/. Copy it to /etc/systemd/system/ before editing, a package upgrade replaces the one it owns.

Four directives in it are worth knowing about:

• ExecReload=/bin/kill -HUP $MAINPID is real. pistra re-reads both files on SIGHUP and keeps the previous config serving if either does not parse. The deployment document lands on every node, under its version rule, see the file is still there. A document with a version is also applied by a plain restart when it is newer, so a config-management run that raises the number and restarts the unit is a complete rollout. The certificate manager is built at startup, so changes under tls need restart, not reload.
• StateDirectory=pistra is the raft log at /var/lib/pistra. systemd creates it, owns it to the service user and keeps it across restarts and upgrades. A node whose log vanishes rejoins as a stranger.
• TimeoutStopSec=60, because leadership transfers on a clean stop and that is worth waiting for rather than making the cluster elect its way out.
• AmbientCapabilities=CAP_NET_BIND_SERVICE so the gateway can hold 443 without running as root.

MemoryDenyWriteExecute=yes is deliberately left out. The optional native detector tier maps executable pages through dlopen, and the seccomp filter that option installs blocks it. The gateway starts and then fails to load a detector, reported as a missing library rather than as a sandbox refusal. Do not add it.

The unit reads /etc/pistra/env for the credentials the config references, and reads it optionally, so a deployment with none still starts.

$ sudo systemctl enable --now pistra
$ journalctl -u pistra -f

TLS

Three shapes, and the choice decides who owns the certificate.

pistra terminates, ACME with DNS-01. The gateway obtains and renews its own certificate. The raft leader is the only node that issues, storage replicates through the log, so all three serve the same certificate. DNS-01 needs no inbound reachability, so it works on a private VM where HTTP-01 cannot. If your CA is not Let’s Encrypt it probably wants External Account Binding.

pistra terminates, static files. tls.cert_file and tls.key_file, rotated by whatever already rotates certificates on these hosts. On AWS this is also the path for an ACM certificate issued as exportable: you export the chain and key onto the host and re-export when ACM renews, because nothing pushes a renewed ACM certificate to an instance.

The load balancer terminates. An ALB or Application Gateway holding the certificate, pistra serving plain HTTP behind it. This is the only way to use a normal, non-exportable ACM certificate, and it means the gateway is no longer the TLS owner. It receives whatever the load balancer decided to forward.

The load balancer

Publish only the data plane. Ports 7000 and 7001 are the raft transport and write forwarding between the three hosts. Port 8485 is the control-plane API. Port 9464 is /metrics. None of them belong on a public listener, put them in a security group that admits the three hosts, and your Prometheus for 9464, and nothing else.

Port	What	Who reaches it
8484	Data plane, /v1/, /mcp/, /healthz	The load balancer
7000	Raft transport, mutual TLS under the cluster secret	The other two hosts
7001	Write forwarding, membership, mutual TLS under the cluster secret	The other two hosts
8485	Control-plane API, when enabled	Your own network, never the front door
9464	/metrics (metrics_listen)	Your Prometheus, or nobody if you push OTLP instead

Health-check the load balancer against /healthz on 8484. It answers from the data plane alone and never grows a dependency on a config store or the cluster, so it stays a liveness answer rather than a readiness one.

Raise the idle timeout, or streams will be cut

This is the failure that does not exist on Kubernetes, where a Service is layer 4 and nothing between client and gateway holds an opinion about how long a response may take.

A load balancer closes a connection that has been idle too long, and the defaults are shorter than a model takes to think. Time to first token on a long reasoning request routinely passes a minute under load. During that window the gateway has sent response headers and no body, which is what an idle connection looks like. The client sees a truncated stream and the gateway sees a write to a closed connection. Neither says “load balancer”.

In front	The setting	Default	Notes
AWS ALB	idle_timeout.timeout_seconds	60s	Layer 7: counts HTTP data. Raise it, the maximum is 4000s
AWS NLB	TCP idle timeout	350s	Layer 4, and TCP keepalive resets it
GCP external ALB	backend service timeoutSec	30s	Caps the whole response, not just idleness
Azure Application Gateway	backend requestTimeout	short	Per-request; raise it for streaming

Two things follow.

TCP keepalive does not fix the layer-7 case. pistra’s listener already has it. Go enables TCP keepalive on every accepted connection by default, probing after 15 seconds idle, but an ALB terminates TCP. Probes on the load-balancer-to-gateway leg never appear on the client leg, and the ALB’s idle timer counts HTTP data, not bare ACKs. Keepalive is the right tool in front of an NLB, which passes the flow through, and no help in front of an ALB.

The gateway will not paper over it. A proxy can hold a connection open by writing SSE heartbeat comments during the gap, and pistra does not, because those are bytes the provider never sent. See Passthrough. The timeout is the load balancer’s policy. Change it there.

WebSocket sessions, OpenAI Realtime, Gemini Live, have the same timeout applied to the upgraded connection, but need nothing from you. The endpoints ping each other, and the gateway forwards those frames verbatim like every other frame.

Check the cluster formed

Run this command:

$ journalctl -u pistra | grep 'cluster joined'

All three should name the same leader. The same facts are gauges on each host’s /metrics (port 9464), and they are the ones to alert on:

Series	Healthy	Otherwise
pistra_cluster_leader_known	1 on every host	0: this host is partitioned, or the cluster has lost quorum
pistra_cluster_is_leader	sums to 1	0: no leader; 2: a partition, briefly
pistra_cluster_members{suffrage="voter"}	3	fewer: a host is gone and quorum is thinner than it looks

Autoscaling groups

Put the three hosts in an ASG for replacement, not for scaling. Fix min, max and desired at 3. Quorum wants a stable odd fleet, and a group that scales to four voters has bought nothing, four still tolerates one failure.

A replaced instance comes up with a blank disk and a new address, so it cannot use peers (that seeds a first configuration, and one already exists). Give the replacements join instead, naming the forward addresses of the members that are still up, and advertise set to the new instance’s own name. It is admitted as a nonvoter, catches up, and is promoted. Set dead_after on the fleet if you want the members that were replaced to stop counting toward quorum without an operator saying so.

If a whole group is lost at once and the survivors cannot elect, joining is no longer available to them either, because a join is a write. Recovery runs offline on one survivor’s disk: see Recover a cluster that has lost quorum.