Detect names with a model from the HuggingFace Hub

The pii detector finds identifiers by pattern and checksum. A person’s name has neither, so names come from the nlp detector: a token-classification model, run in the gateway. This is how to get one from the Hub without trusting the Hub.

Pick a model that ships its graph

The detector runs ONNX. Most Hub repositories hold PyTorch weights and no graph, so look for the transformers.js convention, onnx/model.onnx beside tokenizer.json and config.json, which the Xenova/* and onnx-community/* exports follow. For names in ten languages including Arabic, Xenova/bert-base-multilingual-cased-ner-hrl is the export of Davlan/bert-base-multilingual-cased-ner-hrl.

A repository with no graph is refused, with the export command in the message: optimum-cli export onnx --model OWNER/NAME DIR. Push the directory to a repository of your own.

Pin a commit

Run this command:

$ pistra models pin Xenova/bert-base-multilingual-cased-ner-hrl
# https://huggingface.co/Xenova/bert-base-multilingual-cased-ner-hrl at 4c1d7e2a90b1 (resolved from "main")
# graphs in this commit: onnx/model.onnx, onnx/model_quantized.onnx
models:
  models:
    - name: bert-base-multilingual-cased-ner-hrl
      hf:
        repo: Xenova/bert-base-multilingual-cased-ner-hrl
        revision: 4c1d7e2a90b1f0e5d7c3a8b6e2f1d0c9b8a7f6e5

A gateway accepts a commit and nothing else. A branch would let the artifact behind a detector change under a configuration that did not, and pin is the one place a branch is followed, on your machine, when you ask. Pass -model onnx/model_quantized.onnx for the smaller graph. It prints the model: line too.

Reference it from a detector

Add this configuration:

models:
  models:
    - name: names
      hf:
        repo: Xenova/bert-base-multilingual-cased-ner-hrl
        revision: 4c1d7e2a90b1f0e5d7c3a8b6e2f1d0c9b8a7f6e5

guardrails:
  detectors:
    - type: pii
      name: patterns
    - type: nlp
      name: names
      schedule: overlap
      nlp:
        ref: names
  rules:
    - name: no-names-to-the-provider
      when: 'annotations.exists(a, a.entity_type == "PERSON" && a.score > 0.8)'
      requires: [names]
      action: deny

On the first build each node lists the commit on the Hub and takes the Hub’s own digests, a sha256 for every LFS file and a git blob id for the rest. It fetches the three files into cache_dir and verifies every byte. The resolved manifest is cached under the commit, so a restart resolves without the network. Labels come from the repository’s config.json. PER is renamed to PERSON so a rule written against Presidio’s vocabulary keeps working.

schedule: overlap runs the model alongside the provider call, where a transformer belongs. See Run a slow detector without paying for it.

Measure it before it enforces

Choose the checkpoint and the threshold from pistra guardrails eval on your own text, rather than from the model card. For Arabic PERSON through the int8 exports, XLM-R (Davlan/xlm-roberta-base-ner-hrl) beat mBERT on every set, 0.770 against 0.747 exact F1 on ANERcorp and 0.748 against 0.671 on Gulf names in chat turns. Its misses sit on three hazards the per-tag report names: names that are ordinary words, spelling variants, and the clitic it was trained to keep inside the span. Measure a detector has the tables and what each line means.

The graph on an inference server

Add this configuration:

guardrails:
  detectors:
    - type: nlp
      name: names
      nlp:
        ref: names
        remote:
          endpoint: http://triton.internal:8000
          model: ner

With remote, the gateway still tokenizes and still owns the offsets. Only the logits come from the server, over the KServe v2 protocol that Triton, KServe, Seldon and OpenVINO Model Server serve. Only the tokenizer and config.json are resolved and fetched, the graph is never downloaded, and the repository need not hold one. Point hf at the original checkpoint and put whatever export you serve on the server. The gateway cannot check that the server runs that commit’s graph. That pairing is yours to keep.

A gated repository, or a mirror

Add this configuration:

models:
  models:
    - name: names
      hf:
        repo: acme/ner-internal
        revision: 9f3b…
        endpoint: https://artifactory.example/artifactory/api/huggingfaceml/hf-remote
        token: ${secret:HF_TOKEN}

token is a credential. A literal is refused wherever the document travels, like every other one. It is sent only to endpoint, never to a host the download redirects to. endpoint follows the HF_ENDPOINT convention, the one Artifactory and Nexus proxies of the Hub present.

Air-gapped

Resolve once on a machine that can reach the Hub and copy cache_dir across. The manifest and the files are content-addressed and the commit is immutable, so the copy is exactly what the gateway would have fetched. Or write the manifest down yourself:

models:
  models:
    - name: names
      files:
        - {role: model, url: https://files.internal/ner/model.onnx, sha256: "…", size: 711462912}
        - {role: tokenizer, url: https://files.internal/ner/tokenizer.json, sha256: "…", size: 1620}
        - {role: config, url: https://files.internal/ner/config.json, sha256: "…", size: 1017}

Either way the digests are what the gateway believes. The host is only where the bytes are.