Invariants

Sigil’s behavior is governed by four invariants. If any of these is violated, the system fails closed — no ALLOW, ever. These are not configuration options. They are not feature flags. They are properties of the design that testing enforces.

1. Fail-closed

Any error — any panic, any unhandled exception, any unreachable code path, any missing config, any unparseable scenario, any Docker failure, any LLM timeout — downgrades the decision to REVIEW or BLOCK.

There is no code path in Sigil that converts an error into ALLOW. In 0.36.0 the decide gate was put under Kani proofs, and every path a 2026-09 audit found failing open now fails closed: a sigil decide error inside sigil ci, subset and holdout-less evals, PRs that edit Sigil’s own policy, unbound artifacts and endpoints, OPA errors, out-of-range or non-finite scores, and configured scanners that do not run.

2. Isolation wall

Holdout scenario content — the scenario source, its identifiers, its failure messages, its expected values, anything that could let the agent infer what the holdout tests — never appears in output the agent can read.

The feedback generator is the single source of truth for this boundary. It receives only opaque step labels (step_1, step_2) and pass/fail bits. Integration tests assert this by running adversarial inputs through the feedback generator and checking that holdout content never leaks.

3. Reproducibility

Every eval carries a full provenance tuple:

• Artifact digest — content-addressed hash of the PR image.
• Baseline digest — content-addressed hash of the baseline image.
• Scenario-set hash — hash of the full (visible + holdout) scenario bundle used.
• RNG seed — the master seed from which all sub-seeds are derived deterministically.
• Control ref hash — hash of the policy configuration that governed the decision.
• Evaluator version — semver of the Sigil binary that ran the eval.

Given this tuple, any evaluator can re-run the same eval and get the same result bit-for-bit, modulo non-deterministic judge responses (which are themselves seeded).

4. Freshness gate

Before any ALLOW, sigil decide syncs the ledger, and the sync must finish within the staleness window ([ledger] max_staleness_for_allow, default 60 seconds). Decide then re-checks what the sync pulled: the hash chain, trust (which can only go down), and whether a newer eval of the same artifact arrived. If the sync fails, is too slow, or any re-check fails, the decision downgrades to REVIEW.

This prevents a split-brain condition where two evaluators independently decide ALLOW based on divergent ledger state. It also means that any ALLOW is backed by a ledger entry that is globally consistent at the moment of the decision.

Why these four

These are the smallest set of properties that prevent the failure modes that make evaluation systems untrustworthy:

• Without fail-closed, bugs in Sigil become green lights for bad code.
• Without isolation, the agent learns to game the evaluator.
• Without reproducibility, evaluations cannot be audited, and disputes cannot be resolved.
• Without freshness, evaluations become unmoored from the shared source of truth.

Any other property is nice to have. These four are load-bearing.