Verdict Observability

Mipiti's verdict pipeline normally answers one question per control: do the assertions submitted for this control suffice to prove it is implemented? The verdict-observability layer adds two more questions, each backed by a cheap LLM call and surfaced where the operator can act on disagreement:

The verdicts run in the background and surface as a divergence list on the model — operator-attributable triage work where the LLM judgment disagrees with what the model currently asserts. The coverage check only observes: it never changes committed coverage or compliance numbers. The group-sufficiency judgement also decides posture: a complete mitigation group counts as mitigating its objective only when a judgement of its current controls affirms that it covers it (see How it relates to evidence verification).

What this protects against

Two failure modes the structural assurance layer can't catch by itself:

Both cases are silent under the structural-only path. Verdict observability surfaces them so the operator can either accept the LLM's view (re-mapping the control, adding a missing layer) or dismiss it (the structural model was right; the LLM was wrong).

Reading the divergence list

A model's divergences live on the Verdict Divergence panel (one per model). Two sections:

Coverage divergences — one row per (control, control objective) pair where the LLM and the recorded mapping disagree, with a kind:

Each row carries the LLM's probability (p_covers), a one-sentence rationale, and the timestamp at which the verdict was computed.

Group-sufficiency divergences — one row per (control objective, mitigation group) where the LLM rates the group as clearly insufficient (p_suffices ≤ 0.3 by default). Each row decorates with:

The confidence band

Not every LLM verdict surfaces. The divergence read endpoint applies a dual-band confidence floor so only confident disagreements show up:

Probabilities in the middle band — p_covers in [0.3, 0.7], or p_suffices > 0.3 — mean the LLM is uncertain. Surfacing those would generate triage work the operator would reasonably ignore. The verdicts are still cached for later inspection; they're just not surfaced as actionable rows.

These are the thresholds for surfacing a row on this panel; see Surfacing thresholds.

Accepting a divergence

Coverage divergences support a one-click accept that applies the LLM's view as a mapping update:

A change reason is required (10-character minimum) — the same audit floor the manual co-mapping PATCH endpoint enforces. The accepted change records the operator's reasoning on the new version row, so the audit trail names what changed and why.

A group-sufficiency divergence has no one-click accept — the natural fix isn't mechanical (it's "add a control" or "restructure the group", which is operator-judgment work). When the divergence list reports a critical group-sufficiency divergence, the action is in the methodology layer, not the click layer.

The adjacent case is different, and it does have a click. An objective reading awaiting_judgement has a built mitigation group and no judgement of whether that group covers it — nothing found it wanting; nothing looked. There is no divergence to accept because there is no verdict to disagree with, and adding or implementing controls cannot change the reason. POST /api/models/{id}/control-objectives/{co_id}/judge asks for the judgement on that one objective. Who pays follows what made the judgement necessary. When the objective's last judgement answered exactly its current controls and is stale only because a platform change altered how objectives are judged, it is at Mipiti's cost (the response's billing.billed_to is platform). Otherwise (the objective was never judged here, was imported, or changed since its last judgement) it is charged to you (billed_to is customer, with the estimate in customer_credits), and refused with 402 when your balance cannot cover it; the refusal's message says how many credits it needs. It runs in the background, and it is not a repair: the judgement can come back insufficient and move the objective to coverage_gap / insufficient_by_design, naming real work. Prefer it over the model-wide recompute below, which re-scores every control and every live objective. It returns already_fresh without queueing when a judgement already exists for the objective's current controls and groups (an objective is judged at most once per state of its inputs), judging without queueing when a judgement for it is already queued, and refuses with 409 while control generation is still running or when no control carries a numbered group for the objective (group them first — there is nothing to judge).

Stale divergences

Once a divergence is accepted (or fixed via any other route — manual remap, control refinement, etc.), the underlying inputs change, and the cached LLM verdict goes stale. The change itself enqueues a re-evaluation of what it changed; on completion, the divergence either updates with new probabilities or disappears entirely. Viewing a model never queues or runs a judgement. When a Mipiti release changes how verdicts are computed, the affected judgements read as awaiting judgement until they are judged again, at Mipiti's cost: in the background for organizations where Mipiti has enabled background re-checking, and otherwise when you ask (Judge…). Attempting to re-accept a divergence that no longer applies (e.g., the mapping was already added by something else) returns an explicit "divergence may be stale — re-fetch" error rather than silently no-op'ing.

If the surfaced list looks stale but no recent mutation has fired, POST /api/models/{id}/verdict-divergence/recompute force-enqueues a fresh re-evaluation for every control and live CO on the model. GET /api/models/{id}/verdict-divergence/recompute/quote estimates it first. You are charged for the coverage checks and for the group-sufficiency judgements of objectives that have not been judged as they now stand; the judgements a platform change made stale are queued at Mipiti's cost. The quote's customer_credits is your share, and the recompute is refused with 402 when your balance cannot cover it. The worker debounces, so a flurry of recompute calls collapses to one re-eval per (model, control, kind) — safe to call from a UI "refresh" button.

Filtering and pagination

The read endpoint supports two query parameters for triage at scale:

The summary block always reflects the full unfiltered state; only pagination.filtered_total reflects the kind-filtered subset. A response with summary.missing_mapping_count = 47 and pagination.filtered_total = 47 on a ?kind=missing_mapping request is the same model state viewed two ways — totals (for dashboards) and the active tab (for triage).

Surfacing thresholds

The confidence band above is what decides which verdicts become rows: p_covers ≥ 0.7 for a missing mapping, p_covers ≤ 0.3 for a spurious one, and p_suffices ≤ 0.3 for an insufficient group. A verdict inside the band is kept and can be inspected, but it is not offered as triage work. The group threshold only decides what this panel lists; whether a group mitigates its objective is decided by its judgement's answer, as below.

How it relates to evidence verification

Verdict observability runs in addition to the regular evidence-sufficiency verdict pipeline (Evidence Verification). The three verdict kinds answer three distinct questions:

Verdict kind Question Affects committed math?
sufficiency Does the evidence on this control suffice to prove it implemented? Yes — feeds the Mitigated rollup.
coverage (this layer) Does this control cover this control objective? No — observation only.
group_sufficiency Do this group's controls defeat the attacker? Yes — a complete group mitigates its objective only when a judgement of its current controls affirms it; until one does, the objective reads awaiting_judgement, and a group judged not to cover it reads as a coverage gap.

Coverage stays an observation because a mapping is authored state: the check flags a disagreement for the operator to accept or dismiss, and never rewrites the mapping itself. Group sufficiency is the question posture actually turns on — whether the controls, together, defeat the attacker — so its answer counts, and the divergence list above is where a group judged insufficient is shown with its members.