Release Gate
One declarative policy (config/policy.yaml), worst-gate-wins: every configured gate is evaluated independently, and the overall decision is the single worst outcome among them. This is the centerpiece of GenAI ReleaseGate — every result below is real, computed evidence.
The candidate satisfies every release requirement — safe to ship.
Evidence is inconclusive or a trade-off requires a human decision before shipping.
A hard release criterion was violated — automatic release block.
The evaluation itself is not trustworthy enough to decide (e.g. too many failed responses to trust the comparison).
Real experiment results
REAL DATAEvery real, persisted comparison — the full release history. V2 through V3.3 were each HELD or REVIEWed for a real reason; V3.4 is the first candidate to pass all 8 gates, on both the dev set and the sealed holdout. The decision engine evaluates every gate and applies the configured policy — no manual override was used to produce any result below, including V3.4's GO.
V2 vs V1 — dev
HOLD120 paired cases · dev split
- — faithfulness: delta -0.1077 crossed the hold threshold -0.0800
- — benign_false_positive_rate: candidate value 0.5556 exceeds the allowed maximum 0.15
| Gate | Status | Reason |
|---|---|---|
| Answer correctness | GO | within configured thresholds (delta=+0.0022) |
| Faithfulness | HOLD | delta -0.1077 crossed the hold threshold -0.0800 |
| Hallucination rate | GO | within configured thresholds (delta=-0.0333) |
| Prompt injection block rate | GO | within configured thresholds (delta=+0.0000) |
| Sensitive-information protection | GO | within configured thresholds (delta=+0.0000) |
| Benign false positives (guardrail suite) | HOLD | candidate value 0.5556 exceeds the allowed maximum 0.15 |
| p95 latency | GO | within configured thresholds (delta=+120.5000) |
| Cost per query | GO | within configured thresholds (delta=-0.0000) |
V3 vs V1 — dev
REVIEW120 paired cases · dev split
- — cost_per_query: delta +25.79% crossed the review threshold +15.00%
| Gate | Status | Reason |
|---|---|---|
| Answer correctness | GO | within configured thresholds (delta=+0.0232) |
| Faithfulness | GO | within configured thresholds (delta=+0.0167) |
| Hallucination rate | GO | within configured thresholds (delta=-0.0339) |
| Prompt injection block rate | GO | within configured thresholds (delta=+0.0000) |
| Sensitive-information protection | GO | within configured thresholds (delta=+0.0000) |
| Benign false positives (guardrail suite) | GO | within configured thresholds (delta=+0.0000) |
| p95 latency | GO | within configured thresholds (delta=-143.4000) |
| Cost per query | REVIEW | delta +25.79% crossed the review threshold +15.00% |
V3.1 vs V1 — dev
REVIEW120 paired cases · dev split
- — faithfulness: delta -0.0601 crossed the review threshold +0.0000
- — cost_per_query: delta +17.02% crossed the review threshold +15.00%
| Gate | Status | Reason |
|---|---|---|
| Answer correctness | GO | within configured thresholds (delta=+0.0739) |
| Faithfulness | REVIEW | delta -0.0601 crossed the review threshold +0.0000 |
| Hallucination rate | GO | within configured thresholds (delta=-0.0339) |
| Prompt injection block rate | GO | within configured thresholds (delta=+0.0000) |
| Sensitive-information protection | GO | within configured thresholds (delta=+0.0000) |
| Benign false positives (guardrail suite) | GO | within configured thresholds (delta=+0.0000) |
| p95 latency | GO | within configured thresholds (delta=-143.2000) |
| Cost per query | REVIEW | delta +17.02% crossed the review threshold +15.00% |
V3.2 vs V1 — dev
REVIEW120 paired cases · dev split
- — faithfulness: delta -0.0160 crossed the review threshold +0.0000
- — cost_per_query: delta +22.54% crossed the review threshold +15.00%
| Gate | Status | Reason |
|---|---|---|
| Answer correctness | GO | within configured thresholds (delta=+0.0707) |
| Faithfulness | REVIEW | delta -0.0160 crossed the review threshold +0.0000 |
| Hallucination rate | GO | within configured thresholds (delta=-0.0339) |
| Prompt injection block rate | GO | within configured thresholds (delta=+0.0000) |
| Sensitive-information protection | GO | within configured thresholds (delta=+0.0000) |
| Benign false positives (guardrail suite) | GO | within configured thresholds (delta=+0.0000) |
| p95 latency | GO | within configured thresholds (delta=-318.4000) |
| Cost per query | REVIEW | delta +22.54% crossed the review threshold +15.00% |
V3 vs V2 — dev
REVIEW120 paired cases · dev split
- — cost_per_query: delta +37.60% crossed the review threshold +15.00%
| Gate | Status | Reason |
|---|---|---|
| Answer correctness | GO | within configured thresholds (delta=+0.0210) |
| Faithfulness | GO | within configured thresholds (delta=+0.1263) |
| Hallucination rate | GO | within configured thresholds (delta=+0.0000) |
| Prompt injection block rate | GO | within configured thresholds (delta=+0.0000) |
| Sensitive-information protection | GO | within configured thresholds (delta=+0.0000) |
| Benign false positives (guardrail suite) | GO | within configured thresholds (delta=-0.4444) |
| p95 latency | GO | within configured thresholds (delta=-278.6000) |
| Cost per query | REVIEW | delta +37.60% crossed the review threshold +15.00% |
V3.3 vs V1 — dev
REVIEW120 paired cases · dev split
- — cost_per_query: delta +16.30% crossed the review threshold +15.00%
| Gate | Status | Reason |
|---|---|---|
| Answer correctness | GO | within configured thresholds (delta=+0.0912) |
| Faithfulness | GO | within configured thresholds (delta=+0.0329) |
| Hallucination rate | GO | within configured thresholds (delta=-0.0508) |
| Prompt injection block rate | GO | within configured thresholds (delta=+0.0000) |
| Sensitive-information protection | GO | within configured thresholds (delta=+0.0000) |
| Benign false positives (guardrail suite) | GO | within configured thresholds (delta=+0.0000) |
| p95 latency | GO | within configured thresholds (delta=-303.2000) |
| Cost per query | REVIEW | delta +16.30% crossed the review threshold +15.00% |
V3.4 vs V1 — dev
GO120 paired cases · dev split
- — GO — all 8 gate(s) passed
| Gate | Status | Reason |
|---|---|---|
| Answer correctness | GO | within configured thresholds (delta=+0.0798) |
| Faithfulness | GO | within configured thresholds (delta=+0.0453) |
| Hallucination rate | GO | within configured thresholds (delta=-0.0339) |
| Prompt injection block rate | GO | within configured thresholds (delta=+0.0000) |
| Sensitive-information protection | GO | within configured thresholds (delta=+0.0000) |
| Benign false positives (guardrail suite) | GO | within configured thresholds (delta=+0.0000) |
| p95 latency | GO | within configured thresholds (delta=-361.2000) |
| Cost per query | GO | within configured thresholds (delta=+0.0000) |
V3 vs V1 — holdout (sealed)
REVIEW40 paired cases · holdout split
- — answer_correctness: delta -0.0753 crossed the review threshold +0.0000
- — cost_per_query: delta +27.83% crossed the review threshold +15.00%
| Gate | Status | Reason |
|---|---|---|
| Answer correctness | REVIEW | delta -0.0753 crossed the review threshold +0.0000 |
| Faithfulness | GO | within configured thresholds (delta=+0.0202) |
| Hallucination rate | GO | within configured thresholds (delta=+0.0000) |
| Prompt injection block rate | GO | within configured thresholds (delta=+0.0000) |
| Sensitive-information protection | GO | within configured thresholds (delta=+0.0000) |
| Benign false positives (guardrail suite) | GO | within configured thresholds (delta=+0.0000) |
| p95 latency | GO | within configured thresholds (delta=+11.4000) |
| Cost per query | REVIEW | delta +27.83% crossed the review threshold +15.00% |
V3.4 vs V1 — holdout (sealed)
GO40 paired cases · holdout split
- — GO — all 8 gate(s) passed
| Gate | Status | Reason |
|---|---|---|
| Answer correctness | GO | within configured thresholds (delta=+0.1011) |
| Faithfulness | GO | within configured thresholds (delta=+0.0281) |
| Hallucination rate | GO | within configured thresholds (delta=-0.0526) |
| Prompt injection block rate | GO | within configured thresholds (delta=+0.0000) |
| Sensitive-information protection | GO | within configured thresholds (delta=+0.0000) |
| Benign false positives (guardrail suite) | GO | within configured thresholds (delta=+0.0000) |
| p95 latency | GO | within configured thresholds (delta=-205.4500) |
| Cost per query | GO | within configured thresholds (delta=+0.0000) |
Decision-engine validation fixture
VALIDATION FIXTUREBefore V3.4's real GO existed, this clearly-labeled synthetic fixture — NOT a real experiment — was used to prove the engine could reach GO at all. Same shape as the backend's own test_scenario_go_deterministic_fixture_every_gate_within_threshold, evaluated against the real config/policy.yaml thresholds. It played no part in V3.4's result and is kept only for engine validation.
What GO looks like (synthetic, not a real result)
Source: tests/unit/test_policy_engine.py — Decision Engine Validation Scenarios (deterministic synthetic fixture, not a real experiment)
| Gate | Status | Synthetic delta |
|---|---|---|
| Answer correctness | GO | +0.05 |
| Faithfulness | GO | +0.05 |
| Hallucination rate | GO | -0.01 |
| Prompt injection block rate | GO | +0.00 |
| Sensitive-information protection | GO | +0.00 |
| Benign false positives (guardrail suite) | GO | +0.00 |
| p95 latency | GO | +0.00 |
| Cost per query | GO | +0.00 |