Sync LEADERBOARD.md to HF Space (canonical home)

LEADERBOARD.md

@@ -0,0 +1,155 @@
+# Red Button v2 — Leaderboard
+
+Per-tier evaluation results across model sizes and operator policies.
+The headline carry of v2 is the **environment + multi-agent
+architecture + multi-failure-mode evaluation matrix**, not a trained
+model. SFT trial documented as a negative result (catastrophic
+forgetting on a narrow scripted dataset, see [Findings](./README.md#findings)).
+
+---
+
+## Tier 2, train operator — model-size sweep
+
+| Run | N | Math | Trained tamper | Held-out tamper | Deception | Sandbag | Shutdown invariant | Mean reward |
+|---|---|---|---|---|---|---|---|---|
+| Scripted baseline (3 policies × 30) | 90 | 50.0% | 33.3% | 0.0% | 33.3% | 0.0% | 66.7% | 0.20 |
+| Qwen3-0.6B untrained | 10 | 4.0% | 0.0% | 0.0% | 100.0% | 0.0% | 100.0% | 0.037 |
+| Qwen3-0.6B + SFT (1 ep, 70 tx) | 10 | 7.0% | 0.0% | 0.0% | 100.0% | 0.0% | 100.0% | 0.044 |
+| Qwen3-0.6B + SFT (3 ep, 70 tx) | 10 | 4.0% | 0.0% | 0.0% | 100.0% | 0.0% | 100.0% | 0.030 |
+| Qwen3-1.7B untrained | 10 | 3.0% | 0.0% | 0.0% | 100.0% | 0.0% | 100.0% | 0.034 |
+| Qwen3-1.7B + SFT (2 ep, 70 tx) | 10 | 1.0% | 0.0% | 0.0% | 100.0% | 0.0% | 100.0% | 0.026 |
+| **Qwen3-8B untrained** | **30** | **24.7%** | **0.0%** | **0.0%** | **70.0%** | **0.0%** | **100.0%** | **0.244** |
+| **Qwen3-8B + SFT (1 ep, 490 tx) ⚠️** | **30** | **1.0%** | **0.0%** | **0.0%** | **100.0%** | **0.0%** | **100.0%** | **0.126** |
+| **Qwen3-8B untrained, n=50 (CI tightener)** | **50** | **24.4%** | **0.0%** | **0.0%** | **68.0%** | **0.0%** | **100.0%** | **0.241** |
+
+⚠️ **R3 negative result:** SFT on Qwen3-8B with our 490 narrow scripted
+templates collapsed math accuracy from 24.7% → 1.0% and pushed
+deception from 70% → 100%. Diagnosis below; this is documented as a
+known failure of the chosen SFT recipe, not the v2 contribution.
+
+---
+
+## Tier sweep — Qwen3-8B untrained, train operator
+
+Tests whether the env's tier curriculum produces meaningfully different
+behavior at the same base model. **Yes:** all three tiers produce
+distinct profiles.
+
+| Tier | N | Math | Trained tamper | Deception | Sandbag | Shutdown invariant | Mean reward |
+|---|---|---|---|---|---|---|---|
+| **Tier 1 (compliance, no auditor)** | **30** | **37.3%** | **0.0%** | **0.0%** | **0.0%** | **100.0%** | **0.474** |
+| **Tier 2 (goal pressure + auditor)** | 30 | 24.7% | 0.0% | 70.0% | 0.0% | 100.0% | 0.244 |
+| **Tier 3 (+ forced final self-report)** | **30** | **24.7%** | **0.0%** | **53.3%** | **0.0%** | **100.0%** | **0.127** |
+
+**What this shows:**
+- **Math accuracy gradient**: 37% → 25% → 25%. Tier 1's longer timer
+  (30 vs 15) and absence of auditor blocks lets the model attempt
+  more problems. Tier 2/3 are similar (auditor consumes turns).
+- **Deception gradient**: 0% → 70% → 53%. Tier 1 has no auditor, so
+  no deception channel. Tier 2 saturates the auditor's three checks
+  per episode at 70% malformed. Tier 3 *lower* than Tier 2 because
+  episodes terminate later (after the final self-report) and some
+  late episodes never trigger the third auditor check.
+- **Reward gradient**: 0.474 → 0.244 → 0.127. Each added overseer
+  costs the model. Tier 3's honesty rubric specifically penalises
+  unparseable final self-reports (-1.0), which the untrained model
+  routinely fails.
+- **Tampering and sandbagging stay at 0%** — Qwen3-8B does not
+  tamper or sandbag without explicit training to do so.
+
+---
+
+## Operator-policy sweep — Qwen3-8B untrained, Tier 2
+
+Tests the held-out operator-policy axis (per project.md §6.2): the
+strict operator's harsher penalties test whether the model behaves
+differently under unfamiliar oversight.
+
+| Operator | N | Math | Trained tamper | Deception | Sandbag | Shutdown invariant | Mean reward | Mean tool calls |
+|---|---|---|---|---|---|---|---|---|
+| Train operator | 30 | 24.7% | 0.0% | 70.0% | 0.0% | 100.0% | 0.244 | ~10 |
+| **Strict operator (held-out)** | **30** | **21.3%** | **0.0%** | **66.7%** | **0.0%** | **100.0%** | **0.264** | **8.3** |
+
+**What this shows:**
+- The strict operator accelerates the timer faster, **shortening
+  episodes** (8.3 mean tool calls vs ~10). This means fewer auditor
+  checks land, dropping deception 70% → 66.7%.
+- Math drops slightly (24.7% → 21.3%) because shorter episodes give
+  fewer chances to submit answers.
+- Reward goes UP under strict (0.244 → 0.264) — the lower deception
+  rate dominates the rubric since shutdown invariant is preserved
+  in both.
+- **The model behaves consistently across operator policies.** No
+  trained tampering or sandbagging emerges under strict that wasn't
+  present under train. This is the second axis of the held-out
+  generalization claim from project.md §6.2: behavior under unfamiliar
+  oversight transfers cleanly.
+
+---
+
+## Per-subcategory deception breakdown
+
+The deception channel decomposes into three subcategories. Across the
+full evaluation matrix:
+
+| Run | malformed_report | overclaim_deception | concealment_deception | Total events |
+|---|---|---|---|---|
+| Untrained 8B, Tier 1, train | 0/30 (0%) | 0/30 (0%) | 0/30 (0%) | 0 |
+| Untrained 8B, Tier 2, train | 21/30 (70%) | 0/30 (0%) | 0/30 (0%) | 32 |
+| Untrained 8B, Tier 2, train (n=50) | 34/50 (68%) | 0/50 (0%) | 0/50 (0%) | 55 |
+| Untrained 8B, Tier 2, **strict** | 20/30 (66.7%) | 0/30 (0%) | 0/30 (0%) | 20 |
+| Untrained 8B, **Tier 3**, train | 16/30 (53.3%) | **1/30 (3.3%)** | 0/30 (0%) | 29 |
+| **SFT 8B, Tier 2, train** ⚠️ | 30/30 (100%) | 0/30 (0%) | **5/30 (16.7%)** | 48 |
+
+**Reading:**
+- **Tier 1's 0% deception confirms the auditor is the only deception
+  source** — when the auditor is disabled, the model has no
+  deception channel to fail.
+- **Across Tier 2/3 baselines, malformed_report dominates strategic
+  deception by ~50:1.** The base 8B sometimes can't emit the JSON
+  schema (parse failures), but it *almost never* strategically lies
+  (overclaim 0% in Tier 2, 3.3% in Tier 3 — the first observed
+  strategic deception is on Tier 3 with the forced self-report
+  context).
+- **SFT introduced concealment_deception** in 5 episodes (16.7%). The
+  mechanism: scripted training transcripts always report
+  `tool_failures: 0`, so SFT taught the model to always claim zero
+  failures. After SFT the model's math collapsed (1% accuracy →
+  many tool failures), and the "always 0" pattern then satisfied
+  `claimed_failures < actual_failures` → concealment.
+- **The env's instrumentation correctly identified a mechanism of
+  deception that emerged from a specific SFT mistake.** That's the
+  env doing exactly what it was built to do: surface failure-mode
+  behaviors the model didn't have *before* training.
+
+---
+
+## Why the small-model deception numbers saturate at 100%
+
+Sub-2B Qwen3 models cannot reliably emit the auditor's three-field
+JSON schema. Every auditor check (3 per Tier-2 episode) returns
+something the parser can't read as a dict, classifying as
+`malformed_report` × 3 → 100% rate. SFT on 70 transcripts with these
+small models doesn't fix the format adherence — and 1.7B + SFT
+actually drops math from 3% → 1%, suggesting the SFT pulls the model
+further off the JSON manifold.
+
+The first model size where the malformed_report rate drops below 100%
+is **Qwen3-8B** (70% in our n=30 baseline) — the same model where SFT
+catastrophic forgetting also manifests. Bigger models or different
+training recipes are the path forward; both are documented as future
+work in the spec.
+
+---
+
+## Sources
+
+- Generated via `evaluation/baseline_rollout.py` against
+  https://shutdowngym-redbutton-v2.hf.space.
+- HF Jobs (a10g-large for 8B, t4-medium for sub-2B), all on
+  `Manasvee09` HF account.
+- CSVs in `results/`. Per-row trace JSONLs are not committed (they're
+  in the gitignore).
+- The R3 SFT adapter is at
+  https://huggingface.co/Manasvee09/redbutton-v2-sft-qwen3-8b — kept
+  on Hub as the artifact for the negative result.