Evidence
Every headline number on this site, with what it means, how it was measured, and where the artifact lives. The site reads these from a machine-checked claims registry — numbers cannot silently go stale.
| claim | value | meaning & method | source |
|---|---|---|---|
| native_speedup | 74.2× | Native C++17 training backend vs the Python reference warm median of 3 trials, 1625 vs 21.9 it/s (Phase II profile-driven native optimization; Phase I was 38.11x) | native_mccfr_v2.json |
| native_it_per_s | 1625.02 it/s | Native training throughput warm median | native_mccfr_v2.json |
| python_it_per_s | 21.9 it/s | Python reference throughput warm median | native_mccfr_v2.json |
| release_iterations | 2,000,000 iterations | Training scale of the promoted strategy external-sampling MCCFR on the locked v2 abstraction; exact checkpoint resume | solver_config.py |
| training_seeds | 3 independent seeds | Independent training runs training uncertainty is measured across seeds instead of assumed away | |
| evaluator_hands | 2,598,960 5-card hands | Exhaustive five-card evaluator parity every possible 5-card hand compared between Python and C++ (CI-tested) | native evaluator parity tests |
| calibration_rows | 8,880 infosets | Confidence calibration states infosets from nine exactly solved reduced games with TRUE EV-regret targets, leave-one-game-out | confidence_signal_quality.json |
| coverage_gain | 1.68× | Coverage gained by confidence v2 at equal accepted risk promoted v2 rule (sd<0.2, recent mv<0.1, visits>=20) coverage vs the v1 ACCEPT share, held-out games; accepted wrong-rate 3.0% vs 3.3% | confidence_signal_quality.json |
| test_count | 1,008 tests | Tests collected at this commit counted by pytest --collect-only when the site data was generated — never a stale number | pytest --collect-only -q (this commit) |
| stored_infosets | 144,025 infosets | Information sets in the release strategy the strategy artifact's key table | holdem_v2_native_seed0_2m.npz |
| evaluator_hands_7card | 133,784,560 7-card hands | Exhaustive seven-card evaluator parity exhaustive native vs Python parity, 0 mismatches | evaluator_stress.json |
| parity_iterations | 10,000 training iterations | Python ↔ C++ training parity Python vs native on one shared random tape, bit-identical (CI runs 1,000) | native_correctness_stress.json |
| native_speedup_phase1 | 38.11× | Historical: first native port HISTORICAL: first native port before Phase II profiling, warm median of 3 trials | native_mccfr_v1.json |
| native_it_per_s_phase1 | 836.99 it/s | Historical: first native port throughput HISTORICAL: first native port | native_mccfr_v1.json |
The validation ladder
How do you know a solver is working when the full game cannot be solved? Each rung checks something the previous rung cannot:
- Kuhn poker — exact CFR reaches the known equilibrium (game value −1/18)
- Leduc poker — larger exact game; convergence + known-solution checks
- Exact reduced Hold'em subgames — measures the ABSTRACTION's error directly, not just the solver's
- Python ↔ C++ random-tape parity — 10,000 iterations bitwise identical: same trajectories, regrets, strategies (1,000 in CI)
- Exhaustive evaluator parity — all 2,598,960 five-, 20,358,520 six- and 133,784,560 seven-card hands
- Checkpoint identity — train(a)+save+load+train(b) bit-identical to train(a+b)
- Three independent seeds — cross-run disagreement measured, feeding the gate
- Confidence calibration — 8,880 exact EV-regret targets; leave-one-game-out
- Restricted LBR — an honest lower bound: at least ~100 bb/100 exploitability remains in the full game
Deep dives: validation methodology · native parity · confidence v2