What the project measures
Poker is used here as an imperfect-information benchmark. The research questions are about the methods, not about winning: train fast, validate against games with exact answers, and quantify where the result should not be trusted.
The solver
External-sampling MCCFR over a locked heads-up no-limit Hold’em abstraction (legal NLHE sizing, river strength percentiles). It is validated two ways. Exact CFR solutions on Kuhn and Leduc match published equilibria, and the abstraction’s error is measured directly by solving exact subgames and comparing, which is also how the per-street confidence caps were derived. The encoder is imperfect-recall, so the trained strategy is an abstract heuristic strategy, not an equilibrium; a restricted local best response shows it remains measurably exploitable.
The native backend
The training loop is reimplemented in C++17 with bit-exact parity against the Python reference: on a shared random tape, regrets and strategies match to the last bit through 10,000 iterations (FMA contraction disabled to keep IEEE semantics). Warm MCCFR training throughput on the release configuration is 74.2× the Python reference (38.11× from the port, 1.96× more from Phase II profiling). That speed is what made the 2M-iteration release strategy and the three-seed stability analysis practical. Engineering details.
Calibrated abstention
Every information set carries evidence: training visits, movement between checkpoints, disagreement across independently seeded runs, and abstraction-collision dispersion. A gate turns that evidence into ACCEPT / LOW CONFIDENCE / REJECT per lookup, and the product obeys it: a rejected state shows no numbers, only the reasons and the fallback. The gate is calibrated on 8,880 states with exact EV-regret targets, and v2 accepts 1.68× as many decisions as v1 at a similar accepted error rate.
Phase II: controlled experiments
Every comparison has a capacity- or compute-matched control and an independent confirmation run. Exact results come from small or single-street games; none of them changes the release strategy. Null and negative results are listed with the same weight as positive ones.
| question | result | outcome |
|---|---|---|
| Can measured error decide where abstraction capacity goes? | Yes in exact river subgames: 24%–31% lower exploitability than capacity-matched uniform refinement. write-up | POSITIVE |
| Where does native training time go? | 69.5% in river percentile tables; three changes gave 1.96× with an identical digest. write-up | POSITIVE |
| How costly are off-tree bet sizes? | In exact subgames, nearest-in-log-space translation cost 0.60 chips vs 0.076 for pseudo-harmonic mapping (release unchanged). write-up | POSITIVE |
| Does decision-time resolving help, and is it safe? | In exact single-street subgames: the safe gadget never did worse than the blueprint beyond 1e-4 chips at ≥ 1 s; unsafe resolving usually did. write-up | POSITIVE |
| Are red-team failures real, and why? | 17 of 18 top clusters replicated on an independent sample; the failures are abstraction-boundary jumps, and a pre-registered 2M → 4M training run did not fix them. write-up | POSITIVE |
| Can compute go where the solver is unsure? | No replicated gain from prioritized sampling, even with an exact oracle signal. write-up | NULL |
| Is there a better confidence score? | A logistic score raised the false-confidence rate on 8 of 9 held-out games. v2 stays. write-up | NEGATIVE |
| Does more training always help under imperfect recall? | No. In exact Leduc tests one forgetful abstraction got more exploitable as training continued. write-up | NEGATIVE |
| Is policy distance a usable quality proxy? | No. A distilled Leduc strategy 0.064 TV from equilibrium was 3,600× more exploitable. write-up | NEGATIVE |
Charts for the strongest results · Phase II report · 30 follow-up hypotheses
Deep dives
- Evidence: every headline number with its method and source file
- Research lab: interactive surfaces built on precomputed engine output
- Validation ladder (exact CFR → exact subgames → LBR)
- Confidence system v2
- Native solver design and parity methodology
- Full Phase I research details
- System card: intended use, refused uses, known limitations