AI football prediction accuracy
Every forecast is locked before kickoff (normally 24 hours before), hashed into our public ledger and graded on the 90-minute result. Misses stay on the record. Published forecasts and the backtest are reported separately and never mixed.
Model comparison
| Model | n | Correct | RPS | Brier | Log loss |
|---|---|---|---|---|---|
| Final (published)V1: equals the statistical model | 72 | 65.3% | 0.1929 | 0.5128 | 0.8812 |
| Statistical modelElo + Dixon-Coles | 72 | 65.3% | 0.1929 | 0.5128 | 0.8812 |
| ClaudeAI analyst · shadow | 11 | 72.7% | 0.1926 | 0.4929 | 0.8535 |
| CodexAI analyst · shadow | 11 | 72.7% | 0.1936 | 0.4945 | 0.8551 |
| Base rateBenchmark: league H/D/A frequency | 72 | 51.4% | 0.2414 | 0.6147 | 1.0209 |
| Elo onlyBenchmark | 72 | 62.5% | 0.1988 | 0.5291 | 0.9059 |
| Model | n | Correct | RPS | Brier | Log loss |
|---|---|---|---|---|---|
| Final (published)V1: equals the statistical model | 72 | 65.3% | 0.1929 | 0.5128 | 0.8812 |
| Statistical modelElo + Dixon-Coles | 72 | 65.3% | 0.1929 | 0.5128 | 0.8812 |
| ClaudeAI analyst · shadow | 11 | 72.7% | 0.1926 | 0.4929 | 0.8535 |
| CodexAI analyst · shadow | 11 | 72.7% | 0.1936 | 0.4945 | 0.8551 |
| Base rateBenchmark: league H/D/A frequency | 72 | 51.4% | 0.2414 | 0.6147 | 1.0209 |
| Elo onlyBenchmark | 72 | 62.5% | 0.1988 | 0.5291 | 0.9059 |
| Model | n | Correct | RPS | Brier | Log loss |
|---|---|---|---|---|---|
| Final (published)V1: equals the statistical model | 72 | 65.3% | 0.1929 | 0.5128 | 0.8812 |
| Statistical modelElo + Dixon-Coles | 72 | 65.3% | 0.1929 | 0.5128 | 0.8812 |
| ClaudeAI analyst · shadow | 11 | 72.7% | 0.1926 | 0.4929 | 0.8535 |
| CodexAI analyst · shadow | 11 | 72.7% | 0.1936 | 0.4945 | 0.8551 |
| Base rateBenchmark: league H/D/A frequency | 72 | 51.4% | 0.2414 | 0.6147 | 1.0209 |
| Elo onlyBenchmark | 72 | 62.5% | 0.1988 | 0.5291 | 0.9059 |
| Model | n | Correct | RPS | Brier | Log loss |
|---|---|---|---|---|---|
| Final (published)V1: equals the statistical model | 72 | 65.3% | 0.1929 | 0.5128 | 0.8812 |
| Statistical modelElo + Dixon-Coles | 72 | 65.3% | 0.1929 | 0.5128 | 0.8812 |
| ClaudeAI analyst · shadow | 11 | 72.7% | 0.1926 | 0.4929 | 0.8535 |
| CodexAI analyst · shadow | 11 | 72.7% | 0.1936 | 0.4945 | 0.8551 |
| Base rateBenchmark: league H/D/A frequency | 72 | 51.4% | 0.2414 | 0.6147 | 1.0209 |
| Elo onlyBenchmark | 72 | 62.5% | 0.1988 | 0.5291 | 0.9059 |
The n column shows how many matches each row covers, so only compare scores between rows with the same n. Lowest RPS among the rows with the most matches is in bold. Claude and Codex have only run on some matches so far and stay in shadow: they are graded but not used in the published probability (V1). The AI layer section compares them with the statistical model on the same matches.
Calibration
Graded published forecasts in five bins by the top probability, with 95% intervals.
Our picksPerfect calibration
| Top probability | n | Predicted | Observed | 95% CI |
|---|---|---|---|---|
| < 45% | 27 | 41% | 56% | 37–72% |
| 45–55% | 20 | 49% | 60% | 39–78% |
| 55–65% | 15 | 59% | 67% | 42–85% |
| 65–75% | 5 | 71% | 100% | 57–100% |
| ≥ 75% | 5 | 80% | 100% | 57–100% |
By confidence tier
| Tier | n | Correct | Backtest |
|---|---|---|---|
| High | 10 | 100.0% | 70–83% |
| Medium | 22 | 72.7% | about 61% |
| Low | 40 | 52.5% | 39–49% |
High: top probability 65% or more · Medium: 50–65% · Low: under 50%. Totals include Low picks.
By market
"Simple rule" is the benchmark in the note under each market.
| Market | Correct | Simple rule |
|---|---|---|
| Result (1X2)Always pick the home team | 65.3% | 51.4% |
| Correct score: most likely overallExact scores land about one time in eight | 11.1% | – |
| Correct score: most likely in the pick | 8.3% | – |
| Over/Under 2.5 goalsAlways pick the more common side | 59.7% | 61.1% |
| Both teams to scoreClose to the base rate: little skill shown so far | 56.9% | 50.0% |
By league
| League | n | Correct | RPS | Log loss | Sample size note |
|---|---|---|---|---|---|
| Premier League | 4 | 50.0% | 0.242 | 1.103 | Early sample |
| La Liga | 7 | 85.7% | 0.155 | 0.665 | Early sample |
| Serie A | 7 | 85.7% | 0.242 | 0.947 | Early sample |
| Bundesliga | 3 | 33.3% | 0.152 | 1.133 | Early sample |
| Ligue 1 | 5 | 60.0% | 0.191 | 0.786 | Early sample |
| Primeira Liga | 7 | 85.7% | 0.113 | 0.641 | Early sample |
| Scottish Premiership | 1 | 0.0% | 0.557 | 1.877 | Early sample |
| Eredivisie | 3 | 33.3% | 0.291 | 1.073 | Early sample |
| Süper Lig | 4 | 50.0% | 0.153 | 0.844 | Early sample |
| EFL Championship | 12 | 41.7% | 0.234 | 1.089 | Early sample |
| Saudi Pro League | 0 | – | – | – | No graded match yet |
| Champions League | 18 | 83.3% | 0.164 | 0.729 | Early sample |
| Europa League | 1 | 0.0% | 0.140 | 1.287 | Early sample |
| Conference League | 0 | – | – | – | No graded match yet |
Rolling RPS
Average RPS over the last 30 graded matches at each point; lower is better. The window becomes 100 matches once 200 are graded.
Published forecastsBase rate
AI layer – shadow, not yet used in published probabilities
On the 11 graded matches where every layer has a forecast.
| Layer | Correct | RPS | Log loss | RPS vs statistical |
|---|---|---|---|---|
| Statistical model (published) | 72.7% | 0.1916 | 0.8494 | – |
| Claude | 72.7% | 0.1926 | 0.8535 | +0.001095% CI -0.0033 to +0.0045 |
| Codex | 72.7% | 0.1936 | 0.8551 | +0.002095% CI -0.0009 to +0.0068 |
| AI layer combined (shadow) | 72.7% | 0.1919 | 0.8503 | +0.000395% CI -0.0004 to +0.0010 |
Conditions to use the AI layer in published probabilities
- At least 300 graded matches and 6 weeks: 11 graded with the AI layer over 0.1 weeks
- AI layer beats the statistical model on log loss and RPS, on the same matches: AI layer minus statistical: RPS +0.0003, log loss +0.0009 (negative is better; the next condition checks whether the gap is real)
- 95% interval of the RPS difference excludes zero: -0.0004 to +0.0010
- Improvement holds in at least two-thirds of leagues: 1 of 4 leagues with AI forecasts
Not all conditions are met, so the AI layer stays in shadow. Negative RPS difference = better than the statistical model.
Backtest, not live results. The model was run match by match on past seasons using only data available before each kickoff (open historical data (CC0) + live-season results, walk-forward, data cut at D−2). It shows how the method would have done; it is not our published record.
Benchmarks
| Model | Correct | RPS | Brier | Log loss |
|---|---|---|---|---|
| Base rate (league H/D/A frequency) | 44.0% | 0.2304 | 0.6485 | 1.0719 |
| Elo only | 52.0% | 0.2011 | 0.5879 | 0.9863 |
| Dixon-Coles | 52.3% | 0.2007 | 0.5871 | 0.9853 |
| Combined statistical model (published) | 52.1% | 0.2001 | 0.5857 | 0.9827 |
14,184 matches, 2022/23 – 2025/26 (Scotland, Saudi Pro League 2024/25 – 2025/26), 11 leagues, model v1.2-2026-11. Parameters were chosen on earlier seasons and never tuned on this test set.
Backtest calibration
Five bins by the top probability. These bins also drive the tier notes on prediction pages.
Our picksPerfect calibration
| Top probability | n | Predicted | Observed |
|---|---|---|---|
| < 45% | 5,065 | 40% | 39% |
| 45–55% | 4,142 | 50% | 49% |
| 55–65% | 2,554 | 60% | 61% |
| 65–75% | 1,527 | 70% | 70% |
| ≥ 75% | 896 | 80% | 83% |
Grading rules
Graded on the 90-minute score; extra time and penalties are ignored. Every forecast locked before kickoff counts. We lock 24 hours before kickoff, or straight away when a match enters our tracking later than that; each match page shows the actual lock time. Forecasts locked after kickoff are never graded. Corrections are logged, never silently replaced.
Grading methodologyExclusion log
No exclusions so far.
Forecast archive
Every graded forecast with its locked probabilities, result and hash. Download the full ledger as CSV or JSON.
Browse graded resultsOpen the ledgerFor information and entertainment only. Not betting advice. 18+. Predictions are probabilities, not certainties. Responsible gambling