Model Validation
Backtesting Results
How well does our model perform? We backtest against the 2017 and 2019 general elections, aggregating only polls published before each vote and swinging from the previous election's results. 2015 (no prior baseline in our dataset) and 2024 (boundary changes) are excluded, and model parameters are tuned on these same elections, so treat the headline numbers as optimistic.
100%
Largest Party
Elections Tested
4
2015, 2017, 2019, 2024
Avg Seat MAE
20.5
Mean absolute error per party
Largest Party
4/4
Correctly identified
Avg Brier Score
0.13
Lower is better (0 = perfect)
Majority Correct
Majority Missed
Our Prediction
Actual Result
Seat MAE
Vote Share MAE
Brier Score
In 80% CI
Seat Error by Party
Methodology
Each backtest aggregates only polls published before the election and projects seats by applying the implied swing to the previous election's constituency results (2017 from 2015, 2019 from 2017). House effects are hand-set priors in the model configuration; they are not re-estimated per election.
Important caveats:
- 2015 is excluded (no 2010 constituency baseline in our dataset) and 2024 is excluded (boundary changes break constituency matching with 2019)
- Model parameters are tuned in-sample on these same elections, so the headline errors are optimistic
- Reform UK's current surge is unprecedented - no historical analog exists
- Past performance does not guarantee future accuracy