Comparison of 2015 Projections

This is a post I started writing after the 2015 election, but never finished. I provide it in its unfinished state, "for the record."

2015 Final Projections (italics = not model-based)
178-115-44-  1-0-0 (37%, 31%, 22%, 4%, 4%) CVM Election Model
177-  95-53-11-1-1 (38.0%, 30.9%, 21.3%, 5.7%, 3.4%) Teddy on Politics
160-120-50-  7-1-0 (36.7%, 32.0%, 20.4%, 5.2%, 4.0%) The Signal
149-105-81-  2-1-0 David Akin's Predictionator
147-115-67-  7-2-0 (39.3%, 32.4%, 19.4%, 4.1%, 4.1%) Sauder Prediction Market
146-118-66-  7-1-0 (37.2%, 30.9%, 21.7%, 4.9%, 4.4%) ThreeHundredEight
142-119-66-10-1-0 (37.3%, 32.4%, 20.1%, 4.9%, 4.3%) Canadian Election Watch
142-116-68-11-1-0 Election Atlas
140-115-79-  3-1-0 LISPOP
138-117-76-  6-1-0 (36.8%, 31.8%, 22.7%, 4.2%, 4.5%) Le calcul électoral
138-120-75-  1-1-0 Election Almanac
137-120-75-  8-1-0 (36.8%, 32.5%, 21.4%, 4.6%, 4.1%) Too Close to Call
128-120-83-  5-2-0 Election Prediction Project

Sum of absolute seat deviations (divided by 2)
11 Teddy on Politics
16 CVM Election Model
27 The Signal
40 Sauder Prediction Market
41 ThreeHundredEight
42 Canadian Election Watch
42 Election Atlas
43 David Akin's Predictionator
49 Too Close to Call
50 Le calcul électoral
51 LISPOP
55 Election Almanac
61 Election Prediction Project

This time, I was in the middle of the pack. (Here is how I did in 2011.) Note that my projection without the turnout adjustment was off by 34, which is significantly better than average.

Three models did particularly well by this measure. If I understand correctly, Teddy on Politics injects a fair bit of personal judgment into his projections - they're not as heavily based on polls as the other projections. This time, he got things very right (which happens fairly often, if you've been following him, although there are also big misses), foreseeing which way the wind was blowing. The CVM election model appeared to have been lucky: the seat model was too extreme, but the Liberal support was underestimated, and the two errors canceled out. For example, it correctly projected the Liberal Atlantic sweep, but on percentages with which the sweep would not have happened (as the Liberals won some seats narrowly). And it incorrectly predicted an NDP wipeout in Ontario, which didn't even come close to happening.

This leaves us with The Signal, which did very well. A big part of this is that it correctly foresaw that the Liberal Québec vote would be quite efficient. Unfortunately, its description of methodology is not very detailed, so it is hard to see why exactly it did better than everyone else - especially given that it had the lowest Liberal popular vote projection. Still, congrats to The Signal!

Average absolute popular vote deviation for 5 parties
0.46 Sauder Prediction Market
0.84 Canadian Election Watch
0.94 The Signal
1.02 Teddy on Politics
1.16 Too Close to Call
1.30 ThreeHundredEight
1.40 CVM Election Model
1.48 Le calcul électoral

By this measure, I provided the best poll-based projection. The unadjusted numbers were off by 0.90 on average. The polls did quite well this time, and adjusting polls to reflect the latest trends helped. In fact, had my adjustments not been so cautious, the vote (and seat) projections would have been even more accurate - perhaps close to the almost-on-the-dot Sauder market prediction.

Speaking of the Sauder prediction market, feeding its popular vote prediction into most seat models would probably have produced more accurate seat counts than it predicted. This suggests that while market participants were good at guessing the overall vote, the seat count conversion proved challenging. This is why quantitative seat models help!

Number of ridings correctly projected
275 Teddy on Politics
269 ThreeHundredEight
268 Canadian Election Watch
261 Election Prediction Project