Updated to reflect change of lead in Montmagny--L'Islet--Kamouraska--Rivière-du-Loup
The election is over, and we have a Conservative majority. In this post, I will discuss where the projection went wrong, and why. A major reason for most projectors' failure to foresee a Conservative majority is that outside Québec, the polls underestimated Tory support by at least 5% in every single region. This performance is frankly embarrassing. My vote share adjustment allowed me to be closer to the mark by bumping the Tory vote estimate by 1-2%, but that wasn't nearly enough.
In a subsequent post, I will compare my final projection to those of other models using poll averages, as well as to predictions made using other methods. It turns out that although projections based on polling averages didn't fare well, they did better than those based on other methodologies. Moreover, among projections based on polling averages, Canadian Election Watch performed very well.
Atlantic Canada (Projected|Actual)
CON - 12|14 ; 32%|38%
LIB - 12|12 ; 28%|29%
NDP - 8|6 ; 34%|30%
Here, we can see that the polls overestimated the NDP vote share, and underestimated the Conservative one. My vote share adjustment from the NDP to the Tories was not enough to compensate for the bias. Had the actual vote shares been known, the model would have been exact. (There would still have been ridings called incorrectly, but overall numbers would have been correct.) As I had predicted, a 14+ seat performance in Atlantic Canada would point to a Conservative majority.
NDP - 44|59 ; 40%|43%
LIB - 7|7 ; 15%|14%
CON - 8|5 ; 17%|17%
BQ - 15|4 ; 25%|23%
IND - 1|0
The polls in Québec were mostly on the mark. They were a little low for the NDP and the Conservatives, and a bit high for the Bloc. My vote share adjustment appropriately compensated for the bias against the Tories, reduced the Bloc problem, but exacerbated the NDP inaccuracy. Unlike in Atlantic Canada, the projected result would have differed from the actual one even if the actual vote shares had been known: the Bloc would still have been given around 10 seats.
Would simply applying a uniform swing, without taking riding polls and potential swing variability into account, performed better? Well, yes and no: as I said previously, uniform swing on polling numbers would have given the Bloc just 5 seats, which would have been very prescient. However, uniform swing on the actual results would have left the Bloc with just one seat, a significant underestimate. Thus, taking other factors into account and increasing the Bloc's predicted count was correct - however, it should not have been done to the extent that everyone, including myself, did.
If you've been reading this blog, you know that I've been fighting against two serious misconceptions of the Bloc vote, namely that it turns out well and that it is efficiently distributed. In fact, Bloc voters, as usual, turned out less than polls suggested, and the Bloc vote was shockingly inefficiently distributed - even much more than I had thought. Indeed, most of the NDP victories were not tight as the model suggested: the Bloc didn't even come close except in a handful of cases. Many political commentators outside Québec either don't know basic electoral facts about Québec (sovereigntists almost always underperform polls) and/or can't process numbers correctly (how can the NDP win only 5-7 seats with 41% in the polls?).
CON - 62|73; 40%|44%
NDP - 22|22; 27%|26%
LIB - 22|11; 26%|25%
Like in Atlantic Canada, the polls failed in Ontario. While the former can be attributed to small sample size, the latter is somewhat embarrassing for the polling industry. The actual result was well outside the margin of error of the last polls from Forum, EKOS and Léger, and it was right on the upper edge for Angus Reid. Other pollsters were probably saved by their small sample size, which implied wider confidence intervals: no pollster came within 3% of the Conservative tally in Ontario. My vote share adjustment only mitigated the problem by a tiny bit.
On actual results, the model with the GTA adjustment would have done quite well: 72-20-14. It would have been farther off without the GTA adjustment, which on actual results, shifted 3 net seats from the Liberals to the Conservatives. This adjustment has been heavily emphasized on this blog, while it was barely mentioned elsewhere.
CON - 21|24 - 50%|55%
NDP - 5|2 - 29%|29%
LIB - 2|2 - 15%|13%
Once again, the pollsters severely underestimated the Tory vote, and my pro-Conservative vote share adjustment was not nearly big enough. On actual popular vote, the projected count would have been closer, at 23-4-1.
CON - 27|27 - 63%|67%
NDP - 1|1 - 18%|17%
The projection was, unsurprisingly, correct in Alberta, though the pollsters underestimated the Conservative vote here as well.
CON - 21|21 - 41%|46%
NDP - 13|12 - 33%|33%
LIB - 2|2 - 16%|13%
GRN - 0|1 - 8%|8%
Once again, the Tories did much better than predicted. Here, however, the polling inaccuracy did not significantly impact my projection. On the actual provincial split, the model would have given 22-12-2, though I might have made it 23-11-2 via a risk adjustment.
CON - 152|166 - 37.3%|39.6%
NDP - 94|103 - 30.5%|30.6%
LIB - 46|34 - 20.0%|18.9%
BQ - 15|4 - 6.3%|6.0%
GRN - 0|1 - 4.9%|3.9%
IND - 1|0
Overall, the polls underestimated the Conservative vote by 3.7% - and, I repeat, that was over 5% everywhere outside Québec. They would have been even worse had they accounted for the fact that turnout is significantly lower in Alberta than elsewhere. My vote share adjustment shrank the gap to 2.3%.
On actual vote splits, I would have projected 166-167 Conservative seats - bang on! The Liberal count would have been 36, which is very close as well. The Bloc would still have been overestimated (though by less), and the NDP would still have been underestimated because although I kept repeating that the NDP vote is efficient in Québec, even I did not grasp the full extent of it. Outside Québec, the projected count would have come within 3 seats of the actual count for all parties. Based on these observations, I believe that the model was, in fact, very solid.
As for the riding-by-riding calls, I was correct in 259 cases out of 308 = 84.1%. (Etobicoke Centre and Westmount--Ville-Marie flipped late yesterday, both away from the projected winner. I am, however, very pleased at the change in my childhood riding. Update: Montmagny--L'Islet--Kamouraska--Rivière-du-Loup also flipped away from the projection.) This is worse than how such a model usually performs - you would have expected about 280 correct calls - but obviously that was caused by the inaccurate polls and the great uncertainty in Québec.