Latest national poll median date: October 20
Projections reflect recent polling graciously made publicly available by pollsters and media organizations. I am not a pollster, and derive no income from this blog.

Saturday, August 31, 2019

Trends 2019: First Instalment

Update Sept. 1: The trends now reflect the EKOS poll released Sept. 1, which has the same field dates as the Léger poll released Aug. 31.

Below are the projection trends since the first projection issued in early August. The plots show how the projection would have evolved with the current model, had the polls been released in the order in which they were conducted. This means that the seat counts are recalculated when an older poll is published after a newer one, or when I tweak the methodology. The result is not quite the same as the retroactive best guess at what the situation was at the time (since I don't use the subsequent data to "backcast" how things were), but it's closer than just plotting projections as they're issued, as most other sites do.

Details for how these trends were generated are given at the end of the post.

The trends show the following evolution in late July and August:

- The CONs lost their seat lead in late July, and were remarkably flat around 145 seats and 36% support during most of August. The latest EKOS ON numbers and Léger national polls, however, ended this period of stability. It will be interesting to see whether this sticks, like the drop in late July.

- The LIBs have generally been in the 33-34% range and in the low-to-mid 150s in terms of seats. The Dion report had no discernible effect. The latest EKOS ON numbers and Léger polls pushed them above 34% and to nearly 160 seats for the first time. Again, it will be interesting to see if this rise is confirmed or is just a blip.

- The NDP has been flat around 20 seats and 13% support. But don't let the flat trend fool you: there is wide disagreement among pollsters regarding where the NDP is. Update: Also, although the monthly trend is flat, it will be interesting to see if the drop in the past two weeks continues.

- The BQ has quietly made some gains, passing the critical threshold of 12 seats. However, whether it would, in fact, win 12 or more seats remains essentially a coin flip. Many BQ races, both projected wins and losses, are very tight, so the uncertainty is very large relative to the seat count.

- The GRNs have been quietly fading in terms of vote share, for example falling to fourth place in BC. Remember that in 2015, the NDP was riding high during the summer thanks in part to Rachel Notley's win in AB, only to be sorely disappointed in the fall. Does the same fate await the Greens after encouraging results in recent provincial elections? Or do they manage to leverage Elizabeth May's popularity and eat into the Liberal/NDP vote, thereby earning the balance of power (or causing a Conservative majority)?

Going forward, I plan to update trends every week or two.

Here are some methodological details for these trends:
- The dates correspond to the midpoint of the most recent poll in the projection. Polls with midpoint dates within a day or two may be grouped. For example, there are Abacus and DART polls with a midpoint date of Aug. 16.5, and Léger and Ipsos polls with a midpoint date of Aug. 17.5. All of them are first included at the Aug. 17 point.
- Methodological updates (e.g. change in turnout adjustment) and adjustments to ridings identified as special cases lead to recalculations (even if those adjustments informed by subsequent polls, as long as those polls likely mainly reflect things baked in before the first projection). Poll-based adjustments to other ridings or to sub-regions do not lead to recalculations, as they may reflect actual evolution of the situation.
- Regional, sub-regional and riding-level polls are included in the numbers at the same time as the nearest national poll (except, as mentioned above, for certain polls in special-case ridings). For example, the Corbett ON poll with a midpoint date of August 12 begins to be included at the August 11 point, as the nearest subsequent national poll has a midpoint date of August 16.5.

Projection Update: Léger 8/27-29, Mainstreet Riding Poll

The following polls have been added to the model:
Léger's 8/27-29 national poll (current weight among national polls: 28%)
Mainstreet's 8/27-28 Vancouver Granville poll
For a full list of included polls, see previous Projection posts.

The Léger poll is the first national poll in 4 weeks (since Mainstreet's late July poll) showing the Liberals ahead, on the basis of strong Atlantic and BC results. The numbers in all other regions are very close to current polling averages. The poll oversampled QC (1,001 respondents), and provided a very useful breakdown of the province, which I used to update sub-provincial adjustments.

The Mainstreet poll is very close to what the model was expecting - a bit better for the LIBs, and a bit worse for the CONs. Therefore, in Vancouver Granville, the adjustment for the LIBs is changed from -10% to -9%, and the adjustment for CONs is changed from -10% to -11%. (Update Sept. 1 and 5: The adjustments have changed slightly after contemporaneous EKOS and Abacus BC numbers subsequently come out.)

Projection as of the latest national poll (midpoint: August 28)
LIB - 158.8 (34.3%)
CON - 141.9 (35.9%)
NDP - 19.6 (12.7%)
BQ - 13.2 (4.4%)
GRN - 3.6 (9.5%)

PPC - 0.5
IND - 0.5

If you're new to my 2019 projections, view key interpretive information here.

The LIBs have gained 5 seats in the projection in the past week, mostly from the CONs. This is not at all a big change (the CON-LIB gap shrank by less than 1 point), but still the largest we've seen in August. I will likely write a post on projection trends (with past projections retroactively computed to net out the methodological tweaks I've been making) this weekend, so stay tuned!

Seats changing hands since the last projection:
- LIBs regain Charlottetown from CONs.
- BQ takes Laurentides--Labelle and Laurier--Sainte-Marie from LIBs.
- LIBs take Vancouver Granville from Wilson-Raybould.
- LIBs take Victoria from GRNs.

Please enjoy the updated images below:

Polling-Related Adjustments 2019, Round 2

This post reviews polls of smaller areas than the six standard polling regions (BC, AB, SK/MB, ON, QC, Atlantic), and polls providing breakdowns of smaller areas. Resulting adjustments to the model are indicated in bold. You can view the initial round of polling-related adjustments here.

Update Sept. 5: Justason's Vancouver Granville and Vancouver polls added These polls were sponsored by the Wilson-Raybould campaign, and have therefore been removed.

Update Sept. 8: Abacus' ON breakdown added.

Update Sept. 10: MQO's summer polls added, methodology for Atlantic adjustments slightly revised.

Update Sept. 11: Forum's QC poll and 4 Mainstreet riding polls in QC added.

Update Sept. 12: QC breakdown from Léger's 9/6-9 national poll added.

Update Sept. 18: Added comment about regional adjustment applied to Portneuf--Jacques-Cartier.

I. Regional Polls

Atlantic Canada
MQO Summer 2019 polls (7/25-31 NL7/31-8/6 PEI8/6-14 NB, 8/20-27 NS)
Narrative's 7/31-8/22 poll
These polls largely confirm the initial round of adjustments. The biggest tweak is an increase for the CONs in PEI, which is useful for them as PEI is currently unusually competitive. The updated adjustments are (note: may be re-updated when the MQO NS numbers come out):
NL: CON +9.5, NDP -1.5, LIB -2.5, GRN -5.5
NS: CON -3, NDP +3, LIB -1.5, GRN +1.5
NB: CON -3.5, NDP -2, LIB +4.5, GRN +1
PE: CON +4, NDP -3, LIB -5, GRN +4
The final set of numbers from MQO shows a widening LIB lead in NS, to a size similar to what Narrative shows. This caused a slight change in NS adjustments. At the same time, I have reviewed the adjustments methodology to put more weights on recent polls and better account for differences in the size of electorate between ridings - this caused changes favouring the CONs in NL and hurting the LIBs in PE:
NL: CON +11, NDP -2.5, LIB -3.5, GRN -5
NS: CON -3.5, NDP +3, LIB -1, GRN +1.5
PE: CON +4, NDP -1.5, LIB -7, GRN +4.5

Forum's 8/26-29 poll
Léger's 8/27-29 poll
These polls large confirm the first round of adjustments: the CONs are gaining disproportionately outside the Quebec City and Montreal CMAs, and are likely not gaining at all in Quebec City. As the first round was cautious, I have significantly increased the magnitude of these adjustments, which are now:
Quebec City CMA: CON -7 -9, NDP +0.5, LIB +3 (1.5 was typo), GRN -1, BQ +0.5
Montreal CMA: CON -1 (excluding far suburbs), NDP +0.5 (excluding far suburbs), LIB +1, GRN +0.5 (Island of Montreal only), BQ -0.5
Rest of Quebec North: CON +3 (3.5 was typo), NDP -0.5 -1, LIB -2 (-2.5 was typo), BQ +0.5
Rest of Quebec South: CON +3.5 (3 was typo) +4, NDP -0.5, LIB -2.5 (-2 was typo), GRN -0.5, BQ +0.5
(Salaberry--Suroît is mostly outside of Montreal CMA but shares some characteristics with it. Since the Montreal CMA and Rest of QC S adjustments go in opposite directions, I have left this riding unadjusted. Moreover, due to the large difference between Quebec City and Rest of QC North adjustments, I am computing the adjustment in Beauport--Côte-de-Beaupré--Île-d'Orléans--Charlevoix and Portneuf--Jacques-Cartier according to their rough population distribution: 55% and 58% respectively in Quebec City CMA, 45% and 42% outside.)
This can help the Tories make gains outside the Quebec City area, but the two Liberal seats in Quebec City are now unlikely to flip. In fact, the projection for Québec is now within 2 points for all parties of the riding poll published by Mainstreet in July. I had refrained from making an adjustment based on the riding poll because I through the regional adjustment might end up doing the job, and that's exactly what happened.

Léger's 9/6-9 poll
Further tweaks to the adjustment in bold:
Quebec City CMA: CON -10, NDP +0.5, LIB +3.5, BQ +0.5
Montreal CMA: CON -1 (excluding far suburbs), NDP +1.5 (Island of Montreal), +1 (near suburbs), LIB +1 (Island of Montreal), +0.5 (near suburbs), GRN +0.5 (Island of Montreal only), BQ -0.5
Rest of Quebec North: CON +3.5, NDP -1.5, LIB -1.5, BQ +0.5
Rest of Quebec South: CON +4, NDP -1, LIB -2, GRN -0.5, BQ +0.5

Corbett Communications' 8/11-13 poll
Update Sept. 8: Abacus' 8/23-29 ON breakdown
Nothing out of the ordinary here - no resulting sub-provincial adjustment. There is no huge deviation from expectations based on provincial averages. However, now that we have two pollsters providing a breakdown, I am comfortable making adjustments based on moderate deviations where there is some agreement, namely that the NDP does better than expected in the Toronto suburbs (but, per Corbett's finer breakdown, not in Hamilton-Niagara) and Eastern ON outside Ottawa, but worse than expected in the North and Southwest:
Toronto CMA outside 416: CON -0.5, NDP +2, LIB -0.5, GRN -1
Southwest: CON +1.5, NDP -2, GRN +0.5
North: NDP -1.5, GRN +1.5
East outside Ottawa: CON -3, NDP +3, LIB -0.5, GRN +0.5
I have also made a minor change to the adjustment for the 7 central Toronto ridings to CON +2, NDP -5, LIB +2.5, GRN +0.5 (previously +2/-5/+3/0).
All these adjustments are fairly cautious - very roughly half the size suggested by the polls - to account for the possibility that some of the apparent effect might be due to random variation. Because they are broadly consistent with the Corbett poll, for the purposes of retroactive computations to produce trends, these adjustments will be deemed to have been made in response to that poll.

Campaign Research's 8/9-13 poll
EKOS' 8/20-22 poll
Update Sept. 1: EKOS' 8/27-29 poll
Both All of these suggest that the NDP is falling a lot more in SK than in MB, and that, conversely, the CONs and GRNs* are rising much more in SK than in MB. However, due to the very small sample sizes (there were part of national polls) and the ongoing provincial election in MB, I am not making any adjustments for now.
*GRNs was a typo - the numbers didn't show that, even before the update.

Justason Market Intelligence's 9/3 poll
This poll is broadly in line with the current projection for the City of Vancouver (unlike the April poll by Justason, which had some very strange numbers and a very small sample). The Tories are a bit lower than expected, and the GRNs a bit higher. Justason overestimated the BC Greens in the last provincial election, so I'm going to hold off on any adjustments.

II. Riding Polls (by Mainstreet unless otherwise noted)

Note: A riding poll will result in a model adjustment ONLY IF it shows a substantial deviation from regional swing and/or it is in a riding with unusual factors. Most riding polls will therefore be ignored: they generally did not help much in 2015.

Victoria, 8/9-10
This poll did not come as a huge surprise to the model because it takes into account that, in 2015, the Green vote was inflated and the Liberal vote was depressed due to the Liberal candidate dropping out (but too late to be removed from the ballot). In fact, there are several ways to make the adjustment, and I picked one of the relatively cautious ones; the poll suggests that my caution was unwarranted. Therefore, rather than making this a poll-based adjustment, I amended the baseline adjustment (as previously noted in the model post) by CON +1.3 (thus cancelling the CON adjustment), NDP -2.8 (to -6.5), LIB +6.5 (to +22.8), GRN -5 (to -16.3). This shifts all numbers toward (but not quite all the way to) the poll.

Windsor West, 8/21
Because this poll essentially guides the Sandra Pupatello adjustment rather than provides information about a differential evolution of the riding per se, it leads to a baseline adjustment. Details were provided in the model post.

Vancouver Granville, Mainstreet 8/27-28 and Justason 8/28-30
The Mainstreet poll isn't very different from the May Vancouver Granville poll by Mainstreet, if you compare the party figures from the May poll (which are the ones I used) to the candidate figures in this one (which are more relevant now). Similarly, Justason continues to see a moderate lead for Wilson-Raybould, like in its April poll. The adjustment is amended from CON -10, NDP -8, LIB -10, GRN -2 to CON -11, NDP -8, LIB -9 -9.5, GRN -2 -1.5 (update due to contemporaneous Léger and EKOS polls subsequently published) CON -13, NDP -8, LIB -10.5, GRN -1.

Beloeil--Chambly, 9/3
This is almost exactly as the model expects!

Trois-Rivières, 9/3
LIBs and BQ a bit higher than the model expected, and other parties a bit lower, but not enough to warrant an adjustment.

Sherbrooke, 9/3
LIB much higher than expected. A cautious adjustment for now: CON -4, NDP -2, LIB +6.

Louis-Hébert, 9/3
Almost exactly as the model expects for LIB, GRN, NDP. CON a bit lower and BQ a bit higher than expected, but not enough to warrant an adjustment.

Thursday, August 29, 2019

Projection Update: Angus Reid's 8/21-26 Poll and EKOS' 8/25-28 ON Poll

The following polls have been added to the model:
Angus Reid's 8/21-26 national poll (current weight among national polls: 24%)
Update: EKOS' 8/25-28 ON poll (current weight among ON polls: 4%)
For a full list of included polls, see previous Projection posts.

The Angus Reid poll is good news for the Tories in the Atlantic and ON, where they're marginally ahead. And it's excellent for the Liberals in QC.

Update: The EKOS poll is similar to the ON portion of EKOS' recent national release. In the past, I have been reluctant to include regional portions of a national poll released without national numbers by EKOS. This time is a bit different though: we do have a recent national poll by EKOS, which means that the weight on these numbers would be low (as you can see above - and it's effectively even lower if you consider that the weight on the previous EKOS numbers goes down). So this is effectively somewhat like a national poll with an ON oversample, similar to polls by Campaign Research.

Projection as of the latest national poll* (midpoint: August 23.5)
LIB - 155.0 156.5 (33.9 34.1%)
CON - 146.1 145.0 (36.3 36.2%)
NDP - 20.2 19.7 (12.9 12.7%)
BQ - 12.1 12.0 (4.3%)
GRN - 3.7 (9.4%)
IND - 0.5

PPC - 0.5
If you're new to my 2019 projections, view key interpretive information here.
*For now, the EKOS ON poll is treated as if it also has a midpoint date of August 23.5.
(Very minor changes outside ON because I fixed a data entry error for auxiliary information about the Angus Reid poll.)

With the Angus Reid poll, despite the good national headline figures for the Tories, it's the Liberals that gained about an expected seat from the Bloc. Why?
- The weighting formula now puts some negative weight on the July Angus Reid poll to reduce the pollster house effect. That poll (like many previous Angus Reid polls) was terrible for the Liberals.
- This poll undersampled ON, so the CONs' good ON number didn't help them as much as one might expect. By contrast, the QC sample, which was in proportion with the national sample size, really helped the LIBs.
- Additional Tory vote gains in SK/MB aren't very useful in terms of seats.

Update: With the EKOS poll, unsurprisingly, the LIBs gain at the expense of the CONs and NDP in ON.

Seats whose projected lead changes relative to the last projection:
- CONs gain Charlottetown from LIBs.
- LIBs gain Rimouski-Neigette--Témiscouata--Les Basques from the NDP.
- LIBs take Salaberry--Suroît, Laurier--Sainte-Marie and Laurentides--Labelle from the BQ.
- CONs regain Cambridge and Vaughan--Woodbridge from the LIBs. (This no longer happens after the EKOS poll.)

Phone/IVR Polls vs. Online Polls

There has been chatter among some pollsters on Twitter (Frank Graves of EKOS and Quito Maggi of Mainstreet) that the NDP is in huge trouble - perhaps even in danger of being reduced to near zero. Lean Tossup did a blog post about this.

And yet, things don't necessarily look that critical in seat models: I have the NDP at 20.3 (18 projected ahead), 338Canada at 18.3 (16 ahead), the CBC Poll Tracker at 14 projected ahead, and Lean Tossup at 22 projected ahead.

Both EKOS and Mainstreet are IVR pollsters. So this got me thinking: what would my projection look like with only phone/IVR pollsters (who do probability sampling) vs. online pollsters (who can reach people that wouldn't pick up the phone, but have non-random samples)? (I tweeted a few days ago about pollsters' measurement of the crucial LIB-CON gap in ON.)

First, here are the full current toplines (little LIB tick from yesterday due to coming across decimal point data on a poll). Keep in mind that all figures reflect my turnout adjustment, which favours the Tories.

Projection as of the latest national poll (midpoint: August 21)
LIB - 153.8 (33.8%)
CON - 146.0 (36.3%)
NDP - 20.3 (12.9%), 18 ahead
BQ - 13.3 (4.3%)
GRN - 3.7 (9.5%)
IND - 0.5

PPC - 0.5
Effective sample size: 3,304 (single new poll unbiased relative to universe of pollsters)
If you're new to my 2019 projections, view key interpretive information here.

Explanation on effective sample size: my poll weighting method assumes that vote intentions evolve as random walks, and that each pollster is affected by a house effect (constant across all of that firm's polls) drawn from a normal distribution. The effective sample size is that of a poll with the same midpoint date as the latest poll (Aug. 21 here) with zero house effect that would have the same variance as the polling average.

(Note that in addition to the house effect, there is also potential error common to all pollsters. As that cannot be reduced through poll averaging, if it's taken into account, the effective sample size would drop drastically, to something around 370.)

Projection with Phone and IVR Polls only
LIB - 162.2 (34.7%)
CON - 144.3 (36.6%)
NDP - 13.4 (10.9%), 12 ahead
BQ - 12.7 (4.3%)
GRN - 4.5 (10.3%)
IND - 0.5

PPC - 0.5
Effective sample size: 1,240

Projection with Online Polls only
LIB - 150.1 (33.3%)
CON - 146.6 (36.1%)
NDP - 24.1 (13.7%), 21 ahead
BQ - 12.9 (4.3%)
GRN - 3.4 (9.4%)
IND - 0.5

PPC - 0.4
Effective sample size: 2,586

Keeping in mind that the margin of error is smaller around 11-14% than around 50%, and using the effective sample sizes, the difference between the NDP's 10.9% and 13.7% is statistically significant at the 5% level (and remains so if you cut the effective sample sizes by 1/3, since the surprisingly large NDP gap suggests I may have underestimated the variance of pollster house effects). It's hard to say if it's due to the phone/IVR vs. online issue or due to unrelated pollster house effects, but there appears to be a real difference between phone/IVR and online pollsters' measurement of the NDP.

- If you believe phone/IVR pollsters, the NDP is in grave danger of losing official party status: it would have been 50/50 for an election last week, and there are suggestions things have gotten worse.

- If you believe online pollsters, the NDP would have had a good chance of maintaining its seat count outside Vancouver Island and QC, and is very likely to have kept official party status.

The party most directly impacted by this is, unsurprisingly, the Liberals: they're either quite close to a majority (while losing the popular vote by 2%!) per phone/IVR, or basically even seat-wise with the Tories.

Regionally, this happens because, relative to the online projection, the phone/IVR projection has LIBs taking from CONs and the NDP in ON, and CONs taking from the NDP in BC. The effects pretty much cancel out for the Tories, who are in the mid-140s in both projections.

It will be interesting to see if this difference persists through the campaign. I will end by noting, perhaps ominously for the NDP, that in recent elections, when pollsters have disagreed about its prospects, the pollsters having it lower were almost always closer to the mark.

Wednesday, August 28, 2019

Projection Update: Narrative's 7/31-8/22 Atlantic Poll

The following polls have been added to the model:
Narrative's 7/31-8/22 Atlantic poll (current weight among national Atlantic polls: 29%)
For a full list of included polls, see previous Projection posts.

This poll is very good for the Liberals across the Atlantic, except in PEI. As a result of its inclusion, the Liberals have a slightly less tenuous grip on most of the several Atlantic races where they're projected marginally ahead.

Update Aug. 28: I have refined the within-Atlantic provincial adjustments. They're now better for the LIBs/worse for the CONs in NB, and slightly the reverse everywhere else (especially PEI). Since the tightest races in the projection are outside NB, this hurts the LIBs and helps the CONs. Changes below crossed-out and underlined.

Projection as of the latest national poll (midpoint: August 21)
LIB - 154.3 153.7 (33.7%)
CON - 145.4 146.0 (36.3%)
NDP - 20.4 20.3 (12.9%)
BQ - 13.3 (4.3%)
GRN - 3.7 (9.5%)
IND - 0.5

PPC - 0.5
If you're new to my 2019 projections, view key interpretive information here.

I have also made an interim (see explanation below) update to the provincial adjustments in the Atlantic. Here are the new ones:
NL: CON +9 +9.5, NDP -0.5 -1.5, LIB -1.5 -2.5, GRN -7 -5.5
NS: CON -3, NDP +3, LIB -1.5, GRN +1.5
NB: CON -2.5 -3.5, NDP -3 -2, LIB +3 +4.5, GRN +2.5 +1
PE: CON +1 +4, NDP -1.5 -3, LIB -3.5 -5, GRN +4
All Most changes are fairly minor, as the previous adjustments held up well. The LIBs got some good news in NB, while the CONs got good news in PEI, which flips Egmont in the projection.

Please note that the other Atlantic polling firm, MQO Research, has been putting out its summer polling as well. Unlike Narrative, MQO releases its results province by province. MQO's numbers will be included in the polling average when all 4 provincial results have been released - we got NB today, and only NS is missing. However, I did consider the results already released for the interim adjustments above, and will finalize them when MQO's NS numbers are available.

Finally, if you're sharp-eyed, you may notice from the "seats ahead" table that one lead has changed... not in the Atlantic, but in ON! That puzzled me, and I thought I screwed something up. It turns out that the weight changes on the Atlantic components of national polls caused some shifts of under 0.01% due to the requirement that averages based on regional breakdowns of national polls add up to national averages. This flipped Cambridge back to Liberals, but obviously it's not a meaningful change.

Tuesday, August 27, 2019

A Critique of 2019 Projection Models

Update Sept. 16: Too Close to Call is now active, Lean Tossup adjusted its calibration for uncertainty, and Calculated Politics has provided me with some methodological information. (Further update: 338Canada has added more details on its methodology page.) For readability, instead of crossing out the old text as I usually do, I have simply deleted it. New text below is in italics.

Beyond providing my own seat projections, a purpose of this blog is to help readers become better informed consumers of seat projection models, and to propose improvements for other modelers. For example, in 2015, this post regarding uncertainty pointed out that confidence intervals given by some websites, including fairly well-known ones, were way too narrow. It was widely shared (Andrew Coyne retweeted it), and at least one other modeler (Bryan, who runs Too Close to Call) responded to it favourably. The results of the election vindicated the post - and then some: if anything, the intervals given in the post itself were still a tad narrow.

This post discusses the strengths and weaknesses of models currently active (as well as Too Close to Call, which I believe will be active soon). You can find links to all of these sites in the blog's right-hand column.

Overall Assessment
If you need *one* go-to website, I strongly recommend 338Canada due to its treasure trove of information, accurate depiction of uncertainty, and ease of navigation. The methodological description is now also much more detailed - other than remaining vagueness on how demographic factors are included (which is understandable from an intellectual property perspective), the rest of the model is clearly described.

I would suggest this site, Canadian Election Watch, as your second source. As explained below, I believe my projections are now based on the most solid polling averages. I also provide a slightly different perspective by making a (prudent) turnout adjustment based on past elections. My methodology is described in detail, so if you see a result you don't like, you can figure out how it came about. Finally, I occasionally provide commentary, such as this post, that goes beyond describing the numbers and their implications, and helps you better understand how they're generated.

The CBC Poll Tracker and Too Close to Call are tried and tested websites. Worth a visit, as always. LISPOP also has a long track record, though it is more basic than the above websites.

Lean Tossup has a vague methodological description, though I have received helpful additional information about the model from Evan Scrimshaw, the managing editor for the site. They do things somewhat differently (as detailed below), which means that their results often look like outliers, but also that they provide a useful different perspective - so go visit their site! They have fixed their calibration of aggregate uncertainty, which I complained about in an earlier version of this post, so I have removed the warning about their probability estimates. (However, the probabilities given for individual ridings still imply too much confidence, in my opinion.) The articles on the site are also worth a read: sometimes brilliant, sometimes unreasonable, but never dull.

Calculated Politics is a very sleek website, but has no published methodology for now. This review is based on the information that I received via Twitter - pretty basic (so I can't really judge the model's quality), but enough so say that the projections are done seriously. Unfortunately, the 90% confidence intervals given for national seat totals and vote shares are too narrow, so those should be ignored for now (or considered, say, 65% confidence intervals). A fix may be forthcoming, however. I will move them from "Other" to "Main" once the methodology is provided on their webpage.

As far as I'm concerned, other currently active seat projectors have not earned any credibility because they do not describe their methodology at all (as explained in this post, this is why they aren't listed under "Main Projection Sites"), do not model uncertainty*, and have little track record. There are links to them on the right-hand side of the page under "Other Projection Blogs/Sites" for convenience, but rely on them at your own risk.

*Except for Visualized Politics, whose given probabilities are currently unreasonable given where their projection is - if you visit their site, stay away from the probabilities.

Transparence of Methodology
Winners: CBC Poll Tracker, 338Canada, Canadian Election Watch
Honourable Mention: Too Close to Call

The CBC Poll Tracker, 338Canada and this blog are the only sites that describe in some detail how polls are averaged. These sites and Too Close to Call also provide detail about how polls are converted seat projections. Additionally, I fully detail every riding-level adjustment made, although I recognize that the information given by the other sites mentioned above (list of factors taken into account, without listing affected ridings) is adequate.

LISPOP lists the polls considered, but does not explain how they are combined (or show any vote share estimates). It also states that it uses regional uniform swing without incumbency effects. However, something more must be going on for the Greens to be projected to get a 6th seat in BC...

In my view, Lean Tossup has the most room to improve regarding transparence (other than Calculated Politics, of course, who is working on a methodology page). They describe which variables are considered by their model, which is good. However, the poll weighting method is a bit of a black box (though the information provided below by Evan Scrimshaw should shed some light).

Presentation (and Transparence) of Results
Winner: 338Canada
Honourable Mention: Calculated Politics, Too Close to Call, CBC Poll Tracker

There really isn't much to be said here other than the fact that P.J. Fournier's site is a gem: information about model results is very detailed and easy to find, and the page is visually appealing.

Calculated Politics' website is also excellent. In this case, the extensive presentation of results makes up somewhat for the opaque methodology: for example, you might now know how the polling weights are derived, but you can see what they are.

Too Close to Call's seat calculator is a very nice feature that gives readers the ability to make their own hypothetical scenarios. For this, TCTC deserves an honourable mention.

The CBC Poll Tracker's website has a link to historical polling trends, which is interesting. It does not provide riding by riding information, which is understandable given that such information is often misused, which the CBC has some responsibility to avoid putting out.

I improved the transparence of results on this blog a couple of weeks ago by adding a table of regional polling averages at the upper left. It's a simple feature that I wish more sites had - Too Close to Call does, but elsewhere, you have to click through to each region's page (or, for the CBC Poll Tracker, toggle to each region's chart) to access that information. (338Canada and LISPOP have a regional table for seats, but not polling averages.)

Poll Averaging
Winner: Canadian Election Watch

This is an area in which I've put a lot of effort for this year, and I'm pretty confident that my poll averages are now the most reliable. (Note: I'm not talking about the turnout adjustment, where any defensible guess is basically just as good.) By reliable, I mean that it does not overly react to any single poll or pollster, and yet remains sufficiently responsive. I have separate sets of poll weights for each region, reflecting the fact that one needs to weight older polls more heavily in smaller regions to get a decent sample size. The weights I use depend not only on a poll's own characteristics (recency, sample size), but also on the characteristics of other polls. All this is derived from a statistical framework to extract information optimally (in the sense of minimizing the variance of the estimate's error). You can read the details here, here and here.

338Canada's poll averaging method uses the square root of the sample size (like my old method), which ensures that no excessive weight is put on a single poll (as opposed to a single pollster in my current method); time discounting uses a power function. The CBC Poll Tracker discounts polls using an exponential function, and applies a penalty to outliers. Too Close to Call uses a rudimentary time discounting scheme, and applies a penalty to a pollster's older polls. All these schemes are sensible (except for the outlier penalty by the CBC Poll Tracker, which I believe to be statistically unsound), but they are not optimized based on a statistical model like my scheme. It is therefore virtually impossible to answer the question "Under what assumptions on the evolution of voting intentions and the generation of polling errors is this weighting scheme optimal?"

Too Close to Call will also enhance its averages with riding polls (see below).

Lean Tossup doesn't describe its method, and its polling average often deviates noticeably from other websites'. From information provided by Evan Scrimshaw, this appears to be due to two factors: (i) a more restrictive poll inclusion policy (no IRG, DART or, if I remember correctly, Ipsos polls), and (ii) extrapolation from riding polls. The most obvious deviations so far appear due to the former, though I do feel that the weighting of riding polls is a bit too high - at least currently, given the small number of riding polls. As the campaign progresses and riding polls accumulate, however, extrapolation from them could become a significant asset - Too Close to Call will start extrapolations at some point - so keep an eye on them!

338Canada, the CBC Poll Tracker and Calculated Politics also use pollster ratings. Without knowing how they are derived, it is difficult to judge how much this helps (or even whether it is counterproductive).

Model Skeleton
Winners: Canadian Election Watch, Too Close to Call

What I mean by "model skeleton" is how the seat model converts polling averages to vote shares in each riding, before adjusting for riding specific factors like incumbency, star candidates, demographics, etc.

There are two main basic approaches to this: uniform swing, where all ridings change by the same amount (e.g. if a party goes from 20% to 30% within a region, it increases by 10 points in each riding), and proportional swing, where all ridings change by the same proportion (e.g. if a party goes from 20% to 30% within a region, its vote share increases by half in each riding). With proportional swing, an adjustment is required to bring each riding's total back to 100%. Neither approach is clearly superior. LISPOP, Too Close to Call and I use uniform swing; 338Canada, the CBC Poll Tracker and Calculated Politics use proportional swing; Lean Tossup does not specify its method. (Note: all models apply swings at the regional level; using only the national swing just does not make sense for Canada due to large regional disparities.)

I listed Too Close to Call and myself as "winners" because we make enhancements to uniform swing. Too Close to Call introduces sub-regional coefficients in recognition of the fact that some areas within a region have been historically more volatile than others. I adjust uniform swing in recognition of the fact that swings tend to be smaller for parties with very low or very high vote shares (as described here).

The CBC Poll Tracker tries to enhance proportional swing by using each of the last three elections as baseline, and averaging the results. While this helps avoid cases where a skewed baseline overly impacts a result, it may increase the error in areas experiencing meaningful demographic change. Indeed, if the projections based on 2008, 2011 and 2015 baselines move in a direction due to continuing demographic change, using the average of these is worse than just using the 2015 baseline.

338Canada gives "probabilistic floors and ceilings" for regions and districts based on electoral history. This sounds sensible, but it's hard to judge the pertinence of this without more details.

Use of Demographics
Winners: 338Canada
Honourable Mention: Lean Tossup

Two models use demographic characteristics: 338Canada and Lean Tossup. As stated above, neither explains in detail how this is done, but at least 338Canada gives an inkling, and can point to specific examples where this has helped in the past (e.g. QS winning Sherbrooke in the 2018 QC election). Lean Tossup does say that the indicator it uses is education.

Riding-Specific Adjustments
Winners: CBC Poll Tracker
Honourable Mention: Calculated Politics, 338Canada

The CBC Poll Tracker seems to account for the largest set of potential riding-specific factors (e.g. incumbency, star candidates, ministers, etc.). 338 Canada and Calculated Politics also seem to make many adjustments; Calculated Politics additionally takes social media sentiment into account. I also make some riding-specific adjustments, but not to the same extent (about 10% of ridings); Too Close to Call appears to be in the same boat. LISPOP does not seem to make these adjustments (except when a minor party/independent is expected to have a strong showing in a district, as is evident from its current projection with seats for Bernier and Wilson-Raybould), while it is unclear to what extent Lean Tossup factors these in.

Treatment of Uncertainty
Winner: 338Canada

To me, this is *the* major strength of 338Canada: it takes uncertainty very seriously, comes out with reasonably calibrated results, and presents them in a helpful way.

The CBC Poll Tracker now also has reasonable confidence ranges and probability estimates, though it shows less information than 338Canada.

As mentioned at the start of the post, Lean Tossup has recalibrated its aggregate uncertainty, and the resulting probabilities are now much more reasonable than what they had before. (The current high probability of a Liberal majority is commensurate with the high Liberal projection.) However, I do not believe that enough uncertainty is taken into account for the individual seats' win probabilities.

I provide confidence ranges from time to time (here or on Twitter), and distinguish between uncertainty for an election taking place as of the last poll or now (which, as far as I know, is what all other websites provide) and uncertainty for the actual election (which is MUCH MUCH larger).

Calculated Politics provides confidence ranges as a regular feature, but as mentioned above, they seem miscalibrated and should be ignored or re-interpreted for now.

I look forward to what Too Close to Call puts out this cycle (the first projection did not come with ranges). LISPOP does not provide numerical evaluations of uncertainty.

Monday, August 26, 2019

Methodology Update: Weighting of Regional Numbers

The last projection update, which showed a whopping 7 seats change hands in Atlantic Canada, got me thinking: does it make sense to have this many changes based on just 63 poll respondents? The polling average dropped from a ~10% LIB lead in Atlantic Canada to under 3%.

It quickly became apparent that the problem is that I was using the national weights for regional breakdowns. National weights are derived using a statistical model that trades off the need for sample size, recency and pollster diversity. For regional breakdowns, sample sizes are much smaller, which means that ideal weighting would need to work harder for sample size, and therefore sacrifice recency. Thus, older polls should be weighted more heavily for regional breakdowns. As a result, going forward, regional breakdowns should be less volatile - though the national volatility will remain the same.

(Small) Update Aug. 26: Separate set of weights for the BQ. Also I'm now adjusting for imbalanced national polls (polls that oversample in some regions and undersample in others): their effective sample size for the national numbers is smaller.

Update Sept. 25: Standalone regional polls are now weighted differently from the regional breakdowns from national polls. Details underlined at the end of this post.

I'll get to the details later in this post, but for now, here is the projection with the new methodology:

Projection as of the latest national poll (midpoint: August 21)
LIB - 153.6 153.2 (33.7%)
CON - 146.3 (36.4 36.3%)
NDP - 20.4 20.5 (12.9%)
BQ - 12.9 13.3 (4.3%)
GRN - 3.7 (9.5%)
IND - 0.5
PPC - 0.5
If you're new to my 2019 projections, view key interpretive information here.

As you can see, the totals are little changed. The Bloc loses 1.5-2 1-1.5 seats, while the LIBs and CONs each gain 0.5-1 around 0.5 seat. However, many switches in the last projection no longer happen:
- All 7 seats that switched in the Atlantic stay LIB.
- 8 9 of the 10 seats that switched in ON stay CON; only Vaughan--Woodbridge and Cambridge still switches to the LIBs.
- In MB, Winnipeg South stays LIB.

- LIBs regain Laurentides--Labelle from the BQ.
- NDP regains Edmonton Strathcona from the CONs.
- GRNs regain Victoria from the LIBs.

Now the details of the methodology change. How to deal with regional numbers is tricky: it depends on the correlation between how different regions move. Consider these extreme cases:

1. Regions move in lockstep: their difference with the national vote is fixed. In this case, the national average should be calculated with national weights. Regional averages should be the sum of the national average and each region's difference from the country. The latter should be estimated without discounting polls based on age, as the nation-region gap does not evolve.

2. Regions move completely independently. In this case, regional averages should be calculated with region-specific weights, and the national average should be derived as a weighted average of regional averages. These region-specific weights would reflect two opposite considerations:
(i) smaller regional samples mean that one should weight older polls more (as explained earlier), and
(ii) in order for independently evolving regions to generate a certain level of national volatility, regional volatility would have to be much larger, which means that one should weight older polls less.
Consideration (i) wins out for smaller regions, and consideration (ii) wins out for larger ones. On average, the amount of discounting should be similar to what national weights would imply.

Reality is, of course, somewhere between these extreme cases. In both situations, the amount of discounting in optimal national weights remains suitable. Therefore, I do not make any change to how I derive national weights. Additionally, higher weight should be given to older polls for small regions (and also for large regions unless reality is very close to case #2).

Given this, I derive separate weights for each region using a procedure that is a blend of what would be called for in cases #1 and #2. The idea is for the regional weights to help estimate the difference between each party's regional and national support, rather than the regional support directly.

1. Regional weights are derived using regional sample sizes and a formula similar to that used for national weights, with the following modifications:
- The sample variance takes into account the fact that we're now thinking about the difference between regional and national vote (except, of course, for standalone regional polls).
- The time variability parameter is set assuming that the difference between the region and the rest of the country is half as volatile (so ~70.7% of the standard deviation) as the national support level, except in QC, where the two variances are the same. This is consistent with the observation that regional deviations from national movements tend to be smaller than the national movements themselves, except in QC, where they appear similar. An adjustment is made for the fact that the national average incorporates the regional movement (this matters mainly for ON).
- Update: A separate set of weights is computed for the BQ, with a time variability parameter that includes both the national and provincial variability that other parties have.

2. These weights are first applied to the difference between regional and national numbers in national polls with regional breakdowns. The weighted averages are added to the national averages (which also include national polls without regional breakdowns) to produce preliminary regional polling averages. These preliminary averages are adjusted upward or downward to match the national polling averages. This last step is necessary because regional weights no longer match national weights, and intuitively, it makes sense because recent movement in the rest of the country carries some information on recent movement in a given region, as they're correlated. [Of course, this paragraph does not apply to the BQ.]

3. Finally, standalone regional polls are incorporated using the regional weights to obtain the final (pre-turnout adjustment) regional polling averages. An adjustment is made to account for the fact that standalone polls need to be discounted more quickly, since, in addition of being affected by the region shifting relative to the country, they are also affected by national shifts. This adjustment takes the form of using a smaller sample size than the actual one for standalone polls, with the difference growing as time passes. The national polling averages are then obtained as a population-weighted averages of the regional polling averages.

Sunday, August 25, 2019

Projection Update: EKOS 8/20-22 and Mainstreet's Windsor West Poll

The following polls have been added to the model:
EKOS' 8/20-22 national poll (current weight among national polls: 19%)
Mainstreet's 8/21 Windsor West poll
For a full list of included polls, see previous Projection posts.

The EKOS poll was conducted over 3 nights. Although 338 indicates that they were 8/21-23, Frank Graves' (EKOS president and founder) tweets (ON and QC numbers) on Aug. 23 states that the numbers were "as of last night." I could go either way, and I'll treat the poll as a 8/20-22 poll - it doesn't make a big difference.

EKOS shows two main things that roughly cancel out: great Liberal numbers in ON, and great Conservative numbers in the Atlantic. Of course, the Atlantic sample size is tiny, so the margin of error (MoE) is huge (roughly +/- 12%). However, the Conservative number is so good that even if you take the low end of the 95% confidence interval for the Tories and the high end for the Liberals, the poll would still be mildly positive for the Tories (they'd be only slightly behind the Liberals)! Other things to note include poor NDP numbers and good BQ numbers; both of these are consistent with EKOS numbers all summer, which suggests that they may be EKOS house effects. On the other hand, this poll has the worst Green numbers we've seen from EKOS this summer. (The Greens have been sliding in my projection gently, but continually for four weeks...)

The Mainstreet Windsor West poll guides a "star" adjustment, as described here, for Sandra Pupatello, the Liberal candidate. This obviously helps the LIBs and hurts the NDP in Windsor West. However, riding-level adjustments have a countervailing effect elsewhere in the same polling region so that the riding-level projections remain consistent with the polling average. As a result, the result of this poll on total seat numbers is smaller than the effect in Windsor West alone (and in fact marginally positive for the Tories even though only the NDP, LIB and GRN vote shares are adjusted).

Projection as of the latest national poll (midpoint: August 21)
LIB - 153.0 (33.7%)
CON - 145.6 (36.3%)
NDP - 20.3 (12.8%)
BQ - 14.6 (4.4%)
GRN - 3.6 (9.5%)
IND - 0.5

PPC - 0.5
If you're new to my 2019 projections, view key interpretive information here.

The projection is essentially unchanged. The NDP is down a bit, but this just brings it back to where it has been all August before this past week (or would have been, had I introduced the NDP turnout adjustment at the outset): this past week's polls tended to be from pollsters whose results are usually favourable to the NDP. This EKOS poll balances things out.

There were, however, a ton of changes in which seats are projected ahead, due to the poll's unusual Atlantic and ON results:
- In the Atlantic, CONs take 7 more seats from LIBs: Sackville--Preston--Chezzetcook, Central Nova, Avalon, Egmont, South Shore--St. Margarets, Charlottetown, West Nova.
- In Ontario, LIBs regain 10 seats from CONs: Vaughan--Woodbridge, Cambridge, Richmond Hill, Burlington, Oakville North--Burlington, King--Vaughan, York Centre, Whitby, Northumberland--Peterborough South, Newmarket--Aurora.
- In Manitoba, CONs take 1 more seat from LIBs: Winnipeg South.
- In BC, CONs retake 1 seat from the NDP: North Island--Powell River.

It'll be interesting to see if EKOS has identified new trends of Tory strength in the Atlantic and/or additional Liberal strength in ON, or if this is just random variation.

Addendum: You'll note that the "seats ahead" breakdown has the Liberals winning 78 ridings in ON, just two fewer than in 2015. However, their "average" number of wins in ON is 73: they're marginally ahead in disproportionately many tight races. The LIB lead in ON is 7.5% in my adjusted polling average, which is 2.2% short of the 2015 level.

Wednesday, August 21, 2019

Projection Update: DART 8/16-17, Abacus 8/14-19, Léger 8/16-19

The following polls have been added to the model:
DART's 8/16-17 national poll (current weight among national polls: 15%)
Abacus' 8/14-19 national poll (current weight among national polls: 24%)
Léger's 8/16-19 national poll (current weight among national polls: 18%)
For a full list of included polls, see previous Projection posts.

Please note that this projection update reflects an additional turnout adjustment against the NDP. Thus, while the NDP is down relative to the previous projection, it is actually up in seats (though almost perfectly flat in votes) as a result of the inclusion of these polls.
Update Aug. 21: I have also updated some riding-level adjustments for star NDP ex-MPs that lost last time and won't be running again (Halifax, Sackville--Preston--Chezzetcook, Ottawa Centre), using a similar approach as in Skeena--Bulkley Valley. These adjustment are very cautious (i.e. smallest plausible rather than "best guess"), and shift just 0.3 seats from the NDP to the Liberals.

These three polls carry fairly contradictory information in all regions. Depending on the poll:
- CONs now tie LIBs in the Atlantic, or LIBs have recovered a sizable lead there.
- CONs are at their lowest or highest point of the summer in QC.
- LIBs continue to enjoy a significant lead in ON, or there is a virtual tie there.
- The SK/MB race is now a CON blowout, or the race has tightened a lot.
- CONs at what might be an all-time peak in AB, or they're below their 2015 levels.
- Either the CONs or the LIBs have opened up a lead in BC.

Things pretty much balance out when you average them. The Liberals and Greens are down slightly, while other parties are up a bit. But really, considering that more than half of the polling average is new, there is remarkably little movement.

Projection as of the latest national poll (midpoint: August 17.5)
LIB - 152.9 153.2 (33.5%)
CON - 144.3 (36.1%)
NDP - 23.0 22.7 (13.2%)
BQ - 13.2 (4.3%)
GRN - 3.6 (9.6%)
IND - 0.5

PPC - 0.5
If you're new to my 2019 projections, view key interpretive information here.

Due to the change in turnout adjustments, instead of listing which seats flipped, I'll simply provide new maps in the coming days (or later today if I have time). Stay tuned.

That said, you may notice from the provincial breakdown that the NDP gains a seat in NS: it's Sackville--Preston--Chezzetcook. As I mentioned in the early August model post, there's no adjustment in that riding for now due to the lack of a reasonable baseline (Stoffer had represented the riding since its creation in 1997), but much like I recently did for Skeena--Bulkley Valley, I might decide to figure something out here. I did add an adjustment, which flipped the seat back to the Liberals, marginally. But since the adjustment is very cautious, I wouldn't be surprised if the NDP is actually not even competitive in Sackville--Preston--Chezzetcook.

Update: Images below.

Model Update: NDP Turnout Adjustment

I will update the projection for the new Léger poll soon, but first, just a quick note to let you know that I've decided to adjust the NDP vote share downward by 1%, with increases to the LIB and CON shares by 0.5%. The reason is that there has been high pollster disagreement on the NDP vote share among pollsters all summer: some consistently have them around 10-12%, while others consistently have them around 16-18%. In recent provincial elections, the NDP underperformed the polls in ON, AB and NS. They hit the polls in BC and SK, but in both cases, the pollsters were in agreement. They slightly outperformed the polls in MB (2016), but that was likely due to the Greens only running in about half of the ridings, meaning that some Green voters had to vote for someone else. Therefore, if things stay this way, it appears quite likely that the NDP would underperform relative to the polls again.

A knock-on effect of this is that I've changed the Burnaby South adjustment by adding 1 point to Jagmeet Singh's bonus (now +6%): if polls have been overestimating the NDP, then his by-election result was even better relative to contemporaneous provincial polling. I've also reviewed where the bonus is coming from: I crudely had it all come from the Liberals, but given what looks like a tight race, I've decided to take a closer look. Now 3.5 points are from the Liberals, 1.5 from the Tories and 1 from the Greens. (This is very uncertain, though, as the by-election had no Green candidate and a star PPC candidate.)

Tuesday, August 20, 2019

Projection Update: Corbett 8/11-13 and Global News/Ipsos 8/16-19

The following polls have been added to the model:
Global News/Ipsos' 8/16-19 national poll (current weight among national polls: 24 28%; only publicly available national numbers used)
Corbett Communications' 8/11-13 ON poll
For a full list of included polls, see previous Projection posts.

Update Aug. 20: Poll weights updated after information about the number of respondents was obtained. This resulted in small changes (under 0.5 seats and 0.1% per party, as shown below).

Once again, EKOS put out some ON and QC numbers that are much more favourable for the Liberals, relative to the polling average, than the national numbers, which weren't fully provided. Therefore, once again, to avoid biasing the projection by including only the parts of a poll favourable to one party, I am not adding the EKOS numbers to the model (especially since it was also noted that EKOS' unpublished ON numbers from other nights aren't quite this strong for the Liberals).

Please note that this projection reflects an adjustment in Longueuil--Saint-Hubert to account for Pierre Nantel switching allegiance. (The adjustment actually makes no visible difference due to that riding still projected as a LIB/BQ race.)

Projection as of the latest national poll (midpoint: August 17.5)

LIB - 155.2 155.3 (33.4%)
CON - 140.1 139.7 (35.1%)
NDP - 24.9 25.3 (14.1 14.2%)
BQ - 13.0 12.9 (4.3 4.2%)
GRN - 3.8 (9.9 9.8%)
IND - 0.6

PPC - 0.5
If you're new to my 2019 projections, view key interpretive information here.

The Ipsos poll helped the NDP, but had little effect on the projection for the two main parties: it is slightly less favourable to the Liberals than other polls, but including it also reduced (in fact, made negative) the weight on the July Ipsos poll which was much worse for the Liberals. The Global News article showed some very good Liberal numbers in QC and poor ones in ON; however, those were not included because the other parties' numbers were not provided. (The non-inclusion of Ipsos and EKOS ON numbers roughly balances out.) I also had to back out the number of respondents from the margin of error. If this were during the campaign, it would actually be illegal for Global to report the poll without stating the number of respondents... (Update: The number of respondents is available in this tweet.)

The projection improved for the Liberals and worsened for the Conservatives mainly due to the Corbett poll, which shows a healthy Liberal lead in ON.

In terms of seats projected ahead, relative to the last projection:
- LIBs retake Québec from CONs.
- Update: LIBs retake Laurentides--Labelle from BQ.
- NDP retakes Berthier--Maskinongé from BQ. (Note that, as a policy, I do not include leaked internal polling.)
- LIBs retake St. Catharines, Vaughan--Woodbridge, Cambridge from CONs.
- LIBs retake Hamilton East--Stoney Creek from NDP.
- NDP retakes North Island--Powell River (update: and Burnaby South) from CONs.

Sunday, August 18, 2019

Minor Update to 2015 Baseline

I've taken another look at the situation in Skeena--Bulkley Valley, where Nathan Cullen is retiring. There's no new information, but I've decided that holding off on any adjustment was too cautious. You can view the update here.

The NDP is still favoured in this riding, but the Tories are now within striking range (behind by 6-7%). Given that the adjustment is pretty cautious, I wouldn't be surprised if things are even tighter.

This update marginally changes the projection by 0.1 for the Tories and NDP.

Friday, August 16, 2019

Another Small Methodology Improvement

I have refined the method by which I obtain expected numbers of seats, which no longer ignores the party projected third in a district (but is still approximate).

I was expecting this to help the Greens, but to my surprise, it actually helped the Conservatives. Indeed, the seats where the Greens are leading tend to be three-way races (which means that they were slightly overestimated), while in the seats where they are third, they are mostly such a distant third that this doesn't make much difference.

That said, the change for all parties is under 1 seat - as expected, the previous method was already reasonably accurate. The numbers are now:
LIB - 152.0
CON - 144.1
NDP - 23.4
BQ - 13.6
GRN - 3.8
IND - 0.6
PPC - 0.5

Thursday, August 15, 2019

Projection Update: Campaign Research 8/9-13

The following poll has been added to the model:
Campaign Research's 8/9-13 national poll (current weight among national polls: 41%)
The model puts a high weight on this poll due to the next most recent poll being a full week older. Campaign Research's July poll has a weight of -17%.
For a full list of included polls, see previous Projection posts.

This poll was good for the Liberals in Atlantic Canada, but, crucially, not so good for them in ON, where they lead, but not by much. It also breaks out MB and SK, showing a very large gap between the two (even larger than in 2015, to the point that the Conservatives are stronger in SK than AB), but the sample sizes there are tiny, so there are no provincial adjustments yet for the MB/SK region. Finally, this is a weak poll for both the Liberals and the Conservatives in QC, which leads to BQ gains in the projection.

Updates 8/15: Map added, and slight projection change (without any lead change) due to changes to baseline adjustments in Thunder Bay--Superior North and due to this poll:
Mainstreet's 8/9-10 Victoria poll
Update 8/16: Added a potential House of Commons seating chart.

Projection as of the latest national poll (midpoint: August 11)
LIB - 152.4 152.7 (33.2%)
CON - 143.6 (35.7%)
NDP - 23.4 23.2 (13.7%)
BQ - 13.4 (4.3%)
GRN - 4.2 3.9 (9.8%)
IND - 0.6

PPC - 0.5
If you're new to my 2019 projections, view key interpretive information here.

In terms of seats projected ahead, relative to the last update:
- Liberals lose one seat in QC to the NDP: Rimouski-Neigette--Témiscouata--Les Basques
- Liberals lose one seat in QC to the Tories: Québec*
- Liberals lose three seats in QC to the Bloc: Laurier--Sainte-Marie*, Salaberry--Suroît, Laurentides--Labelle
- Liberals lose three seats in ON to the Tories: Cambridge, Vaughan--Woodbridge, St. Catharines
- Tories lose one seat in AB to the Liberals: Calgary Skyview*
- The NDP loses two seats in BC to the Tories: Burnaby South, North Island--Powell River
*Previously mentioned caveats apply.

Tuesday, August 13, 2019

What Would It Take For...?

Now that I've hopefully finished making tweaks and refinements to the model, it's time to look at a number of "what would it take" scenarios. Below are some examples of national support levels that make the estimated expected seats (the numbers in the blog's right column) reach certain benchmarks.

Update Aug. 14: In light of today's news, I've added an alternative Conservative majority scenario primarily through a Liberal fall (rather than a Tory rise).

For each of these, I assume uniform national swing from the current projection: that is, changes in a party's support from current levels are the same in every region of the country. Two important caveats:
1. Regional support patterns can very well change during the campaign - we all remember Quebec swinging wildly the last two times - in which case these numbers can also change significantly.
2. The further down in the post you get, the larger the shifts involved, which means that those are even less reliable.
Therefore, please take everything with a grain of salt... and enjoy!

Tie for first place
CON 36.2% (148.3 seats)
LIB 32.4% (148.3)
NDP 14.0% (24.2)
BQ 4.1% (12.0)
GRN 10.0% (4.0)
The model suggests that if current patterns of regional support hold, the Tories need to win the popular vote by more than 3% just to get a tie.

Liberal majority
LIB 35.1% (170.0)
CON 34.8% (131.5)
NDP 13.1% (21.5)
BQ 4.1% (9.9)
GRN 9.5% (4.1)
The model suggests that the Liberals can get a majority with not much more than 35% support and a tiny popular vote win. In fact, as I mentioned on Twitter a few days ago, a majority with a popular vote loss is possible.

Conservative majority (mainly through Tory rise)
CON 39.1% (170.0)
LIB 31.4% (132.5)
NDP 13.1% (19.9)
BQ 3.9% (10.8)
GRN 9.2% (3.8)
The silver lining for the Conservatives is that a majority is not out of reach. Indeed, if they return to their 2011 level (39.6%), they would likely get a majority, despite their inefficient vote this time.

Conservative majority (mainly through Liberal fall)
CON 37.3% (170.0)
LIB 27.7% (112.4)
NDP 16.1% (32.8)
BQ 4.4% (17.4)
GRN 11.4% (4.2)
If the Liberals weaken significantly, the Tories' majority threshold goes down, and the NDP gets to maintain its total of 28 seats outside Quebec (though with losses in Western Canada and gains in Eastern Canada). The Bloc also benefits.

NDP loses official party status
CON 36.7% (143.4)
LIB 35.4% (166.8)
NDP 8.7% (11.0)
BQ 4.4% (11.0)
GRN 11.7% (4.7)
The model thinks the NDP loses official party status when its support drops below roughly 9%. Fortunately for the NDP, even their worst polls still give them around 10% support, while their best ones give them in the mid-to-high teens. The NDP cratering would likely help the Liberals: notice how close they are to a majority in this scenario while losing the popular vote by over 1%.

Greens gain official party status
CON 33.1% (145.9)
LIB 29.5% (148.6)
NDP 10.7% (19.2)
GRN 19.5% (12.0)
BQ 3.4% (11.0)
The model is not well-suited to deal with this scenario, so take this with a big scoop of salt! In particular, with strategic voting and smart targeting, the Greens could reach 12 seats with a lot less than 19.5%. Presumably, if Green support reaches the mid-teens, we'll get lots of riding polls in districts that look the most promising to them - and incorporating those polls into the model would correct things.

That said, it can definitely happen for a third party's vote to be much more inefficient when it's on the way up (like the Green vote) than when it's well established (like the NDP vote) because voting habits adjust. For example, in the 1983 UK General Election, the Lib Dems surged, and it took them 25.4% (!) national support to get just 23 of 650 seats (which happens to be almost exactly the same proportion as 12 of 338). So it's not crazy to think that it could take the Greens 20% to get to the same place. Then, as the voters got used to the Lib Dems and could see where they're strong, they got twice as many seats (46 of 659) in the 1997 UK General Election on 2/3 of the vote share (16.8%) - somewhat similar to the NDP situation now.

Estimates for Expected Number of Seats

Update Aug. 15: I have refined things (see underlined parts below), but this is still an approximate method.

As you can see, the projected numbers of seats, which are now provided in the right-hand column, now have one digit after the decimal point, and no longer match the numbers of seats projected ahead in the left-hand column. This is because I have decided to provide approximate expected numbers of seats, which should be less subject to "artificial" factors such as a party being projected barely ahead in a disproportionate number of seats. This change should make variations in the projection less puzzling: you won't see a party suddenly gain, say, 12 seats when it goes up by a point, but only 2 seats when it goes up by another point.

How do I estimate expected seat counts, given that as I've said before, I don't want to get into doing simulations? Based on my estimates about polling and seat model inaccuracy, I've determined that the standard error in the gap between the top two parties is about 9-10%, for an election taking place around the time of the last poll. Then I simply look at how far apart the top two parties are, and assign them probabilities of winning the seat based on the normal distribution. For example, if party A is 9-10% (1 standard deviation) ahead of party B, then party A would be given an 84% chance of winning, while party B would be given a 16% chance of winning.

More specifically, I first calculate an (usually too low) estimate for the third party's chance of winning by multiplying the probability that it gets more votes than the first party with the probability that it gets more votes than the second party. (This is usually too low because the two events are usually positively correlated.) Then I apportion the remaining probability of winning between the top two parties proportionally to their probability of finishing ahead of the other.

This is not an ideal method because it ignores underestimates third and ignores fourth parties. For most seats, this is not a problem, as only one or two parties have a significant chance of winning. However, if a party is involved in disproportionately many three- or four-way races as the party ranked third or fourth, that party will be slightly underestimated.

Teeny Tiny Projection Update

No new polls, but reading the news made me realize that there should be an adjustment for Howard Hampton (former ON NDP leader) not running again in Kenora. This swings the riding to a Liberal hold (LIB 31%, NDP 30%, CON 28%).

At the same time, I got rid of some rounding when modifying the 2015 provincial baselines to reflect such adjustments to riding-level 2015 baselines. That swings Québec back to the Liberals (but it's basically a tie either way).

Update Aug. 13: Ugh. I realized that when playing around with regional adjustments, I missed a row when I pasted the one I settled on in the end. That row is Rimouski-Neigette--Témiscouata--Les Basques, which should be marginally Liberal rather than NDP.

Seats projected ahead as of the latest national poll (midpoint: August 4)
LIB - 157 158
CON - 142
NDP - 22 21
BQ - 11
GRN - 5
IND - 1
If you're new to my 2019 projections, view key interpretive information here.

The popular vote projections are not affected by these updates.

External Links

As you can see if you consult this blog using a computer, I have increased the width of the blog and added a second column to the right-hand side, which contains all external links. There were too many things in the left column, which impaired navigation.

I have categorized the pollsters into "main national," "main regional," and "other." The national/regional distinction is self-explanatory. The main/other distinction is based on a subjective assessment of a pollster's length of track record and frequency of voting intention polling.

I have also categorized my "competitors" into "main" and "other." Those in the "main" category must satisfy all three of the following criteria:
- be based on averaging most recent available polling (e.g. not just a specific firm's polls),
- have been active after the 2015 General Election, and
- provide an easy-to-find and (at least) basic description of the methodology used.
Please let me know if you find an active Canadian projection website that I've missed, or if an inactive one starts up again and should be upgraded.

As you can see by clicking on the 2019 Methodology label (and About this Blog label for earlier years), I make a significant effort to explain how my projection works. I believe that, of all the seat projections available online, I currently provide the most precise description of methodology and idiosyncratic adjustments. (It helps that my methodology is relatively simple, as I do not make use of demographic characteristics.) I don't hold other sites to the same standard. However, some do not describe their methodology at all, and are therefore relegated to the "Other" category, along with inactive projection sites (which will be removed if they remain inactive after the campaign starts).

I'll add that the inclusion of a projection site in the "main" category does NOT imply endorsement of its methodology - although most in that category have a pretty solid one. I may comment on the quality of other projections in the future, but that is not the purpose of this post. If you're curious about track record in 2011 and 2015, posts comparing performances can be found under the Final Projection label.

Saturday, August 10, 2019

Map of Latest Projection

Update to Model Parameters and Discussion of Poll Weighting

As explained in the methodological post on poll weighting, the new formula is one of the main improvements that I have made to the model for this election: it derives the weights from a statistical framework rather than basing them on arbitrary discount factors. After a month of data coming in (I include all national polls that include field dates after July 1), it has become apparent that the individual-pollster variance parameter of 0.0001 (1% standard deviation) is too low; I am changing it to 0.0002 to better fit the recent data.

(Update Aug. 11: I have also changed the pollster common variance from 0.0002 to 0.0006 based on the frequency and magnitude of misses in recent provincial elections. This will only affect the confidence intervals, and will be reflected the next time I give such intervals.)

This update should make projections slightly more stable by increasing the model's weariness of concentrating weight on any one pollster. It causes minor changes to the latest projection, which is now:

Seats projected ahead as of the latest national poll (midpoint: August 4)
LIB - 155 (33.3%)
CON - 143 (35.5%)
NDP - 23 (13.9%)
BQ - 11 (4.2%)
GRN - 5 (10.0%)
IND - 1 (Wilson-Raybould)

If you're new to my 2019 projections, view key interpretive information here.

- Rimouski-Neigette--Témiscouata--Les Basques now stays NDP.
- Québec now remains a Conservative pickup (with caveat).
- Richmond Hill now remains a Conservative pickup.
- Hamilton East--Stoney Creek is now an NDP pickup.

What I want to do now is to illustrate how the weighting formula is working. As I explained in the earlier post, negative weights on some polls can arise, and I gave a theoretical explanation. In practice, negative weights are playing an even greater role than I was expecting... and it makes perfect sense.

For a full list of included polls and a link to each of them, see previous Projection posts.

Consider the following:
- 17 national polls combining about 29,000 respondents have been fed into the model.
- The total weight on the three most recent polls, totaling under 20% of the total respondents, by Innovative, Mainstreet and Abacus, is 64%.
- The total weight assigned to each pollster is:
16% Mainstreet
16% Abacus
13% Innovative
11% Forum
7% Léger
6% Angus Reid
6% Campaign Research
6% Nanos
5% Research Co.
5% Ipsos

As you can see, the formula achieves the dual objective of putting high weight on recent polls without giving any pollster undue influence. If you look more closely, you'll note that the three pollsters with the most recent polls (totaling 64% weight) have a combined weight of just 45%. This is achieved by assigning negative weight to those pollsters' older polls, and the sum of those negative weights is a significant -19%. If those pollsters did not have an older poll, then their newer poll would receive a lower weight, as there would be no way to correct for their pollster-specific bias.

Projection Update: Abacus 8/2-6

The following poll has been added to the model:
Abacus' 8/2-6 national poll (current weight among national polls: 28%)
For a full list of included polls, see previous Projection posts.

This Abacus poll was relatively strong for the Liberals in Atlantic Canada and QC. In ON, while it is not as high for the Liberals as other recent polls, the Liberal lead has grown from 1 to 5 points relative to the two previous Abacus polls.

This was also a good poll for the NDP, as were Abacus' two previous polls. So it seems like Abacus is pretty consistently high on the NDP these days.

Seats projected ahead as of the latest national poll (midpoint: August 4)
LIB - 159 (33.5%)
CON - 141 (35.4%)
NDP - 21 (13.7%)
BQ - 11 (4.2%)
GRN - 5 (10.0%)
IND - 1 (Wilson-Raybould)

If you're new to my 2019 projections, view key interpretive information here.

Caution: The Liberals are winning disproportionately many tight races, so the expected number of seats is probably closer to something like 155-145. Still, the projection keeps improving for them thanks to strength in key areas.

Relative to the last projection:
- Liberals take back one seat in NL from Tories: Avalon.
- Liberals take three seats in NS from Tories: Central Nova, South Shore--St. Margaret's, Sackville--Preston--Chezzetcook.
- Liberals take back one seat in PE from Tories: Egmont.
- Liberals take back one seat in QC from the NDP: Rimouski-Neigette--Témiscouata--Les Basques.
- Liberals take back one seat in QC from Tories: Québec.
- Liberals take back one seat in QC from the Bloc: Laurier--Sainte-Marie.
- The NDP takes back two seats in ON from Liberals: Ottawa Centre, Nickel Belt.
- Liberals take one seat in ON from Tories: Richmond Hill.
- Tories take back one seat in AB from Liberals: Calgary Skyview.
- The NDP takes one seat in BC from Tories: Burnaby South.

Friday, August 9, 2019

EKOS Tidbits

Frank Graves of EKOS has also tweeted some provincial results (ON and QC) for polling conducted this week. However:
a) he did not provide the full national numbers, and
b) he did mention that, despite weakness in ON and QC, the Tories lead by 5 nationally.
As a result, adding the ON and QC figures to the model would mean only adding the Liberal-favouring parts of a poll. Therefore, I will not be adding these results to the model.

Nevertheless, the good Liberal numbers in ON are consistent with the two other most recent polls (here and here) - all three of these polls show the Liberals up 11 points or more in ON. At the same time, the poor Conservative number in QC suggests that recent encouraging numbers for the Tories there may be due to random noise.

It will be interesting to see what Abacus shows in its release planned for tomorrow. Abacus numbers tend to be quite stable over time (which suggests that it may weight responses aggressively). Its last two polls had the Liberals up by only 1 point in ON. If Abacus also shows a big Liberal lead there, it could be time to start asking seriously what's going on in ON (or is it just Doug Ford?).