2013 BC Election and the Pollsters

The dust is slowly settling after the surprising results of the 40th BC general election. The geographical divide between BC's different regions was very stark:

Island, Coast and Northwest: 15 NDP, 2 LIB, 1 GRN
Vancouver, Burnaby and New Westminster: 11 NDP, 5 LIB
Rest of Metro Vancouver: 20 LIB, 4 NDP, 1 IND
Fraser Valley and Interior: 23 LIB, 3 NDP
TOTAL: 50 LIB, 33 NDP, 1 GRN, 1 IND

But the salient feature of these results was, of course, its unexpectedness. Why did all the polls miss the boat? This seems to be a recurring theme in recent Canadian elections, but as we saw last fall, polls in the United States have remained quite accurate. Why are American pollsters doing so much better than their Canadian counterparts? Here are a few potential explanations, and how I feel about them.

1. Live Phone vs IVR vs Internet Polling
There is no evidence that traditional polls fare better than non-traditional ones in the United States or other countries. Some traditional Canadian pollsters suggest that new methodologies are behind the erratic polling in this country, but it's unclear why Canada would be different.

2. Resources for Data Analysis (Likely Voter Screen)
That said, methodology probably does matter, but at the stage of data analysis. Some US media organizations, unlike Canadian ones, actually pay substantial sums for their polls. This enables polling firms to do proper research and develop, for example, a reliable likely voter model. The availability of extensive exit polling data in the United States also helps.

3. Polarization and Undecided Voters
As we all know, the differences between Democrats and Republicans are stark, and even a month before the vote, there remain very few undecideds in a US presidential election, and in most US Senate elections. In Canada, the fraction of undecided voters is typically much higher. This produces a potential for sudden shifts that are hard for pollsters to capture.

4. Local Candidates
In the US, polls (except for "generic Congressional ballot" polls, which tend to be inaccurate) state the names that will actually appear on the ballot. In Canada, they typically only state the name of the party. A popular Canadian incumbent from an unpopular party may therefore enjoy more support than pollsters imply.

Hopefully, the Canadian pollsters figure things out before 2015, or the projections for the next federal election could be dead wrong...

Happy New Year!

Happy 2013!

Ironic tidbit: the PQ fulfilled its promise to raise taxes on rich Quebecers only partially so that the top tax rate would remain below 50%. However, with the expiration of the Bush tax cuts on high income Americans and the increased Medicare tax, individuals with federal taxable income above $400,000 and married couples with federal taxable income above $450,000 will be paying a marginal rate above 50% in California, New York City, Oregon and Hawaii. Those four jurisdictions have more than 50 million residents! It looks like Québec wouldn't have been so out of whack after all...

Rudimentary Guide to Election Night

Sadly, work is getting in the way, and I won't be able to post as extensive a guide as I hoped before polls close. The basic thing to remember is that states are typically called immediately only when the margin is greater than 10%. Looking at my state-by-state map, this means that while most states in dark blue/red will be called right away, most (if not all) other states won't. Obama supporters certainly shouldn't freak out if PA isn't called right at 8pm ET, or MI/MN aren't called right at 9pm ET.

We likely won't get a call in any battleground state (tossups and those coloured in the lightest blue or red) until at least 10pm ET. So if you're on the East Coast, you can probably go have a nice long dinner and still not miss any big calls.

If any of PA, MN or MI is called before 10pm ET, Obama is performing at least a little better than expected, and he should probably be viewed as a 95%+ favourite. (Right now, it's more like 90%.) On the other hand, if NC is called for Romney before 10pm ET, then Romney is overperforming, and we're likely in for a long long night.

The Democrats have a decent chance of winning the Senate as early as 11pm ET; if not, it shouldn't be too long a wait after that time. Obama will likely not win the presidency that early, and while I do expect him to win before sunrise tomorrow (most likely between midnight ET and midnight PT), there's a chance that we're headed for long legal battles...

Also, keep an eye out for Marriage Equality propositions in ME, MD and WA, as well as MN's proposal to enshrine discrimination in their state constitution.

Happy viewing!

2012 US Senate Elections

For posts on the Presidential election, click here (state by state) and here (popular vote).

Note: Maine is safe for independent Angus King; it is coloured in light blue since he is likely to caucus with the Democrats.

Some states are less solid than the others. Of the "lean Democrat" states, I'm least confident about MA, IN and VA. Of the "lean Republican" states, ND and NV have the best chance of causing a surprise.

I think the Democrat is marginally ahead in both tossups, MT and WI. However, the advantage is so slight that they might have a better chance of splitting. (For example, if the Democrat has a 60% of winning in each, and the two races are independent, then the Democrats have a 36% of winning both, and the states have a 48% chance of splitting.) For this reason, I view the most likely outcome as the Democrats winning all states coloured in blue and both tossups, but the most likely seat count as 53-47.

Later today: a guide to election night

2012 US Presidential Election: State by State

Click here for a summary of national polls.

Dark blue: Obama by 9 or more
Minnesota: Obama by 7
Oregon: Obama by 6
Michigan: Obama by 6
Pennsylvania: Obama by 5
Nevada: Obama by 4
Wisconsin: Obama by 4
Ohio: Obama by 3
New Hampshire: Obama by 2
Colorado: Obama by 2
Iowa: Obama by 1
Virginia: Obama by 1
Florida: Romney by 1
North Carolina: Romney by 3
Arizona: Romney by 8
Dark red: Romney by 9 or more

Electoral College count: Obama 303, Romney 235
Decisive state: Ohio
Tightest Obama state: Virginia
Least confident Obama state: Iowa (fewer polls than Virginia, and polls were off in 2008)
Tightest and least confident Romney state: Florida

Next up: Senate races

2012 US Election: Summary of National Polls

A small number of pollsters are expected to release their last polls on Election Day. This post may therefore be updated.

Most recent poll of 28 pollsters with national likely voter polls since October 4:
5, 4, 3.32, 3, 3, 3, 3
2.8, 2, 2, 2, 1.6, 1, 1
1, 1, 0.5, 0, 0, 0, 0
0, 0, -1, -1, -1, -1, -2
Average: Obama +1.2
Median: Obama +1 

The 21 with polling in November:
4, 3.32, 3, 3, 3, 2.8, 2
2, 2, 1.6, 1, 1, 1, 0.5
0, 0, 0, 0, 0, -1, -1
Average: Obama +1.3
Median: Obama +1 

Non/bi-partisan, live-operator polls with polling in November:
0 Politico/GWU/Battleground (11/4-5)
1.6 IBD/TIPP (11/3-5)
1 UPI/CVoter (11/3-5)
0 American Research Group (11/2-4)
0 CNN/ORC (11/2-4)
-1 Gallup (11/1-4)
3 ABC News/WaPo (11/1-4)
1 NBC/Marist (11/1-3)
3 Pew (10/31-11/3) 
Average: Obama +1.0
Median: Obama +1

Internet polls with polling in November:
2.8 Google Consumer Surveys (11/5)*
0.5 JZ Analytics (11/3-5)
2 Ipsos/Reuters (11/1-5)
3.32 RAND Corporation (10/30-11/5)
3 Angus Reid (11/1-3)
2 YouGov (10/31-11/3)
Average: Obama +2.3
Median: Obama +2.4
*Voters "100% likely" to vote

Automated phone/mixed/partisan polls with polling in November:
-1 Rasmussen Reports (11/3-5)
0 Gravis Marketing (11/3-4)
2 Public Policy Polling (11/2-4)
4 Democracy Corps (11/1-4)
0 Monmouth/SurveyUSA (11/1-4)
1 Purple Strategies (10/31-11/1)
Average: Obama +1.0
Median: Obama +0.5 

I think you can see why I predicted that Obama will win by about 1% :)

Election Day 2012 in America

Detailed posts coming up overnight or in the morning. But since the polls will be closing in about 20 minutes in Dixville Notch and Hart's Location, NH (these hamlets vote at midnight and are allowed to close their polls when everyone has voted), I'd like to give some approximate projections now.

Obama wins the popular vote by about 1%.
Obama wins between 271 and 347 electoral votes, likely 303.
The Democrats retain a majority in the Senate, with 52-54 seats. (This total includes two independents expected to caucus with Democrats.)

Five Days to 2012 US Election

The state of the race has moved very little from three weeks ago - so little that a few times, I thought about writing an update, but gave up since there was basically nothing to add.

In the past few days, things seem to have shifted marginally - maybe 0.5-1% - towards Obama: the latest national polls give him an ever so slight edge (roughly 0.5%), while Romney arguably had the advantage (by 0.5% or less) last week. To my knowledge, 24 different pollsters have released a national poll of likely voters since the Oct. 3 debate that shook up the race. Here are their most recent results, expressed in "Obama minus Romney" terms, from highest to lowest:

5.46, 5, 3, 3, 2, 1.3
1, 1, 1, 1, 1, 0.1
0.07, 0, 0, 0, 0, 0
-1, -2, -2, -2, -3, -5

Average: Obama +0.41
Median: Obama +0.085

Listing only the 18 polls that started on or after October 20 (or, equivalently, that ended on or after October 24) gives us:

5.46, 5, 2, 1.3, 1, 1
1, 1, 1, 0.1, 0.07, 0
0, 0, 0, -1, -2, -5

Average: Obama +0.61
Median: Obama +0.55

Sixteen of these have updated over the past week. By this I mean that as of a week ago, they had released a post-first-debate poll, and they released another one in the past 7 days. The changes in the Obama-Romney gap are:

-2.3, -2, -1.7, -1
-1, -1, -0.7, +1
+1, +1, +1.44, +2
+2, +3, +3.07, +4

Average: Obama +0.55
Median: Obama +1

This national movement was reproduced in many of the swing states. In particular, Virginia, Colorado and New Hampshire, which were virtually perfectly tied as of my last post, now all appear to have an ever so slight Obama lean. Obama may not win all three, but Romney is even less likely to sweep them.

Similarly, in Florida, Romney's already slight advantage has eroded to the point that the state is almost a virtual tie again. The state is definitely in play, and if Obama wins Florida, it's game over for Romney. However, it's unlikely to be the decisive state since there are 303 electoral votes that appear easier to get for Obama than Florida's 29 electoral votes.

Obama's most likely path to 270 is still through Wisconsin, Nevada and Ohio. This trio puts Obama at 271, barring a huge upset in a state like Pennsylvania, Minnesota or Michigan.

Something to keep in mind: in 2008, the polls grossly overestimated Obama's margin in Iowa. Therefore, even though Obama appears to have a small but stubborn lead there, I'm not too confident that he will win. On the other hand, the 2008 polls grossly underestimated Obama's margin in Nevada (as they did Harry Reid's in 2010), so it seems very likely that he'll carry that state even though his polling lead there is also slight. In Colorado as well, the Democrats were underestimated in both 2008 and 2010, though the effect wasn't as big as Nevada.

US Election: All Eyes on Ohio and Virginia

Three weeks ago, I gave a list of 10 states totaling 125 electoral votes that will likely determine the outcome of the 2012 U.S. presidential election - both candidates need a majority of those 125 votes to win. At the time, Obama led in all 10 of those states, but of course, Romney has gained substantial momentum since then with his strong performance in the first debate.

Right now, based on state-by-state polls, I'd classify those states as follows:
- Clear Obama lead: PA (20), MN (10)
- Obama edge: OH (18), WI (10), IA (6), NV (6)
- Tossup: VA (13), CO (9), NH (4)
- Romney edge: FL (29)
This gives a 277-235 lead for Obama in the electoral college.

(By the way, some other states like MI (16) on the Obama side or NC (15) and AZ (11) on the Romney side are also relatively tight, but they are very unlikely to be decisive. In the above list, it looks like MN is also firmly in the Obama column.)

However, as Nate Silver has pointed out time and time again on his blog, the national polls have lately been somewhat more favourable to Romney than the state polls. If we give the "Tossup" states to Romney, we end up with a very tight race: 277-261. This is very similar to Mr. Silver's "now-cast", which gives Obama a 280.4-257.6 lead (he has NH as leaning Obama).

We can now probably narrow down the list of states to watch to the seven "Obama edge" or "Tossup" states above: OH, VA, WI, CO, IA, NV and NH. This doesn't mean that PA will definitely go for Obama or that FL will go for Romney, but if those things don't happen, the winner is likely to have over 300 electoral votes.

Ohio and Virginia are the keys to the election: if a candidate wins both of these, he forces his opponent to run the table on WI, CO, IA, NV and NH.

If Ohio and Virginia split, then, in most cases, whoever can carry three of WI, CO, IA, NV and NH wins. The exceptions are:
- OH, WI and any one of CO/IA/NV is enough (so VA, NH and two of CO/IA/NV is insufficient)
- OH, CO and either IA or NV is enough for Obama (so VA, WI, NH and either IA or NV is insufficient for Romney)

Polls close at 7 in Virginia, 7:30 in Ohio, 8 in New Hampshire, 9 in Colorado and Wisconsin, and 10 in Iowa and Nevada (all times Eastern) - conveniently, the two most important states close first.

Non-Payers in Canada

As you've probably heard by now, Mitt Romney made some disparaging remarks about the estimated 47% of the US that do not pay federal income taxes. A natural question for us Canadians is to wonder how many non-payers there are in Canada. Glen McGregor suggests that the comparable figure for Canada is 34%, but unfortunately, he is comparing apples to oranges. So here's a more accurate answer.

The 47% (actually 46.4%) might refer to the proportion of estimated total tax units in the US that had no federal tax liability in 2011, according to a model by the Urban-Brookings Tax Policy Center. Or it might refer to the estimated 45.9% of estimated total tax units in the US that had no federal tax liability in 2010, according to actual IRS data for filers and estimates of non-filers.

The 34% refers to the proportion of tax filers in Canada that had no federal or provincial tax liability in 2009, according to actual CRA data.

Obviously, the year used in the computations and the data sources are different. But those are not the main problems here.

One issue that will always arise when comparing the US and Canadian income tax systems is that most married couples file a single return in the US. This is why US data refers to "tax units" rather than individuals. But this, too, isn't the main problem, since it's unclear how this distinction skews figures.

A major issue that is easily fixed is that the 34% refers to Canadian filers that did not pay federal or provincial income taxes. But, as it turns out, over a million Canadians that paid provincial income tax in 2009 did not pay federal income tax. The US statistic refers only to federal income tax, so a more comparable figure would be the 38.2% of Canadian filers that paid no net federal tax in 2009.

Another major issue is that the 38.2% refers to the proportion of Canadian filers, while the 47% in the US includes non-filers. The proportion of American filers that didn't pay federal income tax in 2010 was 40.9%. Now, filing requirements are different in the two countries, so even comparing filers with filers is not a fair comparison. Unfortunately, I can't find an estimate for the adult population in Canada at year-end 2009, but the estimates for the 15+ and 20+ population in mid-2009 and mid-2010 are readily available from Statistics Canada. From those, we can estimate that the adult population was roughly 27.0 million at the end of 2009, which means that the proportion of the Canadian adult population that did not pay federal income tax in 2009 was about 41%.

Looking at 33.9% and 46.4% might drive us to think that, proportionately, much fewer Canadians than Americans escape the federal income tax. But when you realize that the comparison should be between 41% and 46%, the two countries look much closer.

In actuality, the gap is probably even smaller. In the US, the child tax credit gets directly subtracted off your income taxes. By contrast, in Canada, the child tax benefit (as well as the GST/HST credit) is provided throughout the year. So a family that owes $400 and gets $1,000 in child benefits would count as non-paying in the US, while it would count as paying in Canada.

In the end, the proportion of Canadians that avoid federal income tax is probably not much lower than the proportion of Americans. Careless interpretation of data in the media strikes again...

McGregor also mentions at the end of his piece that much more Quebecers (38%) than Ontarians (32%) or Albertans (28%) avoid income taxes. But he's looking at provincial and federal income taxes for ON and AB, while for QC, he's only looking at federal income taxes since the CRA does not collect Québec income tax. The actual fractions of filers that did not pay federal income tax in 2009 are:

NL 41.5%
NB 39.5%
NS 39.5%
MB 39.1%
ON 39.0%
BC 38.84%
QC 38.82%
PE 37.8%
SK 37.3%
AB 31.6%

A slightly greater proportion of Québec filers paid federal income tax in 2009 than Ontario or BC filers!