Tag Archives: nate silver

The rock, the hard place, and the eternally sought-after undecideds

The elections podcast by Nate Silver’s FiveThirtyEight team contained this rather interesting moment near its end on Monday:

NATE SILVER: Neither of these candidates has really won that many people over. Clinton is still at only about 42 percent in the [national] polls, down from about 44 or 45 percent right after the convention. 42 percent is not that good. It’s better than being at 39 percent, which is what Trump is at, but some of the marginal Clinton voters now have gone to [Libertarian candidate Gary] Johnson and [Green candidate Jill] Stein. How could Clinton potential lose this election if her favorables are slightly less bad than Trump’s?  If more of her voters go to Johnson and Stein. I think she needs a plan for dealing with that. If you assume that third party vote will fade… well, maybe… […]  but you certainly can’t count on that. I’ve never seen an election before where the number of decideds like goes up as the election goes on. [Laughter]

In this of all election cycles, maybe we should consider this before laughing.

This is an election cycle where, unlike in the last one, significant swings have proven possible and suggest exactly those unthinkable reversals. A lot of the restrictions I talked about in the last presidential cycle seem to continue to ensnare presidential contenders – most notably that Trump is trying what Romney wouldn’t, to say he’s at once in favor of two diametrically opposed immigration policies. But woven in between first Obama’s and now Clinton’s inability to effectively harness the news cycles and first Romney’s and now Trump’s need to hold two positions at once, there’s an almost supernatural destabilizing element: the decided voter who un-decides.

To fully credit them, there’s most likely no singular bloc of voters who fit that description. Even from the same part of the political spectrum, the motivation for a particular de-decider will vary, and as a result their undeciding can arrive at any number of times. While this seemingly new phenomenon is in some ways a reflection of this race – between two major candidates with net negative popularity, and maybe popular to get buyer’s remorse from – it’s also a manifestation of alienation from the two parties themselves.

That dislike for the two major parties doesn’t precisely fall evenly, and so neither do the un-decided. Amid recent allegations of corruption and other non-ideological criticisms, Hillary Clinton is perhaps more vulnerable to losing support for appearing to embody some of the greatest flaws in the system more generally. For Trump, similar allegations might limit or even undo his support, but the perception of him as an electoral outsider might also soften the blow.

Perhaps more coherently than any other recent presidential election, this one has been predicated on ideas of candidates’ relative flaws. With both major candidates facing limited enthusiasm and low popularity, running against their opponent has played a much bigger and more universal role this year than previously.

One of the problems that strategy poses, however, is that some of your support won’t kick in until it looks like you might lose. On the level of this that we have reached this year, what’s more, some of your supporters won’t necessarily stick around once it looks like you will safely carry the election. Conscientious voting has been raised as an issue in both primaries and into the general election, priming voters to ask themselves that if they don’t absolutely need to make a lesser-of-two-evils choice, then why bother.

2016-09-07_0936(The Princeton Election Consortium’s national meta-margin and FiveThirtyEight’s national polling averages, both showing the “sine wave” fluctuations Nate Silver mentioned earlier in the same podcast.)

For Clinton, someone absorbing support from her left and her right on the basis of her not being Trump, this creates boom-bust cycles of her support, or as Nate Silver put it – sine waves across the electoral polling. Like last year, the two major parties have pretty much played each other into a Democratic-leaning stalemate on the national level.

What seems to be new this year is that the sea is choppy, not that we’re in a different boat. The real proof of this dynamic, of course, will be born out in whether Clinton recovers some of these supporters now that the race is tightening again. Until then, as Silver said, we haven’t yet seen a race where the number of undecided voters goes up… but there’s always a first time.

Tagged , , , , , , , , , , , , , , , , , , , , ,

The image of the country “we are becoming”

In the wake of the 2012 Presidential Election, I wrote about Paul Krugman’s explanation that that election was something of an extension and expansion on the themes in the 2008 elections. Among other indications, it was a symbol that the idea of the United States as an open and diverse society was not merely a fluke of the 2008 electoral cycle, but an increasingly integral part of the country.

Sometimes, it’s difficult to keep sight of that within what I’ve called the “unreality” of politics here. It’s difficult sometimes to remember how utterly unrepresentative much of the media and the politics in the US actually are. To that point, there’s been two major explorations of how wide that gap is in the past two weeks.

comparison of Sunday shows proportionate representation of White men - all greater than 60% aside from Up with Chris Hayes which is 41%
(The comparison of the Sunday shows’ demographics, from here.)

Media Matters recently released this chart showing how Up with Chris Hayes more accurately depicts the actual demographic reality in the United States, especially when compared to other Sunday shows. The problem of overrepresenting male and White perspectives, unfortunately, is merely one that Hayes’ show has challenged in anyway, not one that his show has actually actively resolved. It’s worth noting that while 41 percent of his guests are White men (compared to 39 percent of the overall population), only 37 percent of his guests are women. Again, this is significantly higher than other shows – but the fact that Up is an outlier, while it still so chronically underrepresents women of all races, is cause for concern.

Likewise, as I’ve pointed out before with regards to MSNBC’s coverage, people of color and women are not the only groups systemically locked out of discussions on policies and attitudes that most directly impact them. Still, in spite of its failings, this is chart fairly concisely shows just how out of touch most broadcasts are with who US residents actually are – in terms of just race and gender alone.

But what’s particularly interesting is that a large swathe of the country is moving towards pluralism on such issues even while hindered by a media that rarely allows all of those “others” to air their concerns or perspectives. Nate Silver a few days ago pointed out that aside from Republicans, the United States is rapidly becoming more accepting of marriage equality. The research he cites breaks it down in terms of both partisan identity and general political identity.

Comparison of marriage equality support since 2001 between political groupsComparison of marriage equality support since 2001 between political parties
(Click to enlarge. Pew Research shows long terms increases in support for marriage equality among different political parties and identities over the past decade, from here.)

It’s intriguing to see how support for marriage equality has been steadily gaining support for years – some of them under the notoriously prejudiced Bush Administration. Likewise, although both the increase is smaller and the results less impressive, this is something that’s even improved among Republicans and conservatives.

There’s something in actually visually seeing that fact – that some segments of the media are actually becoming more representative in terms of race and gender and that the polling shows as well that we’re growing more inclusive as a country.

Tagged , , , , , , , , , , , , , , , , , , , ,

By their fruits you shall know them

I started off yesterday with a quick jaunt through some of the major predictions for how the electoral college would pan out yesterday during my live-blogging of the night’s returns. What I think is absolutely necessary to do now is the review those and look at exactly how things turned out. A few more honest predictors have already admitted it, but whether they recognize it or not, a shorter version of this is already well-known and getting passed around the internet today:

Keep calm and trust Nate Silver
(I first saw it here, but who knows who put this together originally.)

That conclusion having already been drawn, I think it’s worth analyzing it further. How much is trusting in Nate Silver actually worth (in comparison to other sources)? There’s a simple measurement to be done in terms of how accurate his calls were in comparison to the overall outcomes (hint: pretty much spot on). But there’s a variety of different problems with other predictions having flaws other than inaccuracy, most typically unwarranted uncertainty. It might be safe for pundits to only predict states that you can be absolutely certain about, but if you can’t provide an estimation for trickier situations, why should viewers rely on you for information? I want to be provided a service and I want it to be excellent. I’ll allow for some uncertainty, as it’s inevitable, but beyond a certain point, it’s an excuse for not doing your job, if your job involves predictions.

With those two potential problems in mind, I think it’s worth looking at both accuracy and overall information provided. To freshen in your minds what the various predictions were and to make comparisons easier to understand, here are the major predictions that were made as of Monday night, in map-form:

This slideshow requires JavaScript.

And here’s how those translate into the usefulness of the prediction:

Prediction Correctly Predicted Polities (#) Predicted Polities (#) Correctly Predicted Polities of Predicted Polities (%) Correctly Predicted Polities of All Polities (%)
Nate Silver (1) 51 51 100% 100%
Nate Silver (2) 50 50 100% 98.0%
RCP 40 40 100% 78.4%
Sam Wang 50 50 100% 98.0%
Michael Barone (1) 43 51 84.3% 84.3%
Michael Barone (2) 43 49 87.8% 84.3%

*I’m counting with 51 electoral blocs, as neither Nebraska or Maine appear to have split, and the District of Columbia gets 3 electoral votes as well.

You’ll notice, it’s relatively easy to get a high score in predictions, if you don’t make any for the tighter states. On the other hand, it’s a bit trickier to translate your predictions into a high degree of accuracy for the entire map unless you actually know what you’re doing (like, say, a professional statistician). In fact, counting only the errors in polity predictions understates the problem in the flawed analyses. I count 25 polities with fewer than 10 electoral votes each which all other these predictions agreed on. Allowing them to be factored in gives everyone a greater than 50 percent prediction rate while only working with 127 electoral votes. To be fair, there’s only 115 electoral votes from 9 states that the various models included here disagreed on. For the terminally curious, those are Nevada, Colorado, Iowa, Wisconsin, Ohio, Pennsylvania, New Hampshire, Virginia, and Florida.

Looking just within those selected states creates a very different perception:

Prediction Correctly Predicted Select (#) Predicted Select (#) Correctly Predicted Select of Predicted Select (%) Correctly Predicted Select of Total Select (%)
Nate Silver (1) 9 9 100% 100%
Nate Silver (2) 8 8 100% 88.9%
RCP 0 0 N/A 0%
Sam Wang 8 8 100% 88.9%
Michael Barone (1) 1 9 11.1% 11.1%
Michael Barone (2) 1 7 14.3% 11.1%

So there you have it. Sam Wang and Nate Silver are both highly accurate and highly useful. Real Clear Politics was accurate, but only because they literally said nothing about where it really mattered. As for Michael Barone, whose hilarious predictions can be read here in full (although they might disappear shortly), there’s a reason he could only get published at Rasmussen. Take note, statistics just defeated an actually baseless “sense” in yet another round. I’ll hope that Real Clear Politics’ cowardice and Michael Barone’s wrongness don’t go unnoticed, but I won’t hold my breath.

Tagged , , , , , , , , , , ,

Which ones actually are swing states?

Yesterday, a lot of people suddenly seemed to notice that there are disagreements between various pollsters and politicos over what states are actually “up for grabs” by either Romney or Obama. On The Rachel Maddow Show, Maddow briefly covered it, noting subtle discrepancies between the two campaigns (as Obama continues to focus on Florida and Virginia, while Romney seemingly feels more comfortable there). She left it up to her viewers to deduce why the NBC predictions she also referenced were distinct from either of the major campaign’s focuses in including New Hampshire and Wisconsin as “swing states” (but dropping Nevada, too). Clearly, there’s some politics involved in simply deciding what states are vulnerable in the election.

Jonathan Chait earlier in the same day made a similar point, arguing that:

“[the Romney campaign] is carefully attempting to project an atmosphere of momentum, in the hopes of winning positive media coverage and, thus, creating a self-fulfilling prophecy [… while the race is somewhat close,] Obama enjoys a clear electoral college lead. He is ahead by at least a couple points in enough states to make him president. Adding to his base of uncontested states, Nevada, Ohio, and Wisconsin would give Obama 271 electoral votes. According to the current polling averages compiled at fivethirtyeight.com, Obama leads Nevada by 3.5 percent, Ohio by 2.9 percent, and Wisconsin by 4 percent. Should any of those fail, Virginia and Colorado are nearly dead even. (Obama leads by 0.7 percent and 1.0 percent, respectively.) If you don’t want to rely on Nate Silver — and you should rely on him! — the polling averages at realclearpolitics, the conservative-leaning site, don’t differ much, either.

The only problem with his statements is that while Real Clear Politics provides the polls to prove Obama’s small but consistent advantage, it also provides the predictions that keep insisting that Michigan and Pennsylvania are some of the “toss-up” states. Looking over their map of predictions, it’s hard not to see the “horse race” that gives the media the ratings it loves. After all, states they classify as “toss-ups” hold more electoral votes than all of Romney’s “safe” states, either Romney’s or Obama’s “likely” states, and their combined “leans”. It’s only smaller than Obama’s “safe” states by 11 electoral votes. Their list contains all of the states either campaign considers worthy or visiting right now, the two additional ones listed by NBC, and two more – Michigan and Pennsylvania. A full fifth of the states are being contested in their predictions.


(Behold the gray faces states that hold our future in their hands. The screen-shot of their electoral college prediction is from this morning.)

There’s a key word at the beginning there – they classify states. Based on what? That’s not really said – but given their job as an aggregate polling firm, which collects polls from different pollsters to give a broad overview of what races are looking like, it’s hard to believe that polling data are totally irrelevant to their classification of states. If that’s true though, that polls are at the center of their predictions – then they really look like they have a double standard between what gets classified as an identifiable preference for Romney and an apparent choice of Obama.

Focusing on just the “toss-ups”, there’s immense variation between states’ polling results within that category. Admittedly, some of them look like what you’d imagine. The recently collected polls for Colorado show low results for either candidate, with quite a bit of alternation between who’s leading. There’s been a mix of polls showing either Romney or Obama leading throughout October, and into the summer. It really is difficult to feel confident that the state will go one way or the other. There’s a few other states that also fit this overall pattern – New Hampshire, Florida, Virginia and Iowa.

Slightly distinct are the “toss-up” states with some mixed polling results, but a clear tendency towards Obama. Nevada, for instance, has consistently seen extremely small leads for Obama, with no polls in the past few months showing a Romney lead (although there were two ties). Looking at Ohio gives similar results, as there’s a clear imbalance between the campaigns in convincing voters to support them, but the difference is extremely small. While there is adequate uncertainty to question the victory of the incumbent in those states, labeling both of them “toss-ups” seems to imply a degree of equal opportunity that seems unfounded.

On the other hand, there are some “toss-ups” that seem to be anything but. Michigan hasn’t seen a tie or Romney victory in the polls since late August. Wisconsin hasn’t seen either since mid-August. Pennsylvania hasn’t seen one or the other since February. Multiple months have seen no polls indicating a Romney win or extremely close race. Over those weeks, there have been periods where the incumbent enjoyed double digit margins of victory. Those polls are provided by a diverse group of pollsters – from the right-leaning Rasmussen to left-leaning Public Policy Polling, but all of them have found substantial Obama victories in those states for at least a month and a half, if not more.

And yet, these are still “toss-ups”,  because apparently some one in Real Clear Politics head office still isn’t really sure if they can even modestly suggest that Obama will carry them. In contrast, North Carolina was reclassified from being a “toss-up” to being a  “leans Romney” state on October 18. If you bother to look at their state-specific polling data, they changed their prediction after 17 days without any Obama wins in the polls – and with only five polls showing single-digit support for Romney. Now, I actually agree with both Real Clear Politics and Five Thirty Eight that North Carolina is more likely to end up in Romney’s column than Obama’s, but Real Clear Politics’ standards for reaching that conclusion seem at odds with their choices for, say, Pennsylvania – which has 261 days without any Romney wins in the polls and 39 polls showing occasionally double-digit support for Obama.

So while Real Clear Politics polling aggregation might suggest the same conclusions as “biased” Nate Silver, their predictions don’t match, provided the prediction would be of an Obama victory. Sounds like Chait might have been too charitable there in attributing major media predictions of a Romney win to confusion, rather than willful intent.

Tagged , , , , , , , , , , , , , , , , , , ,