Skip to main content

Can’t Lose Counterfactuals: On Elections, Commentary, and Comparisons

After every presidential election, before the “normal” produced by its results takes hold, election season is extended through proliferative and variegated (if not particularly conclusive) analyses of what led to the outcome under examination.

The crucial elements of this period—what makes it so instructive and important—are two facts that are never raised. First, elections tell us what happened, but not why. Second, most American political commentators come to the job outfitted—like derivatives traders circa 2006—with equipment suited to their job, if not the job.

For the US population at large, it goes without saying that America is exceptional. Exceptional not just as in really good (although that’s certainly an important aspect of what is meant), but as in the normal rules and relationships do not apply. As in sui generis. As a shibboleth of the Republican Party, American exceptionalism comes up with some regularity as a sort of patriotic fealty test, particularly when it is invidiously used to highlight the anti-Americanism of Democrats, liberals more generally, and/or Barack Obama.

But beyond this slapstick Republican ritual, America as exceptional is a real thing—at least in how Americans are socialized. Maybe not everyone uses the term, but the feeling that America is unique—incomparable—is a belief that permeates American life from the American Enterprise Institute to public television to sports stadiums to furniture showrooms to coop cafes to constitutional law classes to city bodegas to Midwest grain silos. The phrase “America is the greatest country in the world” is familiar to every schoolchild in the land.

This belief in exceptionalism naturally directs attention inward—how can we look outward for comparison when we are the exception? And it is this incessantly inward look that brings us to the counterfactual.

In the first half of 2012, just for instance, Barack Obama argued that, had it not been for his policies, the US economy would have been in much worse shape. Thus, rather than comparing the relative conditions of the US and Europe (verboten) to demonstrate conclusively that the absence of his policies would have led to far worse economic outcomes, he used a counterfactual argument.

Precisely what, then, is the counterfactual—and why does its use matter?

Modern use of the counterfactual has its roots in John Stuart Mill’s comparative method of difference. In sum, Mill explained that if in two highly similar instances there is a different outcome, what differentiates the instances causes the different result. But when you only have one exceptional instance, there is no comparison. The counterfactual neatly sidesteps this problem by inventing a comparison through thought experiment: posit a change to a potentially causal factor in a past sequence of events, then imagine what that change would have done to the outcome in question.

Example proposition (a classic, in high school classrooms and International Relations departments alike): The assassination of Archduke Franz Ferdinand caused World War I. Counterfactual: posit that he had not been assassinated, but everything else had been the same; would World War I have happened? If yes, then according to the method of dissimilarity, this was not a causal factor. If no, then it was a causal factor.

There are some rules to creating a good counterfactual in the social sciences (explicated below using one of the great “what ifs”: the Bush v. Gore presidential election of 2000):

1. Plausibility: the posited change must be reasonable and imaginable so that it is relevant to understanding the world in which we live.

What if Al Gore had won the 2000 election? Plausible. What if the 2000 election had caused a civil war in the United States? Not plausible.

2. Minimalism: the change should be a small as possible, to ensure that everything else between the imagined and actual world remains as similar as possible. This is essential for application of the method of difference.

What if Al Gore’s vote count had been a few hundred higher in Florida? Minimal. What if Bill Bradley had won the Democratic primaries? Not minimal.

3. Close connection to the outcome in question: the question should be, how would the outcome be different after its apparent cause is changed? (As opposed to the more tempting but less rigorous question, how would the entire world be different?)

If he had won, and 9/11 had still happened (remember, minimalism), would Al Gore have taken us to war in Iraq? Closely connected. Would Al Gore have prevented the economic crisis? Not closely connected.

As with all rules of argumentation, the rules governing counterfactuals are generally immediately ignored in arguments that employ them, in large part because ignoring the rules opens the door to endless contestation. So, the (white) American curmudgeon’s greatest desire—“What if things were still like they were in the 1950s?”—violates the rules of plausibility, minimalism, and close connection to the outcome (i.e., all three), and yet you may not find a more common use of the counterfactual.

When the rules are followed, rare as that may be, the counterfactual can be used to great effect. For example: Barack Obama lost the first presidential debate of 2012 worse than anyone has ever lost any presidential debate in the television era and yet he won the election. This election therefore confirmed that debates don’t matter (are not, that is, of causal importance), correct?

This is a counterfactual; in fact, it is one of two possible counterfactuals. The first argues no difference in outcome had Obama, say, tied the debate. The second argues no difference in outcome if the election had been held on October 14, 2012.[1] But when put this way, the first counterfactual would lead most on the victors’ side to suggest a much larger win for Obama, at a scale that led some to hope (in September of 2012, at the height of Obama’s lead) that the House of Representatives might go to the Democrats. The second, on the other hand, would have them quaking in their boots. The first counterfactual is therefore a good indication that the debate mattered; the second is a good indication that everything that came after the debate also mattered. Both undermine the cause of those—like some political scientists who suggest elections are determined purely by economic conditions—who think that none of it mattered. And both also show that, in certain instances, using the counterfactual well can expose the inner-workings of an argument, in this case to quickly refute it.

Or take a counterfactual that has been plaguing the economy and American politics for three years: What if the 2009 stimulus package had been $1.2 trillion rather than $800 million? At least as an economic thought experiment, this is plausible, minimalist (if not in dollar terms), with a clear and relatively close connection to an outcome like GDP or unemployment. Contestation has been in the last, of course (the debate over stimulus versus confidence), which highlights the trouble counterfactuals can cause even when they are done correctly.

The whole point of comparison is to use the observed variation in the world to influence our perspectives. Yet the counterfactual requires inventing variation for comparison, because no proper comparison exists. The counterfactual is therefore an altered reflection of the world we already see, so it has a tendency to reinforce (or distort) perspectives, rather than influence them. This is especially salient during a post-election period filled with recrimination on the side of the losers (who endlessly—and, to the other side, entertainingly—re-contest the election under constantly varying circumstances) and triumphalism on the side of the winners (who proclaim victory as their birthright, unalterable under any change of circumstances).

But the fact is, elections do not answer our questions because there is always only one election.[2] And in 2012, as in every other relatively close election, the causes of the outcome were jointly sufficient.

Joint sufficiency is one of the most tempting, but fraught, circumstances for counterfactuals. When a group of factors jointly produces an outcome, it gives immense power to those counterfactual-makers suggesting changes to just one factor on the basis of a presumption that one change makes the whole thing fall apart. This gives a veneer of very close connection to the outcome, and the logical bona fides that come with it, but it obscures some of the most egregious violations of the other two rules.

The very business of political punditry is based on making strong (not minimalist) claims about what caused what, with at best superficial plausibility. Pundits can say that demography won Obama the election; that negative advertising in the summer won him the election; that gay voters won him the election; that Latino voters won him the election; or that African American voters won him the election. The last three in particular show the affinity between joint sufficiency and bad counterfactuals. The fact is that if you got rid of any of a number of large chunks of Obama voters, it would probably impact the election result, a fact as obvious as it sounds yet missing another obvious point, that those large chunks of Obama voters are, in fact, large chunks of Obama voters. Imagining a world in which Obama voters were not Obama voters, in other words, is neither minimalist nor particularly plausible—thought it would certainly result in Romney having won the election.

Republicans can say that Mitt Romney couldn’t deliver a good message; that the Romney campaign’s election-day software failed; that the media shied away from the explosive truth about Benghazi; or that Romney lost because the party needed to be different. Mitt Romney himself, a man who went very quickly from president to pundit, can say a week after the election that the Obama administration’s largess—“gifts”—to its base voters won the election for the Democrats.

These are all implicit counterfactuals. And because they are also bad counterfactuals, violating any number of the rules, they are impossible to sort out in a single run, closely contested election.

Lack of clarity gives immense argumentative power to those who analyze our politics for a living: on their terms it becomes basically impossible to disagree without making counterclaims of basically the same type.  And all of these bad counterfactuals are also utterly free of what little consequence there is in being wrong, because for once pundits are trying to explain the past rather than predict the future.

That the state of popular political analysis is lamentable is no great insight, but the point here is that the regular recurrence of the bad counterfactual (which contributes to this poor quality) is in fact rooted in the profession. Using a bad counterfactual follows a more rigorous logic than that required to produce a good counterfactual. For where there is clarity, there is refutability. One can be right or wrong. Where there is an intriguing (to one’s audience) premise with no possibility of being proven wrong, however, there is only the provocation or accession of the crowd.

The key here is to stress that all of these people bringing us bad counterfactuals are not necessarily bringing us bad analysis (though often they are). More importantly, what they’re doing is capitalizing on the counterfactual. What makes a counterfactual “bad” in a social science sense makes it a powerful tool in the media—it sounds good, changes the result, and you can’t really argue with it based on facts. And like doing business with those derivatives traders in 2006, we may have no idea what we’re being sold, but we love buying it, all while spiraling towards disaster: in this case, living with our media in a made up world.

Daniel Schensul is a sociologist working for the United Nations on poverty reduction, urbanization, and climate change in the developing world. His published work addresses urban development after apartheid in South Africa, comparative historical analysis, climate change vulnerability assessment, and the links between population dynamics and climate change mitigation and adaptation.

[1] Romney’s peak in the New YorkTimes’ analyst Nate Silver’s celebrated election model.

[2] Do past presidential elections provide good comparisons to temper the constant counterfactual? In the fall, we were wondering whether 2012 was a 2000, a 2004 or a 2008 election. It was none—but anyway these comparisons are about what happened, not why. Or let’s take “it’s the economy, stupid.” This is basically the fundamentals argument, and it’s true except when it isn’t.

+ posts

Leave a Reply