Null Results

16 January 2013, 1002 EST

Chris Blattman links to a paper (PDF) that finds no relationship between oil production and violence. He comments:

Regardless what you believe, however, there’s a striking shortage of null results papers published in top journals, even when later papers overturn the results. I’m more and more sickened by the lack of interest in null results. All part of my growing disillusionment with economics and political science publishing (and most of our so-called findings). Sigh…

To which I say, “Yes. Yes. A thousand times yes!”

If we really care about truth-seeking in the social sciences, let alone our own discipline of political science, we would consider null results on significant issues of critical importance. We would certainly consider them more important than the far-too-common paper with a  “positive” result that results from, well, efforts to get to a positive result via “tweaking” (e.g., and, also).

Indeed, no one — and I mean no one — who has ever invoked “Popper” or “falsification” as a standard for scientific inquiry should be allowed into a proseminar while graduate students remain actively dissuaded from pursuing research with null results because everyone knows you can’t get null results published. 

Of course, the non-Popperians in the discipline aren’t much better.

I suspect some of this is related to what Hein Goemans calls “theory deification,” but even that’s perverse, because if we really cared about theory we’d be completely comfortable publishing results that make a plausible case for having refuted important theoretical claims. But as the pattern of favoring positive results may be increasing across disciplines (PDF), there might be more pedestrian “sociology of science” explanations, such as funding scarcity.

Regardless, the hypocrisy… it burns.