The Rhino in the Room

9 June 2011, 0121 EDT

Somehow I managed to delete my mediocre post on peer review. The gist: peer review is arbitrary and capricious; summary rejections offer a cosmetic fix; we need to reduce our reliance on counting peer-review journal articles as a basis for evaluating scholarly worth.

Jarrod Hayes commented:

I am struck by the arbitrariness of peer-review. I have at least once asked to come in as a reviewer when two reviews reached polar opposite conclusions about a manuscript. Both reviews were careful and conscientious, and I found merit in both when I saw them in the decision letter to the author. How could they both be right?

To which I respond: at least they were both conscientious! I sometimes play a game with a friend called “guess which review was mine?” It isn’t much of a game, to tell the truth. Both of us tend to produce long reviews, often with full references and explanations for why citing particular omitted work matters, and that seldom use denigrating language like “this is obviously a seminar paper.”

But Jarrod raises an absolutely crucial point: publish or perish depends a great deal on luck of the draw. Many of the “top journals” in our field engage in a form of “peer-review triage” in which all of the reviews have to be at least fairly strong R&Rs (“revise and resubmit”) to avoid rejection. This means that it is quite possible for a manuscript to accumulate more “accepts” than “rejects” and never see the light of day–at least as at a “prestigious” journal. Is that evidence of a functional system for allocating status and success?

Dan Nexon

Daniel H. Nexon is a Professor at Georgetown University, with a joint appointment in the Department of Government and the School of Foreign Service. His academic work focuses on international-relations theory, power politics, empires and hegemony, and international order. He has also written on the relationship between popular culture and world politics.

He has held fellowships at Stanford University's Center for International Security and Cooperation and at the Ohio State University's Mershon Center for International Studies. During 2009-2010 he worked in the U.S. Department of Defense as a Council on Foreign Relations International Affairs Fellow. He was the lead editor of International Studies Quarterly from 2014-2018.

He is the author of The Struggle for Power in Early Modern Europe: Religious Conflict, Dynastic Empires, and International Change (Princeton University Press, 2009), which won the International Security Studies Section (ISSS) Best Book Award for 2010, and co-author of Exit from Hegemony: The Unraveling of the American Global Order (Oxford University Press, 2020). His articles have appeared in a lot of places. He is the founder of the The Duck of Minerva, and also blogs at Lawyers, Guns and Money.