The Duck of Minerva

What Is Policy Relevance, and (How) Should We Reward International Relations Scholars for It?

Featured

by Johannes Urpelainen

19 September 2013, 1209 EDT

The subfield of international relations seems to suffer from an inferiority complex. While most subfields of political science do their research and trust that the results are relevant to policy for a good reason, many an international relations scholar complains that the subfield is not relevant enough. The latest example is Campbell and Desch, who worry that rankings of departments are biased against policy relevance in international relations scholarship. Not surprisingly, Stephen Walt chose the bandwagoning strategy in this case (gated content, sorry).

While I don’t worry about rankings too much (I am perfectly capable of judging the quality of academic research myself, thank you very much), I do agree with some of their claims. It is odd to exclude books from consideration in a ranking, given that many of the major ideas in international relations scholarship are presented in books even today (for examples of epic books that have changed our field for so much better, see here and here). Exclusion of interdisciplinary journals from consideration is also highly problematic, since solving the world’s most pressing problems require a combination of social and natural sciences.

Where I strongly disagree with Campbell and Desch is on the inclusion of non-peer reviewed work in the evaluation of departments and scholars. Publishing in Foreign Affairs, they believe we should reward publication in outlets that do not use peer review, such as… *drum rolls*… Foreign Affairs:

“One cut at trying to rank programs by the presence of their faculty in non-academic publications involved factoring in Foreign Affairs and Foreign Policy articles.”

This seems wrong to me. Academics work at universities as scientists. Our research differs from that being produced by think thanks in that academic research must survive peer review by other scholars, and this peer review is based on scientific criteria such as theoretical innovation, the quality of the research design, and contribution to knowledge. Academics have only one advantage, and that is the scientific method. If it were not for the scientific method, I would not pay any attention to academic commentary. I have many good friends who work for think thanks, and they are much better informed about current events than I am. That is their advantage. But my research is more systematic than theirs, and that is my advantage. Both types of researchers are needed.

What about policy relevance? Are my standards a recipe for irrelevance and obscurity, as some claim? No, they are not. Good scientific research on important topics is almost by definition relevant to policy. The world is full of non-governmental organizations, social movements, and governments that desperately need rigorous research to improve their programs and policies. Academic policy relevance should be defined as the ability to use the scientific method to contribute to policy formulation. Insightful commentary based on a gut feeling or authority is not academic policy relevance. It results from a fundamental misunderstanding of what the role of academic institutions is in the global society. International relations scholars who feel the need to comment on current events based on their personal views or experiences can do so, but their policy relevance must be evaluated based on their ability to use the scientific method to add value.

I would not mind publishing in Foreign Affairs myself. I certainly would not mind if my academic salary were increased for all of my reports, consulting, and field projects that are changing the world out there even as I write this. I spend a lot of time doing policy relevant research, and I can point out to real change without using misleading proxies such as my name in Foreign Affairs or the number of times my corny jokes have been tweeted. But as an academic, I am more than happy to subject my work to peer review. If my arguments are logically flawed or my identification strategy weak, I should not be rewarded just because some policymaker out there wants to justify a policy by referring to an Ivy League academic who is of the same opinion. International scholars should work harder than ever before to do the kind of research that survives the difficult process of peer review. While peer review is capricious and the outcome noisy, I believe truly excellent and policy relevant research will find its way to a good journal.

Johannes Urpelainen is the Director and Prince Sultan bin Abdulaziz Professor of Energy, Resources and Environment at Johns Hopkins SAIS and the Founding Director of the Initiative for Sustainable Energy Policy (ISEP). He received his PhD in Political Science from the University of Michigan in 2009 and spent the next eight years at Columbia University.

Johannes is the award-winning author of four books and over a hundred refereed articles on environmental politics, energy policy, and global governance. He teaches action-oriented classes on energy and environmental policy to equip the next generation of global leaders with deep knowledge, advanced analytical skills — and a passion for transformational social change. As one of the world’s top energy policy experts, Johannes frequently advises governments, international organizations, and the private sector on energy and environment.

2 Comments

  1. John O.
    John O. on 19 September 2013 at 14.00 EDT

    I agree with much of what you say here, but I interpret the call for “policy-relevant” research a bit differently than you do. I see this as a call for scholars to do more to make their systematic inquiries accessible and relevant to the policy world, something that often calls for translation (rephrasing arguments in ways that avoid the specialized language of IR) and reaching a broad audience by publishing in non-peer-reviewed venues (like Foreign Affairs). My impression is that many scholars would agree that we should NOT sacrifice systematic scientific inquiry for “insightful commentary based on a gut feeling,” but they would argue that scholars need to work harder to make their scientific knowledge accessible to a wider audience. If they don’t do this, scholars risk having their work misappropriated by pundits who fail to understand how systematic inquiry is carried out.

    I remain ambivalent about how central this effort should be to the professional advancement of scholars, but I do think it should be acknowledged as a valuable contribution to scholarly discourse.

  2. RedWell
    RedWell on 20 September 2013 at 15.43 EDT

    Thanks for this: these articles are floating around this week, but few people are making this distinction or pointing out that LOTS of folks are already doing “policy relevant” research. John O. also makes a good point, here. Given that, my two cents: we need to reward people who basically integrate and translate primary research into accessible narratives and other forms. I don’t mean textbooks, which are typically dreary and dense despite efforts to lighten them up. I mean articles, books, websites and the like that involve adroit, readable prose. The concept is not that complicated, but it’s not easy, and few junior people are encouraged to develop the relevant skills. Folks like Fareed Zakaria and Joseph Nye are good examples here. Maybe they oversimplify a bit, but at least their ideas derive from more systematic thinking and, more importantly, people actually read it. FP and Foreign Affairs work to some extent here, but those are primarily elite affairs and usually focused on advancing particular policy agendas. Anyway, the point is that we can envision a middle type of scholarly output between primary, R1 research and direct policy application that undergrads, policy makers and the (allegedly) well-informed public might actually want to read.