This brief post started life as a comment on a Facebook discussion thread about peer reviewing practices but I thought it might deserve a wider readership. The question was raised: is it kosher for a journal editor to request information about good reviewers from the author of the manuscript? The general consensus, with which I agree, was that it’s fine to request those names because editors are always looking for qualified reviewers, and the author’s list might provide names that the editor might not have thought of. Of course, the editor need not be bound by that list, and shouldn’t be, but sometimes people in a subfield (or a sub-subfield) know the specific lay of their part of the intellectual landscape better than an editor does.
That said, this is the kind of thing that can be easily abused, as people game the system and give names of people who are most likely to give their manuscript a thumbs-up. And the peer review system is a creaky beast in any event, so I thought I’d lay down a few imperative commands to journal editors, mainly in order to provoke discussion but also to summarize in concise form my own experience both as a journal editor and as an author and reviewer:
1. Thou Shalt Not Commit “Courtesy Reviews,” in which the person or persons critiqued in the manuscript are given the ability to gate-keep and rule definitively on the publishability of the manuscript. If you do send a manuscript with the subtitle “Scholar X is wrong” out to be reviewed by scholar X (which you probably shouldn’t do in the first place…come on, can you say “conflict of interest”?), make sure that you a) counterbalance that review by asking other people to review the manuscript, and b) discounting scholar X’s review heavily.
2. Thou Shalt Not Commit “Club Reviews,” in which the only people asked to rule on the publishability of a manuscript are people in the same sub-subfield as the author and already agree with the author on most points. Sure, sometimes you need the opinion of some member of the club to help you decide if an apparently-insignificant-to-the-unintiated claim is something that fellow specialists would find intriguing and worthwhile, but you can’t just let members of the in-group make that determination. If you do need to consult a member of the club, counterbalance that with a review by someone who is very much not a club member.
3. Thou Shalt Not Publish A Manuscript The Significance Of Which You Cannot Explain To The Wider Readership Of The Journal (the Lee Sigelman rule). This is the most important editorial commandment, in my view, subsuming the previous two in many ways. I do not think that any journal editor has any business publishing any article that doesn’t seem to her or him to be of substantial, significant importance to the journal’s readership, whether we are talking about “peak” field-wide journals or more specialized sub-field oriented journals. The fact that the reviewers thought the piece was important is never enough, since you might have accidentally broken the Second Commandment. The fact that one of the reviewers thought the piece was crap might signal that you have accidentally broken the First Commandment. There is no substitute for informed editorial discretion, and the basic rule of thumb that should be followed (which Sigelman followed during his tenure as APSR editor; I also followed it as Editor-in-Chief of the Journal of International Relations and Development) is whether the editor can explain the significance of the piece without simply quoting the manuscript. When I was editing JIRD we used to do an “editorial introduction” to each issue of the journal, in which I’d explain the significance of what we were publishing as I understood it; I had to stop that because of concerns about the number of pages in each volume of the journal, but internally, I still made each manuscript pass that test before I and my team signed off on it.
The punchline here is that editing a journal is a discretionary enterprise. Editors have a lot of power, and adhering to these three commandments is important if one wants to do a good job exercising that discretion.
Interesting point, and given the success of JIRD, you must’ve done something right.
Question: do editors have lists that match ms topics with reviewers? And do they share those lists with editors of other journals? I ask because virtually any ms that deals with a particular topic within the wider field of security studies seems to end up on my desk. Never mind that I no longer do that stuff myself.
Good luck with the ISQ. Hope you can do some magic to that birdcage lining…
We did have an informal list at JIRD, but mostly we relied on the personal knowledge of each of us on the editorial team. If someone did a particularly bad review — not bad as in “this piece is terrible,” but bad as in gate-keeping or just not engaging with the argument on the page — we’d make a note of it and make sure not to use them again.
Great post, especially the 2nd Commandment.