For caveats and background, see my introductory post.

Editors write a lot of decision letters. At high-volume journals, editors write so many decision letters that it can become a tedious grind. For authors, though, the information communicated in decision letters matters enormously. It can affect their job prospects, salaries, and chances of advancement. Of course, authors, especially in the moment, overestimate the significance of any single journal decision. But receiving a rejection, revise-and-resubmit invitation, or an acceptance can certainly feel like a defining event. This is especially the case for graduate students and junior academics, who are less experienced in, and more vulnerable to, the vagaries of the review process.

This makes decision letters the single most consequential way that editors communicate with authors. The same is true for referees. We don’t spend a lot of time teaching academics how to craft referees reports. There is, at best, limited consensus about what makes for a good review. So decision letters also become an important way to send cues to referees about the quality of their reports.

If you think about it, all of this places a heavy burden on editors. That burden only seems heavier when we consider how arbitrary and capricious the peer-review process can be

Yeah. Okay. I’m being a bit melodramatic. Editors don’t perform literal surgery. They don’t design airplanes. The stakes are what they are. But I stand by the underlying sentiment: editors have a responsibility to take decision letters very seriously.

In this post, I’ll focus on general issues. In Part II, I’ll elaborate on them in the context of the specific kinds of decision letters.

Preliminary Considerations

Before I discuss my views on the nuts and bolts of decision letters, I want to walk readers through some overarching considerations. These come in two major flavors: “uncomfortable truths” about peer review and “everyone deserves respect.”

Uncomfortable Truths

The peer-review process is held together by masking tape and pixie dust. Most international-relations scholars eventually figure this out. Editors, I hope, learn it pretty quickly. Yet we do not talk enough about some of the troubling issues with the peer-review process. This contributes to a lot of the misunderstandings about journal publishing that circulate in the field. So here are four kludgey aspects of peer review that I think matter a lot for how we think about decision letters.

First, the overwhelming majority of submissions could, with sufficient revisions, become suitable for publication in a prestigious journal. The problem is that most of the better journals get far more submissions than they can publish, so editors have to adopt triage rules. Editors usually do actually mean it, for example, when they say that they decided to reject a piece because it requires very substantial revisions.

Second, the peer-review process isn’t very good at differentiating among good manuscripts. Imagine that we could assign manuscripts an “objective” score from 0-100. The peer-review process seldom tells us whether we’re looking, say, at a 70 or an 85. A 70 may get more favorable reviews than an 85. In a consequence, there’s a lot of subjective judgment and guesswork.

Some of that judgment concerns (for better or worse) what Jusin Esarely calls “the aesthetics of peer review.” Editors have to rely on a number of often idiosyncratic considerations to determine whether the ‘game is worth the candle’ in ambiguous cases. The best editorial teams can do is to try to create consistent standards.

Unfortunately, journals often—in part because they’re now managed by larger and larger teams—don’t establish sufficient consistency. For example, it’s not unheard of for an article to receive a negative review from a referee who objects, in principle, to survey experiments. That’s fine. But if a journal publishes survey experiments, the editors need to be very careful about how they evaluate that report.

Third, the nominal recommendations of referees—’reject,’ ‘revise and resubmit’, and so on—provide less information than the content of their reports. This shouldn’t be all that surprising. Editors make comparative judgements based on a) their overall pool of submissions and b) the accumulated reports that they’ve seen. Referees, on the other hand, know very little about the overall character of submissions; they have a much less informed sense about whether a particular substantive assessment justifies a specific recommendation for a given journal.

Fourth, editors learn to live with some inevitable rate of error. When they receive hundreds of manuscripts, even a very low error rate will produce some real dogs. From an editorial perspective, a low error rate—understood in terms of bad decisions, poorly worded decision letters, dud referees, or whatever—means that you’re doing a pretty good job.

However, a typical author will submit no more than a handful of manuscripts to a journal over the tenure of a particular editorial team. Authors don’t care about the overall rate of error. What matters to them, and rightly so, is how their manuscript gets treated. Editorial procedures can mitigate this disjuncture, but they can’t resolve it.

Everyone Deserves Respect

Given the stakes, receiving a decision letter is usually a fraught experience. Other scholars have passed judgment on your work. You are now reading their assessments. Sometimes those assessments are pretty negative. When we stir in the anonymity of review and the inherent power dynamics between editors and authors, we wind up with having to navigate metaphorical minefields.

Even well-intentioned referees can come across as jerks. Unfortunately, not all referees are well-intentioned. I think we’ve all experienced shabby treatment by reviewers. Many of us have been on the receiving end of genuinely nasty reports.

For their part, editors see such a large number of referee reports—including some really outrageously unprofessional ones—that they sometimes wind up raising the bar too high for what they’ll actually “call out” in a decision letter. Editors also try to make lemons out of lemonades with low-quality or unprofessional reports, and wind up sending mixed signals. I’m certainly guilty of this; it’s easier to rely on subtle signals that authors and referees may not notice, such as barely mentioning, or not referring at all, to a specific report.

Thus, it’s extremely important that decision letters maintain a professional and respectful tone. Decision letters frame and integrate the information conveyed in the reports. They serve as an author’s point of entry into the overall review package; decision letters should serve as a kind of buffer between authors and referees, especially those whose better natures fall victim to the license provided by anonymous feedback.

It isn’t always easy for editors to strike the right tone in their letters. Sometimes editors find themselves annoyed or frustrated with the paper. Sometimes they’re just in a really bad mood. But they absolutely need to do their best. Editors should make it part of their job to ensure that the peer-review experience is at least a bit less unpleasant for authors. Obviously, editors will inevitably fail. I know that I did. And, of course, authors can react badly no matter how much effort you put into crafting their letters.

The RFD

The most important part of the decision letter is its reason for decision. Editors need to communicate clearly and effectively why the decided, for example, to decline a submission, conditionally accept or accept it, or invite a revision and resubmission.

In my view, all decision letters for peer-reviewed manuscripts should disclose the nominal recommendation of the referees. It definitely makes life easier for editors if they can obfuscate the fact that the piece they just rejected had three revise-and-resubmit recommendations. As I noted above, those nominal recommendations are less consequential than the content of reports. But it improves the quality of the RFD if the editors have to explain why they sided with or overruled those recommendations. In general, decision letters should be as transparent as possible.

The RFD has two components: the bottom-line assessment and the warrants that justify that assessment.

The Bottom-line Explanation

This is the tl;dr or the meta-rationale: ‘given the issues that we lay out below, we have to decided to….’ They include such greatest hits as:

While the referees and editors see merit in the paper, the breadth and depth of revisions required exceed what we can allow for in a revise-and-resubmit decision.

And:

Both the referees and the editors are satisfied with the revisions made to the paper. We are therefore pleased to accept your manuscript for publication.

And:

The reviewers recommend that we offer you the opportunity to revise and resubmit the manuscript for further review. We agree with their assessment.

These often come across as boilerplate. That’s because they are. Most editorial teams have templates for each kind of decision letter. While they may tailor some of the language, they seldom depart too far from their scripts.

As I discussed above, this doesn’t mean that such boilerplate is insincere. Editors actually mean it. They make decisions by figuring out which (clichéd) tl;dr basket a submission fits into.

The Warrants

The RFD should contain clear explanations of the factors that contributed to the bottom-line assessment. Decision letters can almost never cover all of the reasons for the outcome, but they should cover the most important ones.

I’m a fan of putting these warrants, or at least summaries of them, in list form. It’s not always possible to do so. But at ISQ the vast majority of our decision letters made use of unordered lists.

Why do I prefer lists?

• They’re easier for everyone to digest. You want to know why the paper’s been rejected? Here are the two or four main issues that, either individually or in combination, pushed the paper out of the range of a revise-and-resubmit invitation.
• They force editors to be more precise and disciplined in their reasoning. In most cases, readers don’t need to go on an intellectual journey with the editor. They just need to know the destination.

In contrast, the more the letter rambles:

• The more likely it is that authors (and referees) will misinterpret it.
• The harder it becomes to distinguish the really important stuff from the more trivial concerns.
• The greater the chance that the letter will inadvertently develop a problematic tone.

Regardless of whether or not a decision letter uses a formal list, it needs to communicate a clear hierarchy of warrants, especially when it comes to submissions sent out for peer review. I’ll talk about this more in Part II. Among other things, this is where revise-and-resubmit letters sometimes go wrong. For now, I’ll note that I tend to think about three main categories of warrants.

• Dispositive issues involve ‘make or break’ concerns. In revise-and-resubmit decisions, these issues must be successfully addressed via revisions. In some cases, the author can successfully push back against them in the revision memo, but this amounts to a high-risk strategy. In rejections, these are the reasons the journal did not move forward with the manuscript.
• Secondary issues matter but aren’t dispositive. In the context of a “revise and resubmit,” it would probably be a good idea to address any given secondary issue via revisions, but it isn’t essential. Pushing back against them in the memo is comparatively lower risk. In the context of rejections, secondary warrants should be filed under “issues that are worth thinking seriously about.”
• Tertiary issues are optional. These are much less likely to appear in rejection letters; when they do, it means that one of the editors thought of them as small things that authors might benefit from being aware of. If they appear in the context of a “revise and resubmit,” the editors note them because they think addressing them would marginally improve the paper.

There is also an implicit fourth category: issues raised by the reviewers that the editors don’t think matter at all. The editors may even actively disagree with some of the concerns raised by one or more referees.

My view on how to handle these non-warrants has been shaped by both successes and failures during my time at ISQ. I used to think that omitting something was sufficient to signal to an author that the editors don’t consider it important. I learned that this isn’t always the case. One reason is that decision letters don’t always provide authors with much guidance. This makes authors understandably wary of assuming that they don’t have to deal with an issue because the editors failed to flag it.

As I foreshadowed above, when crafting the warrants in a decision letter, editors should keep in mind that everyone deserves respect. No matter how editors structure their letters, it’s always important that they choose their adjectives and adverbs with care, find ways of being encouraging even in negative letters, and otherwise treat the author as a respected peer and colleague. Sometimes this involves phrasing an issue less as a failure on the part of the manuscript than as something it needs to do better. I’ll return to these considerations in Part II, because they don’t always play out the same way across different kinds of decision letters.

Regardless of the specifics, I do believe that organizing decision letters analytically—either by drafting them according to a structured template or by taking the time to redraft initial thoughts into cleanly structured prose—generally makes for better decision letters across all of these domains.

A Final Note: It’s About the Paper, Not the Author

I strongly encouraged my team to make the manuscripts the focus of the letters. I sometimes took this to extremes, such as trying very hard to avoid referring to “you” except in the context of instructions directed to authors, or praise.

This practice isn’t simply about a mostly pyrrhic effort to depersonalize decisions for authors. It also accords with the reality of how many editors see the relationship between authors and their submissions.

What do I mean? Authors who receive a rejection from a journal are sometimes puzzled—or even annoyed—when an editor from that journal asks them to review a piece. It’s true that editors are desperate for referees. It’s also true that when an author submits a manuscript, they get on the editors’ radar

But more important is that editors don’t usually see the outcome of submissions as indicating very much about the scholarly merits of authors. Good scholars submit papers that don’t get published. And set aside the vagaries of peer review for a moment: excellent scholars sometimes submit papers that just aren’t very good.

So there are a number of reasons to make the paper, as much as possible, the focus of the RFD.

In the next post, I’ll talk about different kinds of letters. Before I end this, I should probably stress that I have strong views on a lot of these matters, but I recognize that there’s plenty of room for reasonable disagreement.

In fact, editing is a monster of a job, especially when added to other academic responsibilities. It’s also more often thankless than rewarding. Editors make choices that reflect their circumstances and resources. Indeed, toward the end of my editorship I got pretty judgmental about editorial practices. This ended when another editor read me the riot act. I deserved it.