It’s happened to all of us (or least those of us who do quantitative work). You get back a manuscript from a journal and it’s an R&R. Your excitement quickly fades when you start reading the comments. One reviewer gives a grocery list of additional tests they’d like to see: alternate control variables, different estimators, excluded observations. Another complains about the long list of robustness checks already in the manuscript, as it obscures the important findings. Sometimes both of these reviewers are the same person.

And it gets even more complicated if the article ends up rejected and you send it to another journal. Now that list of robustness checks–some of which were of questionable value–expands under a new set of reviewers’ comments. And those reviewers irritated by voluminous appendices get even more annoyed by all the tests included with little clear justification (“another reviewer told me to add this” not being an acceptable footnote).

Continue reading