The boon and bane of our academic enterprise is that we get feedback all the time on our work. Our work is better for it–that the hack-iest stuff I read is always stuff that is not submitted to any kind of refereeing process and relies instead on editors who seem to be blind to the hack-ness. The bane is that, well, rejection and criticism can not only delay publication but also hurt feelings. When well done, reviews further the enterprise. However, sometimes, reviews seem to make the authors dance in relatively unproductive ways. There have been lots of tweets and posts complaining about robustness checks–that reviewers have been asking authors to submit dozens (sometimes hundreds) of additional analyses.
My grievance du jour is something else–reviews that focus on stuff that one “should have cited.”
I am thinking of this because I got teased today after I griped on twitter:
The joy of reviews: so far of the lit I have had to consult, one super helpful piece, the rest not so much.
— Stephen Saideman (@smsaideman) October 18, 2014
@smsaideman name names!!!
— Philippe Lagassé (@pmlagasse) October 18, 2014
@smsaideman Heh. Wasn’t looking for a plug. Was just curious of what meets the Saideman helpfulness standard.
— Philippe Lagassé (@pmlagasse) October 18, 2014
What is the Saideman Helpfulness Standard©? Well, this specific discussion was about the citations a reviewer asked me to include in my work. I was highly annoyed because this meant I had to spend time reading a bunch of stuff that did not change my outlook or add new information. I did find thus far one citation that was useful, but the rest were not (still working on the piece so the ratio may change).*
So, the SHS is this:
- does the recommended citation compel the author to better defend the originality of the argument? That is, if this argument has been made before, the author should justify why it is worth making again.
- does the recommended citation provide an argument that must be addressed?
- does the recommended citation provide information that the piece (article/book) could use to strengthen the argument?
- does the recommended citation provide information that challenges the arguments of the article/book?
- a recommended citation that provides information or arguments that the author has already addressed,
- a recommended citation that is a slightly different version of a piece that is already cited. For instance, saying that one should cite author x’s 2011 piece even though the work cites author x’s 2012 piece presents mostly the same argument and evidence. Why add more cites if the basic argument/info is already included in stuff that is already cited?
What is the SSUS (Saideman Super Unhelpful Standard©)?
- Citations of unpublished works. Yes, the internet is a miracle, but there are limits to one what can do when writing an article. Unless the unpublished work is well known (Stephen Van Evera’s dissertation was perhaps the most widely cited piece… until he published his book), reviewers probably cannot expect authors to know about unpublished stuff.
- Citations of works that come out after the work under review has been submitted for review. Until we perfect time travel, this is what we call a party foul. It is perfectly fine for a reviewer to suggest how new work could improve the piece under review, but cannot expect that omitting this newest stuff should be grounds for rejecting the manuscript.
- Citations of the reviewer’s work. Yes, I have sometimes criticized a piece I was reviewing for not citing me, but I always feel awkward about it and only do it when the author(s) are ignoring not just me but a key argument that they should be addressing. And, of course, when a reviewer asks an author to cite the reviewer’s work even though it is unpublished, that is a double-secret party foul.
I am sure I am forgetting stuff that fits into these three categories (is there a fourth category?), so let me know what I am missing, including stuff I should have cited.
* Of course, I am writing this blog post in part because it is more fun than reading stuff that someone told me I had to read.
Steve, what is your view on reviewers suggesting citations of junior scholars and women purely for the goal of increasing diversity / reducing gender imbalance in citations?