What they do
The most important thing to remember about the task of reviewing is the scholastic camaraderie that lies at the heart of process. Due to the anonymous nature of the exchange, that process can often seem “detached,” yet it need not be. Recall the last time you received comments back, whether positive or negative, on your own work. In the best cases, those comments can be enormously helpful and productive, and in the worst, they can feel like destructive insults. That doesn’t mean that we should sacrifice honesty in our feedback for fear of hurting an author’s feelings. It does mean that we should make the assumption that our words carry great weight and mean a lot to the authors who read them. The review itself can serve as an effective and useful teaching moment, and that is just as true for submissions that are enthusiastically accepted as it is for those that are vehemently rejected.

That said, such a lofty purpose does not require a lofty tone and style. For instance, though many reviewers may be more comfortable referring to “the author” and maintaining a third-person distance, such a convention is not a rule. Since your review is after all a more or less direct communication between you and the author, an I-you rapport can be a very effective and efficient means of explaining and supporting your evaluation. The larger point, though, is to be true to your own writing style and voice in your dialog.

Finally, as many of us know from grading our students’ papers, it is impossible to comment on everything. Using “track changes” can actually increase the temptation to jump in at every moment. Providing too much feedback can be as much of a problem as not giving enough. A useful rule of thumb is to use your comments within the text to address “local” issues and reserve the comments at the end for a more “global” summation.

My reviewer name is ZED. Those aren’t my initials. I just enjoy being ZED for AEQ, and I have been a reviewer here for over three years. One convention that I have adopted in all my reviews is to end with the statement: “I hope these comments are helpful.” Whether I am saying yes, no, or something in between, it is well meant; I also hope that these tips and guidelines are helpful to you reviewers.

Read more about Academic Exchange Quarterly Reviewers


Why they remain anonymous
Anonymity protects reviewers and ensure they can be candid without fearing the consequences.
AEQ decision is modeled on NSF peer-review policy.
         In a decision announced May 24, the DC Circuit Court of Appeals upheld the right of the National Science Foundation (NSF)
         to withhold the names of individuals who review specific grant and award proposals during the agency's peer-review process.
         The Circuit Court's unanimous decision upheld a previous decision in federal district court. Contending that its peer-review policies
         fall under the federal Privacy Act—which protects the identity of people who review government contracts—NSF argued that the
         identities of reviewers ought to remain confidential in order to safeguard the grant process and to protect reviewers from lobbying pressure,
         harassment, or retaliation. NSF grant and award applicants routinely receive the verbatim evaluations of their proposals, but not the names
         of the reviewers. NSF believes this policy promotes candor in evaluations and allows reviewers to provide direct and constructive feedback
         to applicants. "We are pleased the Court has approved NSF's long-standing practice of confidential peer review," said D. Matthew Powell,
         assistant general counsel for NSF. "The importance of reviewer confidentiality in obtaining the candid evaluations of thousands of voluntary
         reviewers is well-recognized, and now endorsed by the courts."—Compiled by Michael Carlowicz. Eos, Vol. 77, No. 24, June 11, 1996 p.225
         Confidential decision [This page may be freely copied.] http://onlinelibrary.wiley.com/doi/10.1029/EO077i024p00225-04/pdf

Read a similar approach: Improving Publication Quality by Reducing Bias with Double-Blind Reviewing...
...Double-blind reviewing improves the quality of decision making by
increasing the focus of the evaluation process on the actual submission, rather than the authors.
Editorial by Kathryn S McKinley
Why extra shield to protect reviewers
AEQ practices double layer of anonymity: (1) reviews are signed by 3-letter codes and (2) the reviewer’s name is not listed in the journal unless requested otherwise by the reviewer. Why?    We like explanation in "Peer review, schmeer review" Published by odyssey on 13 August 2010 in Blog "Pondering Blather"
         If I now have to tell potential reviewers their names will be revealed to the authors, what do you think is going to happen?
         If I ask someone to review a manuscript by Professor Standing-Member-Of-The-Study-Section-Your-Grant-Goes-To, what do you
         think they're going to do? What about a manuscript by Professor Senior-Dood-Who-Has-The-Most-Influence-In-Your-Field?
         Or Professor I-Have-A-Nobel-Prize? Most will find some excuse not to accept the invitation to review
         (especially young faculty if they have any sense). Those that do may not be as unbiased as one would hope.
         Are they really going to trash a manuscript (that deserves it) by someone who has some direct control over their future,
         given that, should the manuscript actually be published the names of the reviewers will be revealed?
         Maybe, maybe not. At least reviewer anonymity provides some semblance of protection against retribution.
        

Here are five examples of authors' reaction after negative reviews, submission rejection.
Authors’ names are not listed in the first three examples (1, 2, 3).
In the remaining two (4 and 5) authors themselves list their names while making a complaint.
1 287404...   The study is original research conducted by a tenure-track professor at a Research I public university!
       While not in the habit of causing trouble, and at the risk of questioning your decision, we urge you
       to reconsider exclusion of our manuscript from the upcoming edition of AEQ... Our research is unique
       to the leadership field and promises a novel scholarly approach to examining teaching and leadership preparation.
2 192134......   I do need to address what I consider highly questionable peer review... I have been publishing
       for over 20 years... and I have NEVER encountered a case where only a BLIND version of a manuscript
       can be submitted for peer review..... Could I publish my manuscript elsewhere?
       Yes, I can, because I happen to be xxxx, and I fortunately have earned respect with other journals
       through the years for my work, and already know it will be welcomed by the xxxx. But if your editorial
       process was set up properly and coordinated better with peer review, my paper should be published
       in the upcoming special issue as it was submitted to be. No matter how imperfect my manuscript was,
       there was no excuse for such inappropriate peer review comments
3 228805......   After much thought and concern about your constant snide unhelpful and confusing advice
       on how to revise this paper, I've decided you and your condescending, hateful racist elitists reviewers
       can take this paper and your redactory fee and cramp it up whatever orifice you find more painful.....
       I have never had more issues with getting a simple narrative reviewed nor have there been so many
       passive aggressive comments made about my work. After my colleagues reviewed the paper and
       your reviewers comments, your disdain and dismissal of the issues the paper exposes and assumption
       of my inability to write is nothing short of racism and classicism.
4 Masood A. Raja, Ph.D., Assistant Professor, Department of English Kent State University.
       Editor, Pakistaniaat: < http://pakistaniaat.org>   This is not the first time
       I have worked with a refereed journal but this is the most cut-throat editorial policy I have ever encountered.
5 This 5th example of author’s dissatisfaction differs from others in that author lists several complaints instead
       of one such as “cut-throat editorial policy.”   We never met anyone as strange as this colorful fellow, known
       by different persona such as Mr. Beall a.k.a. Dr. Beall who often refers to himself as “I’m an academic crime fighter.”

August 2014