Peer review getting you down? Have you been asked to review and don't know where to start? Or have you tentatively submitted a few reviews and never been quite sure if you're doing it "right"?
That's okay: there are tips you can learn to make the process easier and more robust--for you, the authors, and the editor.
Last month we shared five peer review tips and the story behind them from a Publons Academy graduate, Edmond Sanganyado. This month, our reviewing tips come from Publons Academy supervisor, Dr. Paul Wong.
Paul is the Professor Emeritus of Trent University and Adjunct Professor at Saybrook University. He is the Editor of the International Journal of Existential Psychology and Psychotherapy, and recently supported PhD candidate, Lilian Jans-Beken through the Publons Academy.
See what Paul believes makes an excellent reviewer, and if you're interested in improving your writing and learning the core competencies of peer review, check out our free and on-demand Publons Academy.
1. Have the Necessary Expertise
Do not review manuscripts beyond your scope of competency. Journals are more likely to publish papers with serious errors or deficiencies if their manuscripts have not gone through rigorous review by reviewers with the necessary expertise. Such problems may even occur in American Psychologist, the flagship of the American Psychological Association. Here are two cases: one involving the critical positivity ratio (Brown, Sokal, & Friedman, 2013, 2014) and another related to the existential issue of meaning in life (Brown & Wong, 2015). How these critical reviews ever got published are a story worth telling, because they could have been buried without vigorous protest against review bias.
2. Show Some Respect
Reviewers should show some respect by at least reading over each manuscript. Some manuscripts represent the cultivation of several years of research and deserve a couple of hours of reviewers’ time. I have personally seen reviewers rejecting manuscripts simply by reading the first paragraph. One of my own papers, the Death Attitude Profile (Gesser, Wong, & Reker, 1987-1988) was rejected because “there are already too many instruments measuring death anxiety,” even though this was the first paper providing a comprehensive measure of three kinds of death acceptance. Clearly, neither the reviewer nor the editor had taken time to read my paper. That is why I urge all my mentees to summarize what each manuscript is all about, thus indicating that they have read and understood the content of the manuscript.
3. Be Fair to All Manuscripts
It is difficult to be fair in your treatment of all manuscripts, because we all have our likes and dislikes, and no one is above biases. In order to guard against any review bias, one should openly acknowledge and suspend one’s bias for or against certain research at the beginning of a review. Such “bracketing” practice should improve objectivity in judgment. In this regard, one should also refrain from reviewing a manuscript, if there is a clear conflict of interest which may jeopardize one’s critical judgment, such as being a collaborator in some prior research.
Secondly, one should apply the same criterion to evaluate all papers for the higher purpose of safeguarding the integrity of science. One should especially guard against “halo effects;” one should not let the reputation of the author or the prestige of the author’s affiliation dull one’s ability for critical thinking and vigorous assessment.
Photo courtesy of Dr. Paul Wong (above).
4. Be Constructive
It is very difficult to be constructive, especially when the editor instructs reviewers to be very strict because the journal has a very high rejection rate, sometimes as high as 80%. Under such circumstances, reviewers automatically focus on “fault finding” in order to support a recommendation of “rejection.” Such negativity bias is not a healthy practice, because it tends to favor the established elite researchers and discriminate against beginners who do not have a track record in publications. I propose that a reviewer should do the honest job of taking a constructive approach to review and let the editor tackle the problem of maintaining a high rejection rate.
Being constructive means that one looks for both the strengths and weaknesses of each manuscript. Even in the case of recommending rejection because of fatal flaws, provide logical and empirical justification for the criticism so that the authors can improve their research. From this perspective, it takes much less time to accept a manuscript than to reject one, as illustrated in this review, “How to Measure Existential Meaning” (Wong, 2017).
5. Be Inclusive and Balanced
Most authors suffer from the bias of “tribalism.” This practice is revealed in several ways. The most common form of tribalism is to ignore the findings or theories that may question their pet views. The second common practice is that within any domain of research, they only cite the work of their circle of associates or friends, and ignore “outsiders,” even when the most cited work comes from the latter.
Such intentional citation amnesia is neither scholarly nor ethical, because it ignores important publications that are germane to the topic under investigation. A knowledgeable reviewer should be able to call out such omissions, so that the manuscript can become more inclusive and balanced in its literature review and discussion.
6. Do Not be Afraid of Reprisal
Many reviewers, especially young researchers, do not want to offend “big names” because of fear of reprisal. A renowned psychologist once said to me, “I will make sure that this guy will never have another publication,” after bitterly complaining about a paper critical of his theory. I myself have suffered the consequence of daring to write a critical review of a book authored by a prominent psychologist. Over the years, I have witnessed editors who refused to publish any paper critical of an influential theory or author.
The blind review system provides some cover to a reviewer’s anonymity, but not the editor’s identity. In order to protect the integrity of the peer review, both editors and reviewers need to have the courage to do the right thing for the sake of justice and science, even if it involves the possibility of reprisal.
Ideally, a good reviewer serves two functions. Firstly, one serves the professional function of screening out what is not up to standard and improving what is acceptable. Secondly, one serves the humanistic function of contributing to fairness or justice in the distribution of research funding and publication space.
Being an established author does not automatically make one a good editor or peer reviewer; training is needed to develop competent and objective reviewers. I am so pleased the Publons Academy has taken up the challenge to train master reviewers.
Paul's tips have been republished from his website.
If you’re interested in learning more about peer review sign up for our Publons Academy. This is a free on-demand course that teaches you how to master the core competencies of peer reviewing, and to connect with editors at elite journals.
Brown, N. J. L., Sokal, A. D., & Friedman, H. L. (2013). The complex dynamics of wishful thinking: The critical positivity ratio. American Psychologist, 68(9), 801-813. doi:10.1037/a0032850
Brown, N. J. L., Sokal, A. D., & Friedman, H. L. (2014). Positive psychology and romantic scientism. American Psychologist, 69(6), 636-637. doi:10.1037/a0037390
Brown, N. J. L., & Wong, P. T. P. (2015). Questionable measures are pretty meaningless. American Psychologist, 70(6), 571-573. doi:10.1037/a0039308
Gesser, G., Wong, P. T. P., & Reker, G. T. (1987-88). Death attitudes across the life span. The development and validation of the Death Attitude Profile (DAP). Omega, 2, 113-128.
Wong, P. T. P. (2017). How to measure existential meaning. Paul T. P. Wong. Retrieved from http://www.drpaulwong.com/how-to-measure-existential-meaning/
Wong, P. T. P. (2017). How to write a good manuscript review. Dr. Paul T. P. Wong. Retrieved from http://www.drpaulwong.com/how-to-write-a-good-manuscript-review