This is the first in a series of guest pieces and interviews by experienced peer reviewers on advice and insights for early-career reviewers.
Shervin Assari is a research faculty member at the Department of Psychiatry (with joint appointments at Schools of Public Health and Social Work), University of Michigan, Ann Arbor. He has reviewed for 50 academic journals and has a total of 244 reviews on Publons - so he knows how to review!
We recently asked Shervin for some tips to share with early-career reviewers.
How do you go about reviewing a manuscript?
I check the relevance, importance, accuracy, statistics, methods, and structure of the manuscript. First I look at the big picture. To give an analogy, think of looking at a painting, not being distracted by details, and only focusing on the painting as a whole. No evaluation of details like quality of introduction, discussion, or the formatting and accuracy of the references. In this step, I evaluate the importance and accuracy of the research question, and if methodological and statistical approaches are appropriate. By this time, I have spent only 30 minutes and the result is rejection in some cases (in the presence of fatal errors). If the manuscript passes this early and general screening, I begin looking at details. In my opinion the introduction and discussion are considered details, while methods and results are not.
How much time do you typically devote to reviewing a single manuscript?
It really depends. On average, I can spend anywhere between 3 to 6 hours; however, I should add I spend no more than 30 minutes for 10-20% of manuscripts that I review. They are those I quickly reject due to fatal flaws. For 20%, I may spend more than the average, as I think multiple domains require help in which the authors need guidance and direction. In these cases, a reviewer acts as a tutor and tries to help, instead of easily rejecting the paper. In addition, at times, I do not even evaluate the quality of the discussion or results, because the author should redo the analysis which may change everything! So, I simply ask author to send a revised version, and it is in the second round of review that I evaluate the other sections of the paper. I see it as meaningless to review the discussion of a paper which has used inappropriate analysis.
Any tips for how an early-career scientist can get noticed by editors for the first time?
In my opinion, the main first step to become a reviewer is to become the first author. Requesting editors that you know to send you papers for review may also be a good first step.
When should a reviewer reject a review invitation?
Many cases. Conflict of interest (same department, collaboration, direct benefit, etc). Or when the reviewer is not qualified. Or the person feels he/she can not perform an unbiased review, for instance, because do not believe in the findings etc.
Do you ever sign your name to your reviews?
Yes, but rarely.
Most journals still do the double blind review process, many do single blind, and fewer do open review. Some journals give this option to the reviewer. There are journals that give this option, but they are the exception. If the paper is accepted, journals being published by Frontiers list the name of the reviewers on the paper. PLOS One also gives the option of signing the review. But again, it is not very common. There are also a few journals (for instance those published by Springer) that will ask the reviewer if they may transfer the review, and disclose the reviewer names, in case the paper is transitioned to another journal (within the publisher).
Finally, as reviewers, we need to get credit for the reviews we do, and what is more important is acknowledgment of our services. It is less important to us to see our name associated with the exact papers that each of us has reviewed. I am not sure but if reviewers were to sign the review, it may help in improving the quality of peer review.
Any other tips for less experienced peer reviewers out there?
Do not over- or under-estimate the level of expertise that you as a reviewer have. Do not read the discussion of a paper when its method is wrong. Try to be supportive, and guide the author if the paper's problems are not fatal. Do not reject good papers from authors whose first language is not English, just because of grammatical errors.
Be open to reviewing a paper more than once. Do not spend more than enough time on a review. Do not write a review report comparable to the size of the paper itself. Do not reject papers simply because their quality is less than perfect, as there is no perfect paper. Get to know the editors of your field. Review ethics should be your first consideration. Disclose any potential conflict of interest.
Finally, you may still be qualified to do the peer review if you are only an expert of the methods or substantive area, but not both. Remember there are two other reviewers evaluating the work. Do not wait to become the best researcher of the field before accepting review invitations.