This post continues a series of guest pieces and interviews by experienced peer reviewers on advice and insights for early-career reviewers.
Jim Cotter is an exercise and environmental physiologist and Associate Professor at the University of Otago in the School of Physical Education. He has supervised and co-supervised over 20 PhD research students and was recently voted the OUSA Supervisor of the year by postgraduate students. Jim has reviewed in excess of 70 manuscripts for 12 academic journals in the last five years.
You can see Jim Cotter’s peer reviewer profile at publons.com/a/440142.
At what stage does an early career scientist become qualified to peer review?
I think it depends on both the person and their mentorship. Many journals seem to encourage research student involvement - provided that it’s disclosed – and this does seem to be a no brainer for several reasons. I would strongly encourage it.
Any tips for how these scientists might get noticed by editors for the first time?
Perhaps partly as above (i.e., formally recorded by the journal), and encourage or hope their mentors recommend them thereafter as alternative reviewers.
When should a reviewer reject a review invitation?
I wish I knew. Conflicts of interest are obvious and have clear guidelines for reviewers already. In my opinion, some conflicts are more problematic, e.g., (i) a forthcoming and competing dataset, or (ii) strongly conflicting research views or paradigms between the manuscript and the potential reviewer, as would occur if they represent opposing sides of a contentious topic. If the reviewer has any concern, such as whether they can thoroughly review / appraise all aspects of the paper, I believe it’s a good idea to disclose this to the editor. For example, if I get an invitation to review, and know that I’m not sufficiently knowledgeable in a substantive aspect of the literature or methodology on which the study is based, I do try to check with the editor whether they have those aspects catered for among the reviewers.
Do you have any rules of thumb for how many review invitations you accept per month?
Depends on workflow, but I’m a terribly slow reader and unfortunately tend to get bogged down in the details, so ideally no more than 2 for me. For early career scientists it naturally takes longer to review, so I would think that no more than 1 every two months or so would be ample. At the end of the day, what goes around should come around, i.e., it seems a bit rich if we’re happy to submit manuscripts but then not be prepared to take on a proportionate number of reviews.
Writing the review
How do you go about reviewing a manuscript?
I usually try to skim it very quickly to get a general sense of it, then sit and read it with the tables/figs and references visible as well. While reading properly, I note down major and moderate/minor points and things to check on later (e.g., what a referenced study actually showed), and directly highlight minor/specific things in text. I try to briefly summarise it for the editor (overview, contribution, strengths & weaknesses, acceptability) and perhaps its contribution/context for the authors (really just to clarify whether we view it similarly, or not), then prioritise and collate the major points and minor/specific points into feedback. A very poor manuscript – whether in the design or lack of care in presentation – requires much less time and little specific feedback.
How much time do you typically devote to reviewing a single manuscript?
Usually at least a day, but as mentioned, I read too slowly and often review in too much detail.
Open and transparent peer review
Do you ever sign your name to your reviews?
No, only because the process is usually confidential, but would be very happy to do so.
Have you ever published any of your reviews? If so, what did you think of open review?
No I haven’t done so, but I’m generally in favour of reviews being publically available, provided that readers remember that reviewers are unpaid and responding with substantial time constraints (their own time availability, and the journal ‘requirements’).
Reviewing now and in the future
Why do you review?
Mostly because it’s a shared expectation in science. Besides, it can be interesting for reasons of their data and interpretations of those data, revisiting the literature, and engaging with other people’s style of writing and presenting information
What (if anything) needs to change in peer review?
- Journals publishing for the public good, i.e., not for profit, and thus maintaining lower cost structures.
- Journals being realistic of reviewers. Given how long it takes to plan, conduct, analyse, write up and submit research as a manuscript, it seems unreasonable to expect that reviewers should then review it within 10 days or so.
- Open peer reviews would be helpful for several reasons, subject to reservations such as mentioned above.
- Clear guidelines by journals on criteria to apply when rating a manuscript, to enable more consistency across reviewers.
- Editors should uphold journal requirements/guidelines (e.g., on statistics).
Any other tips for less experienced peer reviewers out there?
Don’t be afraid to ask more senior people about something, and read guidelines on reviewing. Remember that no study is perfect, and provide feedback in a manner that you would want to receive it.