Publons' Peer Review Awards are announced each year in September during Peer Review Week. To give you insight into what it takes to win a top spot, we interviewed some of the winners from our 2017 edition.
Up this week, we speak with the winner of our inaugural Sentinel Award, Irene Hames. Irene has been a tireless advocate for high-quality scholarly peer review and has generally helped to define what robust review looks like today. It's for that reason she won our Sentinel Award - for outstanding advocacy, innovation or contribution to scholarly peer review.
Irene gained her reputation as an independent research-publication, peer-review and research-integrity specialist with a PhD in cell biology (learn more here). She has now retired after nearly 40 years in scholarly publishing, making us all the more grateful that she's offered the following words of wisdom about peer review diversity and more for researchers new and experienced...
Publons: Why do you think the peer review process is important?
Irene: In scholarly communication/publication, peer review is about getting appropriate people with the relevant expertise to assess and offer opinions on research outputs, most commonly research articles. If it’s been done well, peer review helps us know what we can trust and what we need to treat with caution. But, unfortunately, the quality of peer review is very variable, and we’re seeing a worrying number of real failures of peer review. Many editors are also reporting that it’s getting harder and harder to find reviewers, so the system is getting more and more stretched. A lot of innovation and experimentation is underway to try to find ways to improve things.
“If it’s been done well, peer review helps us know what we can trust and what we need to treat with caution. But, unfortunately, the quality of peer review is very variable, and we’re seeing a worrying number of real failures of peer review.”
To me, the most important role of peer review is to provide an assessment of whether a piece of work is methodologically sound, whether it has been carried out sufficiently well, and whether the results have been reported and interpreted appropriately. Research builds on previous work and if that’s faulty then subsequent work may be too, resulting in a waste of resources, human and monetary. Another important role is to provide constructive feedback to the authors on how problems with the work or the way it’s been reported can be addressed, and how the manuscript can be improved. But I have concerns that the current pressures on the global peer-review effort may be affecting the extent to which that’s being done.
What makes a good quality review and why?
Basically, one that is helpful to both the editor and the authors. Editors may ask reviewers specific questions or to focus on certain issues, and those requests should generally be complied with. But otherwise, to anyone carrying out a review I’d say: be objective, constructive and avoid making derogatory personal comments (remember you’re reviewing the work, not the author/s); mention any parts of the manuscript you haven’t been able to assess, and why; write clearly and concisely, with numbered points to help navigation and responses; don’t make sweeping statements without providing specific examples, e.g. if you’re reporting that the meaning isn’t clear in places (where?), or that some interpretations are faulty (which?), or that similar work has been done by others (by whom?, provide references); and always treat the authors’ manuscript and its review as you would like your own to be treated.
How can we encourage diversity and inclusion in peer review?
Ideally, the composition of the global reviewer and editor pools should reflect that of the global research community and research disciplines, but this isn’t the case. On the positive side, many people and organisations are looking at ways to embrace diversity and increase inclusion.
Last year I came across a post by Meghan Duffy on the Dynamic Ecology blog describing a very simple strategy to help increase reviewer diversity – namely, to use ‘recognition, not recall’ when choosing reviewers. The first names that spring to mind when choosing potential peer reviewers may not be the best ones, probably reflecting both a lack of awareness and our inherent biases. By thinking about it and consulting resources (perhaps building up your own), it’s likely that more suitable and more diverse individuals will be identified. An area where resources are being created is gender. Gender disparity in peer review is currently a very prominent problem and some great lists and directories are being developed to highlight the work and expertise of women in various disciplines. These can be used to identify suitably qualified women peer reviewers as well as potential conference speakers, authors and subject experts.
“Gender disparity in peer review is currently a very prominent problem and some great lists and directories are being developed to highlight the work and expertise of women in various disciplines.”
Can you share any tips for early-career researchers keen to break into the world of peer review?
The first place to start is to ask your supervisor if they could, when they are invited to review manuscripts, ask the journals if you can co-review these with them. Many will be happy to agree, and many actually encourage this, as it’s the traditional way to learn how to review. Asking for permission before co-review is important for two reasons: firstly, so the journal can check for any potential conflicts that might exclude you from reviewing, and, secondly, so the journal knows you’ve been involved with the manuscript review and you can receive due credit for that (and the opportunity is created for you to be invited to review direct another time).
For guidance on the ethical aspects of being a reviewer, I recommend early-career researchers take a look at the COPE Ethical Guidelines for Peer Reviewers. Supervisors may also find these helpful, especially as a resource to help them with mentorship of junior researchers.
Many societies, institutions and organisations are introducing peer-review training schemes. Check these out and sign up for the ones that are most interesting or relevant to you. Some of the courses are very popular, so also sign up for alerts about future workshops.
There is a now an innovative new way to gain experience of reviewing - preprint journal clubs with online posting of the discussion. One such platform is PREreview, which also has a step-by-step guide on how to create your own preprint journal club. One group of postgraduates and early-stage researchers who posted their journal-club discussion on the preprint server bioRxiv found it to be a positive experience, and the format a suitable one to introduce students to reviewing.
“Don’t be afraid to approach and engage with the editors and publishing staff, and find out about reviewing opportunities and how to get involved in peer review.”
Finally, there is the personal approach. Many journals, societies and publishers have stands at conferences, and often they’ll hold ‘meet the editor’ sessions. Don’t be afraid to approach and engage with the editors and publishing staff, and find out about reviewing opportunities and how to get involved in peer review. In my years as an editor at conferences I loved meeting enthusiastic young researchers and talking with them about peer review and research publication. Some went on to become some of our most valued reviewers!