Go to publons.com

Why saying 'no' also matters in peer review

Publons' global Peer Review Awards are creeping up fast! To celebrate their impending arrival (announced on September 12th during #PeerReviewWeek18), and to give you insight into what it takes to win a top spot, we interviewed some of the winners from our 2017 edition.

Up this week, we speak with Developmental Psychologist Sue Fletcher-Watson. Sue was the top peer reviewer for her institution in 2017, the University of Edinburgh, and is also the top peer reviewer for SAGE's journal, Autism, on Publons.

Sue offers a range of advice in our interview, including why saying 'no' can be as important as saying 'yes,' how young academics can break into reviewer pools, and why we should look outside the research community when thinking about diversity in peer review...

Publons: Can you start off by telling us a bit about yourself and your research?

Sue: I’m a Developmental Psychologist. I work in the Patrick Wild Centre at the University of Edinburgh and my research focuses on trajectories of child development, especially those which follow a path that deviates from the norm - as in the case of autism, and babies born early.

What do you think it takes to be a top reviewer in your field?

I think we should be aiming for quality rather than quantity in reviews. So many requests come in, and often the papers seem very interesting and important at first glance. It is easy to say yes to too much and the problem there is that paper reviews end up being done in a rush. I try to limit the number of reviews I accept (maybe 2 per month) but even so I do nearly all my paper reviewing on trains and planes, or in the evening, or in the cafe by the park where my kids are playing. If you’re rushing you can end up writing an unhelpful review that misses some of the detail in the justification for the study or the complexity of the analysis. So I guess I’m saying being a top reviewer means doing fewer reviews. One way to enact this is to focus on reviewing for a smaller number of journals but, while it can help you say No, ultimately if there’s an interesting-looking paper and you feel you have the expertise to help make it even better, you’re going to want to accept.

“If you’re rushing you can end up writing an unhelpful review that misses some of the detail in the justification for the study or the complexity of the analysis.”

What would the world look like without peer review?

I dread to think. Pseudoscience really plagues the autism community, and peer review is one of the few bastions we have to defend ourselves against exploitative and scare-mongering quackery. However, even with peer review, we do have a lot of poor quality science slipping through the net - or perhaps I mean non-representative science. It remains easier to publish significant results than null findings, not least because people who get a null finding are quick to blame themselves for failing to “properly” conduct the experiment. I think endeavours like pre-registration and registered reports are going to clamp down on this - and peer review is a key component of these too.

“Peer review is one of the few bastions we have to defend ourselves against exploitative and scare-mongering quackery.”

Do you think research would be better or worse off if a more diverse range of people were asked to review papers? Why?

Definitely better off. I work closely with people in the autistic community in my research and I feel that a lot of research in that field would be improved if autistic people, family members of autistic children and practitioners from relevant services - education, health and social care - had a chance to comment on the work we do as academics. It would be important to make a distinction regarding the purpose and remit of reviews from within and outside academia I think, but very valuable if this could be achieved.

How do you think this might change or improve the peer review landscape?

Inviting community representatives to get involved in peer review would have a few benefits I think. One is that it would trigger a conversation about the purpose of review. Journals might have to be more explicit about the function of different reviews being sought from different people. For example you might invite an “impact review” from a practitioner focusing on the relevance of the research to a practice setting like a hospital or school. A “respect review’ or “personal perspective” might be sought from a person from the same population as the one being studied - like a cancer patient for an report on a trial of a new drug treatment.

“Inviting community representatives to get involved in peer review would have a few benefits I think.”

A second advantage, of course, is that it would encourage academic writers to consider these perspectives more deeply when designing their studies and reporting them. I think a third advantage might be the opportunity to publish these reviews alongside the final article. They would provide valuable contextual information for the original research and help to make academic output more relevant to people’s lives. Of course a major barrier would be recruitment and training of the non-academic peer reviewers, and I doubt we’d ever be able to build a big community who would do the reviews for free. It’s really extraordinary that we have managed to create a system where academics do this as a matter of course, despite minimal return on their time and expertise!

What advice do you have for early career researchers who are reviewing?

Start by co-reviewing with someone more experienced and who you trust to give a scientifically-robust, but kind, review. Remember the work that goes into every new piece of science and be gentle in your wording and tone, while also holding people to a high standard. Judge the paper on what they set out to do, and did do, not on what you might have done instead. You don’t have to sign your reviews but imagine you were going to and check that you stand by everything you wrote.

“Judge the paper on what they set out to do, and did do, not on what you might have done instead.”

How do you suggest getting more researchers at your institution involved in peer review?

I think a key thing is to get people involved in peer review from earlier in their careers. Now that I have published a fair amount and am known to a few editors, I get a lot more review requests but I have less time to do them - and stand to gain less from the experience than I did ten years ago (though still a big reason to do peer-review for me is the opportunity to learn from a detailed look at what others have done - especially in the methods and stats).

“I work with a growing team of research associates, PhD students, and post-docs all of whom are more than capable of providing effective, constructive peer review if they are given the right training and support early on.”

I work with a growing team of research associates, PhD students, and post-docs all of whom are more than capable of providing effective, constructive peer review if they are given the right training and support early on. Reviewing will improve their own study design, analysis and reporting too. In particular, the opportunity to review grant proposals is invaluable for people just starting to writing their own grants. However at these early career stages, folk don’t get many requests. We need to find more effective ways of getting newbie reviewers support to learn the trade (like the Publons academy) and invitations to review.

Thanks, Sue!

Will you be named one of our 2018 Sentinels of Science?

To give yourself the best shot at becoming a Sentinel of Science, make sure you upload all your reviews to Publons by September 1st (winners will be announced on the 12th). Stay tuned for updates by following our Awards page, our blog, and our #SentinelsofScience hashtag on Twitter.

Interested in reading more?

comments powered by Disqus

Subscribe to our mailing list

* indicates required