Publons, and reviewer recognition in general, offer the potential to substantially change peer review. We have seen criticisms levelled at us that we encourage superficial or excessive review. Of course we don't believe this to be the case or we wouldn't provide the service we do. But how has joining Publons changed our users' reviewing behaviour? We realised that we didn't necessarily know but that we knew people who did.
We asked some of our users if they had perceived any shift in their reviewing behaviour since joining Publons and if so, how?
Of course no one will publicly admit to establishing a review mill to game the system but these responses indicate some interesting side effects to joining Publons. If the effect of our service is to encourage more considered, open and appropriate review then we think we're on the right track.
"Since joining Publons, I have felt recognized for my peer review efforts which essentially is an honorary service we perform for science. While the rigor of my peer- review has not changed, but my outlook in general for this important role has been re-energized."Natraj Krishnan
"I think if anything has changed since joining Publons (and Academic Karma), it's that I'm more careful to review work closer to my area of expertise. When I first started I feel like I reviewed everything on offer, trying to get runs on the board. In being able to see how much I've reviewed relative to others and relative to how much I've published, I feel more confident in being more selective about what I review."Nicholas Badcock
"Just one thing changed: before Publons I used to help colleagues with reviewing manuscripts now, if a colleague asks if I could help, I suggest that they give my e-mail address to the Editor in charge that I could receive the credit for my help/comments."Ana-Maria Florea
"NO, not at all, my reviewing behavior is same as it was before Publons. 'If you ask me personally, I do not that much care about the my individual rankings in the Publons reviewer list and do not believe in racing competitions in this respect'. I am happy with the fact that now on just one-click anybody (including me) can see my scientific reviewing record (a verified record) and it is indeed an asset to my scientific career. Indeed I am thankful to Publons for initiating such a platform."Yogendra Kumar Mishra
"Since I joined Publons, I was able to see how my reviews compare to other Publons reviewers reviews, especially in terms of length and journals. This helped me to do my best to elaborate sometimes."Fares Alahdab
"A bit -- yes: (i) In some cases I've started considering disclosing my name; (ii) a graph indicated that average length of my reviews is longer than that of others -- prompting me to be more concise and 'generic' in the 'Minor comments' part; (iii) now, having sent my review, I try to take care of receiving the "thank you" email from an editor :)"Galina Paramei
"Yes, I am more likely to carry out a review now. I feel that Publons does deliver on helping me get credit for peer review. Since joining Publons I have also moved over to exclusively signing my reviews. I can see the benefits of doing this and feel that it should help to improve review quality. While getting increased Publons credit wasn't the motivation for open reviewing it is nice to get the extra points :-)"Alex Bateman
"Reviewing can be frustrating, particularly when authors ignore your comments, as it can seem to have little impact or influence. Not anymore. If a review falls in the Publons woodland, it will make a sound. This is empowering; each of my reviews now has the potential to have an impact beyond the parties involved in the pre-publication review process. My behaviour has changed accordingly. In particular, knowing that my impact can be greater acts as an incentive to accept more reviews. I always choose to make my reviews public on Publons, which I now bear in mind when writing them. I make more of an effort to communicate with clarity for a broader audience. The accountability associated with publishing a review also ensures that I maintain a friendly and professional tone; something I have on occasion found lacking when on the receiving end of peer review. I now indicate to the editors and the authors that my comments will - if the article is published - be made public on Publons. I cannot say for certain what impact this might have, but I hope it will encourage authors to give greater attention to my review in the knowledge that their implicit response in the final manuscript could be judged publicly. Publons also acts as an incentive for post-publication peer review, which is changing the way I read published articles on a daily basis. I now read articles with a mind to reviewing them. Usually this does not result in me actually writing a review, but nevertheless it encourages me to be more analytical. In this way Publons helps me to be a better researcher."Chris Sampson
"I do not think that joining Publons has affected my decision on accepting or not to review manuscripts. I do not feel more tempted to accept to review a manuscript that is not within my field of expertise just to gain more Publons merit points. In fact, because I now publically post all my reviews of papers that get published (if journal policy allows it), I feel more responsibility to accept to review only those works I am apt to review, and feel more duty to do a good, thorough and fair job at it."Rafael Santos
"I have been trying to be more conscious about the length and completeness of my review. Therefore after completing it, I go back to the manuscript and then revise my own text, trying to imagine how it is going to be received by the authors: did I miss something? Is the overall tone positive? Does my review reflect the manuscript actually written or at least the authors' intention?"
"I assume this is a natural consequence of getting experience as a reviewer, but Publons helps as it allows me to evaluate my own review history and compare it to other reviewers. The metrics are also a good suggestion of factors that might correlate with a good review. However I do not strive to review more papers per time unit, since this might jeopardise my effort of providing more useful reviews."Leonardo de Oliveira Martins
"I am definitely trying harder to write prepublication reviews as if I were not anonymous, so that I could stand for what I say directly to the authors. I am hoping to make most of my reviews public on publons in the future, if/when a manuscript is accepted for publication. I am also seriously considering to start performing open, non-anonymous post publication reviews, but I haven't made that big jump yet."Pierre Bellec
Are you a Publons user? Do the sentiments above sound familiar? Let us know in the comments below whether using Publons has changed the way that you perform peer review or not.