Go to publons.com

Not All Peer Review is Equal: Recognising Excellent Peer Reviews.

Introducing our exclusive new Excellent Reviews feature to recognise reviewers who go above and beyond.

We believe exceptional peer review can propel research forward. Comprehensive, timely and informative reviews ensure the highest quality, most impactful research is communicated to the world faster and more often.

We want to incentivise more great reviews by recognising the instances where researchers have made an outstanding reviewing contribution.

While some reviewers only deliver short and meagre reviews, for several different reasons, some reviewers invest a lot of time and effort to deliver a fair and well balanced review.

Currently, there has not been a good forum to properly address this important issue, but now Publons offers the possibility for editors to reward reviewers for this extra work.

I think this could improve peer reviewing by providing incentive to reviewers to go the extra mile to provide a more extensive peer review, something that the scientific society definitely would benefit from. <>

-- Eric Peter Thelin, Editor, Scientific Reports

How does it work?

Editors are now invited to rate the quality of reviews for manuscripts they handle on a four-point scale, against four criterion, as depicted in the image below:

review rating form

A review is deemed 'Excellent' if it receives a score of 9 or more out of a possible 12.

  • Poor = 0 point
  • Below Average = 1 points
  • Above Average = 2 points
  • Exceptional = 3 points

Excellent reviews receive a gold star against the review record on the reviewer's profile. See Rahul Raj's profile below for a real example:

rahul raj profile

Excellent reviews are also highlighted on your downloadable verified review record so you can include proof of how many of your reviews were deemed 'excellent' by editors.

Providing written feedback to reviewers

When scoring a review, editors can also provide written feedback to the reviewer. Written feedback is a great way to help reviewers improve, as well as a useful source of testimonials that researchers can utilise as proof of their reviewing performance.

NOTE: Scored reviews which do not meet the requirements for excellence will not be indicated on your profile or any other public part of the site.

Exact scores are completely confidential. Written feedback is privileged between the editor and reviewer.

FAQs

How was the methodology developed?

The excellent review schema is based off feedback from over 200 editors and our user-base of nearly 100,000 researchers. The four criteria (thoroughness, timeliness, clarity and helpfulness) were the aspects that most believed, when done well, made for an excellent review.

How do I know if my review has been rated excellent?

In addition to having a gold star appear next to the review on your profile, we will send you a notification to let you know your review has rated "excellent".

We also (confidentially) send you any written feedback from the editor.

Does this mean my other reviews are no good?

Absolutely not! Reviews marked as excellent are simply an additional indicator of the quality of a review. They do not in any way imply that reviews not marked as 'Excellent' are sub-standard.

If a review does not have an excellent rating, it does not mean it has been scored poorly. It may not have been rated at all.

What if my review gets a low score?

As mentioned above, excellent reviews are an added indicator about the quality of a review and do not imply that other reviews are of a sub-standard quality.

Only excellent reviews are publicly demonstrable. Actual scores are completely confidential and not made available anywhere on the site.

How can I get an editor to rate my review?

At present, editors can only rate reviews if they are adding a record of the manuscript they handled. We intend to incorporate the review rating feature into everyday editorial workflows to ensure as many reviews as possible can be rated.

If you would like to ensure your editors have access to the review scoring rating and feedback system, get in touch.

How do you prevent people rating their own, or their friend's, reviews?

We have a number mechanisms and checks in place to ensure people cannot:

  • rate their own reviews; or
  • insincerely rate the reviews of friends or colleagues.

For example, behind the scenes we work closely with Publishers to verify review and editorial activity.

If you have any questions about how we deal with these issues, or want to discuss specific activity on Publons, feel free to get in touch to discuss.

What's in it for editors?

Editors can now recognise and reward their reviewers that go the extra mile, further incentivising these experts to not only review, but review well.

What next?

We are also looking at ways for incorporating author(s) feedback and ratings into future versions of the schema.

We acknowledge there are other ways to measure review quality and that every methodology has its shortcomings. With that in mind, we note this is the first iteration of our review rating schema and, as always, we welcome your feedback and suggestions for improvements.

comments powered by Disqus

Subscribe to our mailing list

* indicates required