Signed openly by
Andrew Preston and Daniel Johnston, co-founders of Publons
Our mission at Publons is to speed up science by harnessing the power of review. Step one in our plan is all about giving researchers credit for peer reviews. We're already seeing evidence that this is improving the speed and quality of the peer review process.
Publons now serves more than 40,000 reviewers and contains over 175,000 reviews. So we've put serious thought into how we can maintain the necessary levels of confidentiality for authors and journals while giving reviewers the freedom to list and verify their contributions, should they wish to.
Recently, COPE posted a request for comments on this topic, asking "who 'owns' the review?" Given how much time we've spent on this issue, we thought it might be helpful to share what we've learned, outline the policy we've found to work best, and respond to specific questions raised by COPE.
What we've learned
To start with, the question of who "owns" the review is pretty straightforward from a legal perspective: unless the reviewer signs an explicit transfer, the reviewer owns the copyright to their review. For journals that publish reviews, reviewers typically assign the publisher a non-exclusive license to publish their review (typically CC-BY).
The real issue is much more interesting than pure intellectual property rights. At its heart, this is a question of how to develop a policy on peer review that takes into account the needs of authors, reviewers, and editors, is flexible enough to deal with the editorial policies of 27,000 different journals, and still makes it possible for the research community to innovate.
Blind reviews are not always needed
The majority of peer reviews today are single blind and unpublished. This is primarily:
- To ensure that reviewers are willing and able to speak freely, which is particularly important for early career researchers.
- To protect authors from having their papers "scooped" before they are published.
These are not universal rules. They are pragmatic decisions made to benefit research, and unsurprisingly there are tradeoffs. For example, while blindness helps reviewers to speak freely, many authors can point to times where they've felt that an anonymous reviewer was lazy, asked for unnecessary citations, or otherwise pushed their own agenda.
These tradeoffs are everywhere and we, as a research community, owe it to ourselves to continue to weigh the pros and cons and adjust our policies for what is on the balance best for society.
With this in mind, we have to ask why should we prevent a reviewer from signing their name to a review if they wish to do so? Similarly, if an author and a reviewer agree to publish the reviews of a manuscript, after the manuscript has been made public, why would we stop them?
Policies must support many types of review
Reviews from thousands of different journals flow through our platform on a daily basis, meaning we see just about every review policy possible. Besides the common 'blind', 'double-blind', or 'open' peer review, we've seen countless variations on:
- whether the authors' names are revealed to the reviewers
- whether the reviewers' names are revealed to the authors, and/or made public
- whether the reviews are published
Nature sometimes blinds authors, always blinds reviewers, and never publishes review content. PeerJ never blinds authors, allows reviewers to choose whether they remain blind, and allows authors to choose whether the review is published along with the manuscript. Siggraph blinds authors and reviewers and never publishes content. The Frontiers journals never blind authors or reviewers but also never publish review content. eLife has a collaborative review process, resulting in a single summary review that is prepared by the editors. F1000 Research, on the other hand, reveals everything at the time the article is submitted.
Any general policy on peer review should support these publishing arrangements, and more.
Should reviews survive manuscript rejections?
What should we do about reviews of rejected manuscripts? The reviewer still needs to get credit, otherwise they will have an undue incentive to accept submissions. It seems wasteful to throw away reviews for a rejected manuscript that will immediately go through the review process at another journal (some publishers now have implemented measures to reduce duplicated review efforts for 'cascading' manuscripts) but is it unfair on authors to share?
Again, we need to weigh the desires or reviewers, authors, and journals against what is good for research.
What happens after publication?
Through all of this variation in journal policy, we have discovered that many journals do not yet have a clear policy for how to treat the above parameters after publication. Specifically, most publishers we have spoken to hadn't fully considered the question of reviewers posting their reviews after the review process until reviewers started doing it to share their insights and demonstrate their reviewing competence on their Publons profiles.
An easy response is to prohibit the disclosure of review information after publication, but is this what authors, reviewers, and other stakeholders want? Is that what's good for research?
Our practical experience (from a sample of 175,000 reviews) is that reviewers - like journals - have a wide range of preferences when it comes to disclosing information about their reviews. 16% of reviewers opt to sign their name to reviews at least some of the time. 11% opt to publish some of their reviews (either anonymously or with their name attached). 6% opt to publish some of their reviews with their name attached.
It is also worth nothing that we've reached out to hundreds of authors that have articles with published reviews, providing the option to request the reviews be hidden. No authors made the request.
We have also run a larger survey asking researchers how they would feel if reviews of their manuscripts were published. Those that answered "no problem" vastly outnumbered those that said they wouldn't like it. About 40% answered "it would depend on the review", highlighting a range of preferences among the research community. (Full survey results to follow on our blog in the near future.)
In consultation with reviewers, authors, many editors, and our publishing partners, we've developed the following policy for Publons:
- In every case, reviewers can not share any details about the manuscript (manuscript title, review content) until the manuscript is published.
- After publication, the reviewer decides if they publicly associate their name with the article (unless this is forbidden by the journal).
- After publication, the reviewer decides if they publish the content of their review (unless this is forbidden by the journal).
- The author, editor, or journal can request a published review to be hidden at any time, although we do ask that they consider why this is necessary.
- Journals can specify blanket policies (eg) to apply to all reviews added to Publons for their journal. We see this as a pragmatic way to gauge what authors for a given journal want.
This is just a start. We are constantly looking for ways to improve this policy - especially where we can find alternative approaches that might improve the peer review process and research as a whole. For example, we are considering whether it would make sense to restrict reviewers from publishing reviews unless both the reviewer and author(s) agree.
In soliciting feedback on this topic, COPE had some specific questions. Our comments on these below:
Does it violate confidential/blind peer-review to reveal reviewer comments even after publication?
The right question to ask is whether opening up review after publication, with the consent of the reviewer and other stakeholders, improves the research process.
Signing a review or posting the comments after publication improves the transparency of the review process without requiring any changes to the way journals conduct their peer review process. We, as a research community, need to discuss whether there are any potential harms (such as the potential that authors may not be comfortable with this) that outweigh the benefits.
From the perspective of the reviewer, revealing the review content after the fact allows them to demonstrate the work they put in and the contributions they made to the finished article. Reviewers have told us that they do a better job of reviews when they know they that they may later publish the review, and that sharing their reviews has increased their standing among their peers.
We see the best approach as leaving the choice up to the reviewer, except in cases where another stakeholder (the journal or the author) forbids it.
What can journals do to make sure reviewer comments remain confidential?
The first point to discuss is whether reviewer comments should remain confidential after publication. And if so, why? And who should decide - the reviewer, the author, or the journal?
For our part, we try to work with every journal to specify what information their reviewers are permitted to disclose on Publons (and already do so for hundreds of journals).
How can reviewers ensure they are able to share and get credit for their work?
We have already taken care of this problem to the extent that 40,000 reviewers are getting credit, all while verifying their contributions and managing the complex privacy considerations above.
The benefit of Publons is that it allows reviewers to get credit for their peer review in a way that is consistent with the varied policies of the journals they review for. Even in cases where they are not permitted to disclose additional details of their reviews, we enable reviewers to get verified credit of their peer review activity in a format they can use in promotion and funding applications.
Can service providers work with journal editors, publishers and reviewers to help facilitate openness and transparency in peer review?
We have definitively answered this question in the affirmative over the last year, with a series of partnerships with publishers that are objectively improving the peer review process. Early results show higher review request acceptance rates, quicker responses to review invitations, and faster turnaround of completed reviews.