Go to publons.com

Should I sign my review?

In the almost 3 years of Publons' existence we've managed to accrue a lot of interesting data around peer review. The following is the first in a series of posts designed to communicate the interesting trends and tidbits we glean from these data.

If you've visited a journal detail page on Publons recently (eg. https://publons.com/journal/293/peerj) then you may have noticed the addition of research fields. These research field data were extracted from the ASJC (All Science Journal Classification) Codes found here.

The research fields do more than demonstrate the areas of research relevant to a particular journal; they allow us to make inter and intra-field comparisons providing deeper insight than was previously possible.

The first thing we wanted to do with these connections was to analyse which research fields have the highest prevalence of signed peer review. While signing pre-publication peer reviews is still contentious, it's not uncommon for reviewers to sign reviews submitted even to journals who practice double blind review. Note, "signing" reviews is, at least for the purposes of this post, distinct from publishing their content.

Signing reviews is a powerful way to increase the transparency of the peer review process and allows reviewers a further (until recently the only) method of gaining recognition for their work. Early career researchers have the most at stake in signing reviews and have little way of knowing just how common it is within their field of research.

What we found when we ran the numbers was somewhat surprising.

field % of reviews signed number of reviews
Psychology 52.38 1894
Medicine 44.26 19024
Neuroscience 33.33 4045
Agricultural and Biological Sciences 26.62 9064
Biochemistry, Genetics and Molecular Biology 25.01 9641
Pharmacology, Toxicology and Pharmaceutics 5.98 418
Social Sciences 5.83 960
Energy 5.38 372
Chemistry 5.13 3314
Materials Science 5.09 2772
Engineering 5.06 1383
Decision Sciences 4.46 269
Environmental Science 4.42 1676
Earth and Planetary Sciences 4.38 434
Immunology and Microbiology 4.28 911
Chemical Engineering 4.27 1848
Nursing 3.91 128
Business, Management and Accounting 3.45 290
Veterinary 2.9 482
Physics and Astronomy 2.77 1875
Computer Science 2.73 477
Arts and Humanities 2.44 491
Dentistry 2.32 345
Economics, Econometrics and Finance 1.27 157
Mathematics 0.96 415

Many of the high profile adopters of signed peer review so far have been either biomedical (BioMed Central, BMJ Open, Archives of Medical Science) or multidisciplinary (F1000, PeerJ, PNAS) journals/publishers. This isn't entirely at odds with the numbers above but there are some interesting differences from what we might have assumed.

According to data published by PeerJ regarding the prevalence of signed reviews (performed for PeerJ) across various fields 'Paleontology' is one of the most common disciplines to have signed reviews in PeerJ. The numbers gleaned from Publons' data put palaeontology at just 15.72% of 159 reviews signed.

Our data simultaneously suggests that Psychology is the discipline most likely to have signed reviews on Publons, which is not consistent with anything we've seen asserted anywhere else. PeerJ, for instance, puts Psychology at #8 on their list of disciplines by number of signed reviews.

High energy physics, mathematics and computer science are often thought of as the earliest adopters of new standards in scholarly publication. Extensive use of the arXiv, Timothy Gowers' organised boycott of Elsevier journals and the SCOAP3 initiative to convert several High energy physics journals to open access models are all prime examples.

Where are these firebrand physicists‽ Obviously not on Publons. Only 7% of physics related reviews on Publons have been signed, around half the rate of chemistry or social sciences reviews. Even Atmospheric Chemistry and Physics which practices what they call Interactive Public Peer Review, have a mere 36% of their reviews signed on Publons.

It's possible that early adopters of open review in these fields have naturally gravitated toward open access mega journals which aren't specifically denoted as physics journals in our database. Or perhaps forward thinking physicists are less motivated by formal recognition.

At present you can't search Publons by these research fields, that'll be coming in the near future. Before we get that far, consider letting us know via the comments below how you imagine you would like to use journal field data. Would you want to browse similar journals to your first choice to find more candidates for submission? Browse your fields for interesting open reviews? Or something completely different we haven't thought of yet?

comments powered by Disqus

Subscribe to our mailing list

* indicates required