Go to publons.com

The STAP scandal: a post-pub review success story

In late January of this year, two papers published side-by-side in the journal *Nature *caused a wave of hype in the biology community by asserting that white blood cells could be transformed to a stem cell-like state via a simple acid bath. The potentially groundbreaking technique was dubbed 'STAP': stimulus-triggered acquisition of pluripotency. After a series of investigations, disciplinary hearings, and pressure from co-authors, Haruko Obokata, the first author on both papers, has agreed to retract the articles. So, what exactly went wrong with STAP? And perhaps more importantly, what went right, allowing the scientific community to so quickly identify flaws in the studies?

image

Too good to be true?

The publication of the STAP papers caused massive excitement in the stem cell biology field and, notably, a rush to replicate the method. Pluripotent stem cells, which have the ability to give rise to essentially any cell type in the adult organism, are typically found only in developing embryos - hence the ethical and legal debates surrounding the use of embryonic stem cells. In recent years, however, scientists developed methods for generating induced pluripotent stem cells from differentiated adult cells, like skin cells. This method has allowed a whole new field of research into disease development, drug testing, and regenerative medicine.

Unfortunately, the established method for obtaining induced pluripotent stem cells is time-consuming and has a low efficiency, so one can imagine the excitement when Obokata's papers presented a faster, easier way to convert adult cells into pluripotent cells, termed 'STAP cells'. The primary Nature paper outlined the STAP method, while the secondary paper demonstrated STAP cells' potential to differentiate into different tissue types.

The stem cell community received the studies with hopeful excitement, tempered by a healthy dose of skepticism. Frankly, it sounded almost too good to be true: could such a simple technique really turn differentiated cells back to a stem cell-like state? Within a week, researchers had already begun voicing their doubts.

Scientists, connected on the interwebs

Paul Knoepfler, a stem cell researcher at the University of California at Davis, immediately posted an initial review of the STAP studies on his popular stem cell-focused blog, raising a few key questions that remained unanswered, but overall sounding hopeful. However, doubts about the method's validity soon began to take hold on the blog and in discussion forums like PubPeer. In February, Knoepfler began posting crowdsourced results from researchers around the world documenting their attempts to replicate the STAP method, which also sparked lengthy discussions in the comments section. Stem cell researchers overwhelmingly reported negative findings when testing the method for themselves. Kenneth Ka-Ho Lee, a stem cell researcher in Hong Kong, extensively reviewed the STAP methods paper on ResearchGate, and in May published a paper detailing his lab's failure to reproduce the method in the online journal F1000Research.

Misconduct afoot

Unfortunately, the lack of reproducibility was not the only problem facing Obokata and the rest of the studies' authors. Soon after publication, evidence of potential scientific misconduct arose, with allegations of image manipulation, duplication of an image from Obokata's graduate thesis (from an entirely different experiment), and plagiarism. Genetic analysis called into question the true identity of various cell lines used in the studies, and other co-authors began calling for retraction of the papers. Riken, the Japanese institute at which the experiments took place, found Obokata guilty of scientific misconduct, but has not yet reached a conclusion on the validity of the research findings and is conducting its own replication studies.

Although Obokata initially refused to retract the papers, the peer pressure and disciplinary action eventually wore her down. As of mid-June, Obokata, along with most of her co-authors, have agreed to retract both papers.

Currently, both papers still stand. Nature Publishing Group's retraction policy states that all co-authors must agree to the retraction and sign a statement detailing the error(s).

Notably, Charles Vacanti, a researcher affiliated with Harvard Medical School, is the senior author of the STAP methods article and has not yet agreed to a retraction. As of his most recent public statement, he still vehemently stood behind STAP and Obokata's work. Nature has reported that it is conducting its own evaluation of the studies, but no further information has been provided yet.

The saga continues…

The STAP saga raises a number of questions. How did these flawed studies get through the peer-review process? Should Nature editorially retract the papers? Does this case show that science is actually self-correcting?

In May, Knoepfler wrote an editorial calling Nature to editorially retract both papers, stating: "To this day, nobody has gotten the STAP method to work and perhaps even more importantly, it is clear that both papers are irredeemable due to many serious and unfixable problems." Although the typical retraction policy requires agreement by all authors, an editorial published in Nature Medicine states that "[e]ditors, publishers or funding agencies can decide to retract a paper, but the usual policy is that the authors themselves must retract. Only in extreme circumstances would an editor retract a paper without the agreement of the authors". Does the burden of proof, including the clear scientific misconduct and continued failure of replication, constitute such extreme circumstances?

Some commentators are also calling for Nature to release more information surrounding the review process of the two papers. Scientist Robert J. Geller wrote, "I'd like to see Nature, while protecting the identity of the referees, disclose (on their website) all of the editorial correspondence (redacted appropriately) and every version of the submitted papers, so we can all judge whether or not the decision by Nature to accept the papers was appropriate based on the information available to them at the time." A more transparent review process would certainly shed some light on the situation, and perhaps reveal misgivings that the reviewers may have voiced, despite ultimately accepting the papers. The STAP situation calls to mind a similarly disputed study that came out in 2011 claiming to have discovered a bacterium that could incorporate arsenic into its DNA instead of the typical phosphorus. A record of verified peer reviews from the journal Science is available on Publons, providing a clearer picture of the study's context and acceptance process.

Is science self-correcting?

Although the peer reviewers failed to catch the problems in the STAP studies, the scientific community did catch them. This may represent one of those rare, highly public cases in which science was actually self-correcting: a situation in which the scientific community was able to quickly spot flaws as well as document the irreproducibility of a method. It's important to note that the STAP method was unusually high-profile. Simply due to the broad implications and universal utility of the STAP method, it was guaranteed that stem cell researchers all over the world were going to try to use the method in their own labs. For many other studies, the findings may only be relevant for a small handful of labs, making it more difficult for this "self-correction" to happen, especially in such a short timespan.

Another remarkable feature of this story is the centrality of the Internet as a forum for communication: the majority of the discussion took place on sites like PubPeer and ResearchGate, as well as blogs like Paul Knoepfler's. In addition, Knoepfler was able to compile the replication attempts of researchers from around the world - something that would have been far less feasible a couple decades ago. Google Trends nicely illustrates the "STAP cell" search term's popularity. The searches rapidly peaked immediately following publication, and by now have mostly petered out.

The importance of post-pub review

In addition, the STAP story demonstrates the importance of post-publication review and discussion of papers, even once they've passed the pre-publication check. By no means is peer review an infallible system, and it's the job of the scientific community to critically examine papers after publication.

It's certainly encouraging to see that although a couple deeply flawed studies got through the publication system, the scientific community as a whole, connected via the Internet, was able to quickly point out misconduct and potential issues with the method.

At Publons, we are creating a community around peer review - a centralized location for transparent pre-publication peer reviews, as well as post-pub discussions like the one that sprang up organically around STAP. Even better, all of your comments and reviews are linked back to your Publons profile, so you can keep track of your contributions as a measurable research output.

Join the conversation on Publons - make your profile and start discussing your favorite (or least favorite) papers now!

___

Alicia is a neuroscience graduate student in California. She's into health, tech, and science communication, and is super excited to write about peer review and academic publishing for the Publons blog. You can find her on Twitter @AliciaShiu.

comments powered by Disqus

Subscribe to our mailing list

* indicates required