“Improving functional magnetic resonance imaging reproducibility” Cyril Pernet and J.B. Poline GigaScience Commentary

Study reproducibility is an important issue that is not often addressed. In particular, neuroimaging has been particularly remiss about study replication and the re-analysis of study data. This is a large and complex topic, however, and really needs a broader treatment than is given here. A short commentary with a “five-step plan” seems a bit too glib for a topic fraught with well-known sociological and technical issues. A more in-depth article exploring all of these issues would have been welcomed.

The paper is somewhat unclear on the difference between empirical replication vs. analytical reproducibility. Some additional discussion on the distinction between independent replication of a study (new data collection, independent analysis, but arriving at the same results or not) versus the sharing of acquired data so that someone else can repeat the analysis is needed. This distinction is fundamental and important for what is meant by “reproducible”.

What might be the role of journals and funding agencies for encouraging study replication/reproduction and promoting the sharing of data so that results can be validated/confirmed/extended? What other stakeholders might be in the mix here?

What about benefits for the education of the next generation of brain imaging scientists? Does being able to reproduce a result have educational value? I believe that it does and would be worth mentioning.

Subject anonymization (e.g. for HIPAA compliance or similar in non-US countries) has always been a thorny issue. How do the authors suggest this be handled for open sharing of data? Do software tools or other services exist to perform this function in advance of sharing? Is this responsibility of the investigator or the data base curators?

The Alzheimer’s Disease Neuroimaging Initiative (ADNI) is often held up as the prime example of neuroimaging data sharing. These data are stored in the well-known LONI Image and Data Archive (IDA) (http://ida.loni.usc.edu). The resource includes in addition to the entire ADNI data set imaging data from the Michael J Fox Foundation, Alzheimer’s Association, HCP, and other major multi-site imaging trials. Many of these are “open” though do require one to ask for access. But this important data resource is not mentioned but should be.

Finally, the authors make the following small errors but which are easily corrected: - For the INCF’s NeuroImage Data Model, I think you mean the acronym to read “NIDM” not “NIMD” - Likewise, the NeuroImaging Tool and Resources Clearinghouse should be “NITRIC”.

Again, this is a well-written piece, if somewhat unfulfilling, discussion of how to improve the reproducibility for neuroimaging studies. While the ideas presented are reasonable and it is hard to disagree, it would have been nice to see a broader discussion of this important topic as a fully-fledged article.

Level of interest An article whose findings are important to those with closely related research interests Quality of written English Acceptable Declaration of competing interests Have you in the past five years received reimbursements, fees, funding, or salary from an organisation that may in any way gain or lose financially from the publication of this manuscript, either now or in the future?

No

Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this manuscript, either now or in the future?

No

Do you hold or are you currently applying for any patents relating to the content of the manuscript?

No.

Have you received reimbursements, fees, funding, or salary from an organization that holds or has applied for patents relating to the content of the manuscript?

No

Do you have any other financial competing interests?

No.

Do you have any non-financial competing interests in relation to this paper?

No.

If you can answer no to all of the above, write 'I declare that I have no competing interests' below. If your reply is yes to any, please give details below.

I declare that I have no competing interests.

Author response:

http://www.gigasciencejournal.com/imedia/4514699461599493_comment.pdf