Content of review 1, reviewed on August 14, 2017
@@
This is an amusing paper that talks about the very different "lives" of two samples, one in a lab that has good metadata collection and automated analysis pipelines, and the other in a lab that ...does not. It's a great story with lots of implications and I think it fits the mission of GigaScience very well.
The one weakness of this paper as a rhetorical device is that it fails to discuss the up-front costs to the "good" lab in any detail. What are these costs? What systems/software need to be put in place? How much does this cost in terms of expertise (hiring, training) and maintenance (software systems, database) and software cost? etc. I am aware that open source is "free" but it's not free in terms of maintenance. This is where my (small) lab has the most trouble - we simply can't afford the maintenance costs for these systems, and so have jerry-rigged github and other approaches in a not-entirely-smooth system.
review numbering p2 line 33 - "old-timers" is sort of an odd choice here, since in my experience the old-time biologists don't think that. Perhaps "experienced people" would be a good replacement.
p3, 55-56 - neither Jupyter notebooks nor RMarkdown notebooks are good for long running analyses, so why would they be used here? wouldn't a simple markdown document suffice? or perhaps the jupyter notebook would contain the analyses (graphing, reporting) done AFTER the long-running analyses?
p4 line 16-17 polish -> publish?
p4 line 21-22 is the FTP site "open" or private? should specify :).
p4 line 47 not many labs have this ability... "100 samples as effortlessly as a single one"- how realistic is this with current software systems?
p5 line 10 - Docker images and hubs should not be considered archival (which is fine, but maybe should be noted). See concerns about the life of docker containers here (https://thehftguy.com/2016/11/01/docker-in-production-an-history-of-failure/) and here (http://ivory.idyll.org/blog/2017-pof-software-archivability.html). Also, the Dockerfiles are as or more important and should be placed in version control...
p5 line 14 - the story is kind of biased if the experiments didn't work in the second sample. This has nothing to do with the metadata. If this is a real life story, all well and good; but if it's a moral story, then I would suggest amending in some way to suggest that questions arose and could not be answered, even though the analysis "worked".
p5 34-35 - Behind the scene <- should be scenes
p5 49-45 what does future proof mean? how realistic is it to future proof database design in a research lab? maybe "future aware and flexible" is more realistic?
While I infer from the letter at the end that the editors suggested cutting down on the story of the "unfortunate" sample, I must disagree. I think the story would be a bit more complete if you pointed out in a bit more detail where the specific failings were with the second sample. (If you can't figure out how to do this in a sentence or two, then there is no need to work hard at it - the paper reads fine without it.)
Level of interest
Please indicate how interesting you found the manuscript:
An article of importance in its field
Quality of written English
Please indicate the quality of language in the manuscript:
Acceptable
Declaration of competing interests
Please complete a declaration of competing interests, considering the following questions:
Have you in the past five years received reimbursements, fees, funding, or salary from an organisation that may in any way gain or lose financially from the publication of this manuscript, either now or in the future?
Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this manuscript, either now or in the future?
Do you hold or are you currently applying for any patents relating to the content of the manuscript?
Have you received reimbursements, fees, funding, or salary from an organization that holds or has applied for patents relating to the content of the manuscript?
Do you have any other financial competing interests?
Do you have any non-financial competing interests in relation to this paper?
If you can answer no to all of the above, write 'I declare that I have no competing interests' below. If your reply is yes to any, please give details below.
I declare that I have no competing interests.
I agree to the open peer review policy of the journal. I understand that my name will be included on my report to the authors and, if the manuscript is accepted for publication, my named report including any attachments I upload will be posted on the website along with the authors' responses. I agree for my report to be made available under an Open Access Creative Commons CC-BY license (http://creativecommons.org/licenses/by/4.0/). I understand that any comments which I do not wish to be included in my named report can be included as confidential comments to the editors, which will not be published.
I agree to the open peer review policy of the journal.
Authors' response to reviews: (https://drive.google.com/open?id=0B0V9UazwxfgRa0lrYnRmdUNuT3c)
Source
© 2017 the Reviewer (CC BY 4.0).
