Content of review 1, reviewed on December 17, 2014
Major Compulsory Revisions
While the manuscript presents a comprehensive study of statistical characteristics of different WGA methods, it is not clear who is the target audience that can benefit from knowing the presented differences of raw sequencing data. In particular: 1) Sequencing data often become a subject of in silico error correction. Can such correction affect the statistical characteristics of the data? In other words, there may be a bias in the comparison caused by (correctable) sequencing errors and this is not discussed in the manuscript. 2) As a researcher who performs de novo genome assembly of amplified sequencing data of bacterial genomes, I do not find this study much helpful as it does not address comparison of WGA methods from the perspective of subsequent de novo genome assembly. To this end, I would appreciate to see a comparison of assembled genomes from differently amplified sequencing data. 3) Can the amplification methods be also compared in terms of some "higher-level" applications (e.g., gene detection)? Results of such comparison would be more attractive in the abstract when the current somewhat raw characteristics. 4) I can imagine that people doing sequencing may consider not only quality of the methods but also their price -- some indication of how costly is each particular method may provide them with better guidance.
Statistical component of the paper appears a bit raw. It contains a lot of statistical estimations, but there are almost no potential explanations of the observed phenomena. The statistical analysis is not always straightforward. For example, in the sentense "Notably, the average normalized depth distribution for DOP-1 most fitted a standard Poisson-like curve (correlation coefficient 0.98)", the authors use correlation between data density and Poisson density to check if the data follows Poisson distribution. This is really strange, because there are plenty of standard tests to check goodness of fit hypothesis (for example, Pearson's chi-squared test). Also, there are no explanations why the authors prefer one test over another in the samples comparison, and it is not always even clear exactly what hypothesis they are checking.
It is not explained how some evaluation parameters are computed (e.g., SNV detection efficience, ADO rate).
It is confusing to see the 7th WGA kits (yikon genomics Single Cell Whole Genome Amplification kit) in the Methods section, since there are only 6 kits introduced in the "Data Description" section. According to the data description, the MALBAC-amplified WGS sequencing data is downloaded. Thus it is unclear if the MALBAC kit is used for WGA process.
Apparently, most results related to MALBAC methods are collected from elsewhere. The authors did not give a reason for not obtaining MALBAC result themselves, whether because the collected information are sufficient and correct enough for the comparison, or due to limitation of conducting experiments with MALBAC methods. Relatedly, it not described what WGS sequencing methods were used for the MALBAC amplified data, while the WGS methods used for DOP_PCR and MDA amplified data are clearly described. It may weaken the results assessment if MALBAC-amplified data were sequenced with different methods than DOP_PCR/MDA amplified data.
Minor Essential Revisions
Oxford commas are consistently missing throughout the manuscript, which makes some sentences hard to digest. For example, second sentence in Abstract->Background on p.3 and first sentence in Background on p. 4 contain multiple terms connected with 'and' without any punctuation, which makes them hard to follow.
The manuscript is heavy in jargon, which is loosely explained (e.g., "mapping efficiency" on p. 5). I would suggest to remove jargon from the abstract as much as possible and presents results in terms of which methods work best in what applications (see above).
In the abstract: "SNV" is not defined.
Some other minor errors: (Page 9, 2nd paragraph, line 4) "...had a average ..." -> "... had an average ..."; (Page 10 2nd sentence) It is not clear which method "demonstrated higher evenness and reproducibility than the other WGA methods".
Level of interest An article whose findings are important to those with closely related research interests Quality of written English Needs some language corrections before being published Statistical review Yes, and I have assessed the statistics in my report. Declaration of competing interests I declare that I have no competing interests.
Authors' response to reviewers: (http://www.gigasciencejournal.com/imedia/3021547641562807_comment.pdf)
Source
Content of review 2, reviewed on February 23, 2015
Major Compulsory Revisions
1) In Response 3, the authors say "Using the 30X whole genome sequencing data, it is difficult to assemble a human genome accurately". I cannot buy this argument, since it is not clear why the human genome is chosen as the target at the first hand. For example, single-cell sequencing is now quite popular for bacterial genomes (of significantly lower complexity than the human genome). While many genome assemblers have no trouble assembling bacterial genomes (e.g., E. coli genome) from the standard (multi-cell) sequencing data, they may experience certain difficulties with single-cell data. It therefore would be very helpful to see if this depends on the choice of an amplification method. Furthermore, this would allow one to compare different technique pairs of amplification+assembler. I truly wonder whether different genome assemblers may be best paired with different amplification methods, or there is a clear winner among them.
2) Relatedly, with respect to sequencing error correction, it is claimed that the low quality reads have been excluded. This is not practical from the genome assembly perspective, since low quality reads can often be recovered with an error correction tool, and after correction such reads significantly help to improve the resulting assembly.
3) The target audience and applications are still not clearly identified in the manuscript. In Response 3, the authors stated that "In our study, we majorly focused on the reference based re-sequencing and variation-calling effort of the human and cancer cell-lines." If the authors do not want to go beyond these particular applications (e.g., not deal with bacterial genomes and/or genome assembly as suggested in the comment 1 above), the title and abstract of paper should be made more narrowed and specific, mentioning that the comparison is performed from the perspective of these particular applications of interest. For example, the current title is "Comparison of Whole Genome Amplification Methods used in Single-cell Sequencing" is too broad and makes me immediately wonder if such comparison can help in choosing a right pipeline for de novo single-cell sequencing of bacterial genomes (and it does not help at all!).
4) I do not understand "which is in consistent with the previous report". Does it mean "which is INCONSISTENT with the previous report" or "which is IN CONSISTENCE with the previous report"?
5) The authors wrote: "In the revised manuscript, all the statistics were calculated by Wilcoxon Mann-Whitney Test, which is one of the most powerful of the nonparametric tests for comparing two populations and does not require the assumption that the differences between the two samples are normally distributed." This is correct. But the authors used the WMW test for comparison of several (more than 2) groups, so they need to use the Bonferroni correction or some multiple comparison test. Since the paper is about comparison, I believe the authors should consult to a statistician.
Minor Essential Revisions
Some typos: 1) Page 25, paragraph 2, line 8, "To evaluate the bias in the comparison cased by the correctable sequencing errors", "cased". 2) Page 29, last line, "variantion". 3) Page 32, "Additional file 12: Table S7. Genes annnotation", "annnotation".
Level of interest An article whose findings are important to those with closely related research interests Quality of written English Acceptable Statistical review Yes, and I have assessed the statistics in my report. Declaration of competing interests I declare that I have no competing interests.
Authors' response to reviewers: (http://www.gigasciencejournal.com/imedia/1966304336168046_comment.pdf)
Source
Content of review 3, reviewed on May 01, 2015
The review adequately addressed my previous concerns.
Minor Essential Revisions
On p. 26, all instances of "sequencing error" should be in plural form: "sequencing errors".
Level of interest An article whose findings are important to those with closely related research interests Quality of written English Needs some language corrections before being published Statistical review Yes, and I have assessed the statistics in my report. Declaration of competing interests I declare that I have no competing interests.