Abstract

Background: There is a rapidly increasing amount of de novo genome assembly using next-generation sequencing (NGS) short reads; however, several big challenges remain to be overcome in order for this to be efficient and accurate. SOAPdenovo has been successfully applied to assemble many published genomes, but it still needs improvement in continuity, accuracy and coverage, especially in repeat regions.Findings: To overcome these challenges, we have developed its successor, SOAPdenovo2, which has the advantage of a new algorithm design that reduces memory consumption in graph construction, resolves more repeat regions in contig assembly, increases coverage and length in scaffold construction, improves gap closing, and optimizes for large genome.Conclusions: Benchmark using the Assemblathon1 and GAGE datasets showed that SOAPdenovo2 greatly surpasses its predecessor SOAPdenovo and is competitive to other assemblers on both assembly length and accuracy. We also provide an updated assembly version of the 2008 Asian (YH) genome using SOAPdenovo2. Here, the contig and scaffold N50 of the YH genome were similar to 20.9 kbp and similar to 22 Mbp, respectively, which is 3-fold and 50-fold longer than the first published version. The genome coverage increased from 81.16% to 93.91%, and memory consumption was similar to 2/3 lower during the point of largest memory consumption.

Authors

Luo, Ruibang;  Liu, Binghang;  Xie, Yinlong;  Li, Zhenyu;  Huang, Weihua;  Yuan, Jianying;  He, Guangzhu;  Chen, Yanxiang;  Pan, Qi;  Liu, Yunjie;  Tang, Jingbo;  Wu, Gengxiong;  Zhang, Hao;  Shi, Yujian;  Liu, Yong;  Yu, Chang;  Wang, Bo;  Lu, Yao;  Han, Changlei;  Cheung, David W.;  Yiu, Siu-Ming;  Peng, Shaoliang;  Zhu Xiaoqian;  Liu, Guangming;  Liao, Xiangke;  Li, Yingrui;  Yang, Huanming;  Wang, Jian;  Lam, Tak-Wah;  Wang, Jun

Publons users who've claimed - I am an author
Contributors on Publons
  • 4 authors
  • 3 reviewers
Followers on Publons
  • The revised version of the manuscript is much improved and I appreciate all the additional work performed by the authors.

    Major Essential Revisions

    I examined the SOAPdenovo2 performance on the GAGE data sets and on the Assemblathon data set. There are definitely clear improvements from the version 1 (1.05). However I do insist that the GAGE comparison along with the table is included into the main body of the paper, and not in the supplementary material. I think readers do need to be aware of the assembler performance metrics and compaisons on both the faux and the real data sets. I reiterate that in my experience the assembler's performance on the faux data is not directly indicative of the performance on the real data, but I do respect the results and effort of the Assemblathon project and I think that the evaluation on the Assemblathon data deserves to be included in the paper.

    Level of interest: An article whose findings are important to those with closely related research interests

    Quality of written English: Needs some language corrections before being published

    Statistical review: No, the manuscript does not need to be seen by a statistician.

    Declaration of competing interests: No competing interests.

    Published in
    Reviewed by
    Ongoing discussion
  • The authors have done a good job responding to all the criticism. We thank them for the changes and additions they made to the manuscript. We particularly appreciate the effort that went into the supplementary material.

    We therefore recommend to accept the manuscript for publication, pending some minor revisions.

    Major remarks:

    The authors write in their response "We will release the source code of SOAPdenovo2 as soon as the paper is accepted." We still feel strongly that the reviewers should have been given access to the source code of the program(s). However, we decided not to reject the paper because of this omission.

    The authors write on page 5 "Notably, SOAPdenovo1.05 was released two years after SOAPdenovo1 and already included several improvements and new features from SOAPdenovo2, including the new contig and scaffold construction improvements, but without the new error correction and gap closure modules. " We are wondering whether it then is fair to compare SOAPdenovo2 only with SOAPdenovo1. The authors should have included SOAPdenovo1.05 as well for testing the new 100bp PE reads. At the very least they should acknowledge the limits of their comparison.

    Minor Essential Revisions

    Main text

    • page 3, in the sentence "However, the error correction module in SOAPdenovo was designed for short Illumina reads (35-50bp), which consumes excessive amount of computational time and memory on longer reads, say over 150GB memory running for two days using 40x 100bp paired-end Illumina HiSeq 2000 reads." It is not clear what is referred to for the 40x. Is this a genome the size of the human genome?

    • page 5: "The SOAPdenovo2 assembly also had a much lower amount of copy number errors, but did have more substitution errors." Please include a description on how these measures are calculated

    • please be careful to keep the colors consistent among figure 1 and 2, to avoid confusion

    • table 1: Copy Number Error-rate --> how is this defined, what is the unit?

    • table 2: please indicate v1 or v2 for the 'version' column of each row; please explain 'scaffold size'

    • both tables: what are the units? (usually bp)

    Supplementary

    • could page numbers be included in the table of contents of the supplementary material?

    • section 1: the text mentions 'SOAPec-1.0' and 'SOAPec-2.0', however, these terms are used nowhere else. Please be consistent

    • section 1: "The algorithm is based on k-mer frequency spectrums (i.e. KFS), but the algorithm is quite different from other KFS tools" Please reference these other tools.

    • section 7: "As shown in Table 2, the scaffold N50 of SOAPdenovo2 overwhelmed ALLPATHS-LG and increased more than 4-fold compare to SOAPdenovo. But the contig N50 of ALLPATHS-LG is the longest" We object to the word 'overwhelmed'. There is a difference, but it is not overwhelming. In addition, the larger contig N50 of Allpaths is equally 'overwhelming' soapdenovo2… Please rephrase.

    • section 7: "However, the contig N50 could be further improved for SOAPdenovo2 by using 3’-end connected reads and larger k-mer size as ALLPATHS-LG do." Why didn't the authors try this then?

    Discretionary Revisions

    • we recommend the authors to ask a native English speaker to have a look at the text, especially of the Supplementary material

    Level of interest: An article of importance in its field

    Quality of written English: Needs some language corrections before being published

    Statistical review: No, the manuscript does not need to be seen by a statistician.

    Declaration of competing interests: I declare that I have no competing interests

    Published in
    Ongoing discussion
  • General comments:

    The manuscript presents SOAPdenovo2, an upgrade for the popular short-read sequence assembly tool. In order to demonstrate the novel features and improvements of this new version the authors have applied SOAPdenov2 to two different datasets: the raw data used for the Assemblathon 1 project and the YH human whole-genome sequencing data. These datasets are distributed as part of the manuscript’s submission in the spirit of the GigaScience journal. This is very valuable data as allows the reviewers and readers to reproduce the results and evaluate other aspects of the software interface. We welcome this approach and hope this manuscript will set the standard for similar bioinformatics software publications in the future. We have evaluated this manuscript both based on the quality of the presentation of the article and the assembled sequences.

    The new developments in SOAPdenovo2 are focused on four areas:

    • more efficient algorithms and data structures to reduce the memory requirements for the step of the de Bruijn graph construction;

    • use of a multiple k-mer approach to improve the handling of errors and low-coverage regions;

    • better handling of heterozygosity with a reduction of misassemblies and chimeric sequences at the scaffold level; and

    • an improved method to implement gap closure.

    Although we have seen significant improvements in the past couple of years the assembly problem for short reads remains an important challenge. The main conclusion from some of the recent competitions and comparative studies such as the Assemblathon and GAGE is that there is no “one-fits-all” solution, rather every study and datasets seem to require a tailored and customised tool. In this context only few tools have been successfully applied across a wide range of genomes and SOAPdenovo is one example. We believe that the authors decision to focus on the improvement of the genome structure rather than the contig level is the correct direction as the weaknesses in the current tools are indeed in the scaffolding stage. We therefore would like to recommend the publication of this manuscript and associated datasets in Gigascience. In order to ensure the best quality for the final version, however, we ask the author to address some major points before the article is accepted for publications.

    Major Compulsory Revisions:

    • A major contribution of Gigascience is to provide a framework to publish “data-driven” scientific studies alongside the datasets following open-access open-data principles. As mentioned above we believe this is an important advance in the way algorithms and software are published in bioinformatics. With this in mind we would like to request the authors to provide a detailed flow-chart with the steps to follow to reproduce (at least partially) some of the results reported in the manuscript. The parameters and software used should also be described to allow the reviewers and readers to run the tools as done by the authors. We understand there will be certain points that might require manual interventions or the use of unpublished software but those steps should be clearly indicated. We tried to follow the steps described in the “readme” document obtaining the following results using the Assemblathon 1 data and running just up to the assembly step:

    • Before scaffolding: there are 77,636 contigs longer than 100,
      Total sum 114,997,317 bps, with average contih length 1,481.
      Contig N50 is 3,146 bp, contig N90 is 818 bp.
      460,293 contig(s) longer than 32bps

    • After scaffolding: longest scaffold 21,794,522
      Scaffold and singleton number 4,093
      Total sum 118,860,825 average length 29,040
      Scaffold N50 8,018,180 N90 3,409,497

    Although we have tried different parameters and effectively done four different assemblies, no combination of parameters actually allowed us to get similar statistics to the figures mentioned on Table 1, so this is why we think reproducibility will be greatly improved by inclusion of the requested flow-chart. In a similar way, a list of all the software or tools used should be included, detailing whether it is publicly available, being published in this work, soon to be released or to remain closed.

    • It is not clear whether the impressive improvement in the YH assembly is a result of the new libraries added to the dataset or the novel algorithms in SOAPdenovo2. There are two new libraries, one of them with insert size 4-times larger than anything used before. Was the previous version of SOAPdenovo able to cope with this type of data effectively?

    • From the readme document distributed with the data it is clear that a consensus step (step 6) was conducted after the assembly. Was this step used to modify the assembly result? Are the metrics (i.e. substitution rates) affected by this step?

    • Although the authors indicated SOAPdenovo2 outperforms SOAPdenovo in memory usage, it isn't clear from Table 1 what is the level of improvement. The usage of extensive read filtering and correction should be taken into account and be mentioned either as an equal or different scenario.

    Minor Essential Revisions:

    • More detail is needed to understand some of the new algorithms and improvements. This is a list of points that we suggest the authors should address to improve the quality of the final version of the manuscript:

    • Sparse graph. It is not clear which sparse graph approach has been used for SOAPdenovo2. From the reference in the text the indication is that a "sparse de Bruijn graph" as implemented by sparseassembler1. A more recent version of this approach, sparseassembler 2, has tackled some of the problems of the original development based on a "sparse k-mer graph". There seems to be a mistmatch between the description in the text and the citation. If this was the first version, could this stage be improved by the use of the new approach? More concretely: at which stage is the sparse graph used? From running SOAPdenovo2 ourselves we believe this new data structure is only used at the initial stage of building the graph resorting to a traditional graph for later stages. The two example assemblies have apparently not been generated with the sparse graph, is this because of some trade-off or is the sparse-graph now the default and has been effectively used in all the assemblies? On a technical note will the sparse graph construction directly impact the ability to preserve important meta-information such as sequence coverage? How did this information loss affect the final assemblies?

    • Multi k-mer approach. The authors recommend to start with a small k-mer, how small? How many k-mers are tried and what are the criteria to preserve information (if anything at all) from one k-mer level to the next one? The relevant ‘-m’ option is apparently only used on the contig step of the assembly, so we assume that the k value is still used as before, but on traversing the graph on the contig step larger k could be used on ambiguous regions? Is this a contig step only modification on is there some kind of support from the other steps? This parameter forces the use of the configuration file as parameter too, is this because it is effectively re-evaluating k-mer content from the input files?

    • Heterozygosity. The authors mention that a topology-based method has been developed to work around heterezygous sites. Which kind of structure/topology this method detects in the graph?

    • Read correction. The main goal is to reduce memory consumption and remove the source of spurious contigs. How is the procedure used in SOAPdenovo2 comparable the other assemblers, particularly with the SOAPdenovo run on YH? Is there any evidence that this help SOAPnovo2 getting longer and more reliable contigs? Should read correction always be used as a previous step on SOAPdevono2? We would also like to suggest the authors to describe any rules and principles to set the parameters for read correction. From the data it seems that the only reads corrected are from PE libraries, is this linked to the chimeric contigs introduced by these libraries?

    • Library insert size. They appear to be very exact for some of the libraries (the ones coming from v1 YH, including the LMP) and just an estimation for other libraries. Is this parameter checked/calibrated at the scaffolding stage? Is there any particular restriction on the distribution of insert sizes that could be taken into account for the scaffolder to work optimally? The GapCloser configuration file specifies a min and a max parameters for insert sizes: how are these values calculated or set? In particularly we would like to know why those same limits aren’t the same as the ones used in the previous steps.

    Discretionary revisions:

    • The results from the Assemblathon 1 datasets suggest a modest enhancement in the contigs but a remarkable improvement on the scaffolds (even if not on the range of 44-fold). This scenario is consistent with the described features of SOAPdenovo2 and should be emphasized.

    • Although the N50 statistic is a widely used metric for assembly quality, it is important when evaluating assembly algorithms to look at the full set of contigs/scaffolds and their lengths. A plot of accumulated sequence length vs. contig count could provide better insights on the improvements of the algorithms than simply N50, which is effectively a single first-derivative value of this curve on the point where it reaches half of its final value.

    • The authors suggest that chimeric scaffolds are mainly introduced by smaller PE libraries. Is this just an effect of the sequence depth as usually shorter inserts are generated with more coverage? In that case: wouldn't it be just a simple step to deal with chimeras through coverage?

    • The term "coverage" is used for two different concepts across the text when referring to:

    • how much sequencing data has been generated with respect to the estimated target genome size, and
    • how much of the target genome/sequence is assembled into contigs. Although in most cases this can be resolved from the context it is potentially confusing for the less experience readers.

    • The solution to resolve heterozygosity by taking a "majority rules" strategy is in principle fine but will inevitably lead to mosaic consensi where two or more haplotypes will be represented within a single scaffold/contig. This is not per se an important issue but we suggest this is clarified in the text.

    • What is the error profile of the sequence added by the gap closer? Is it clearly different from the rest of the assembly in this aspect? Is there some situation in which the use of such a tool is highly inconvenient or this reason? We believe this is an important issue that hasn’t been properly discussed in recent publications. Hopefully as assembly tools improve this point will be raised as a priority.

    • We suggest the authors name files in the submission with a schema that is consistent with the procedure used to generate the data

    • Running scripts depend on some pre-existing tools as SGE, this could be stated for clarity, and although there are options not to use those features it is not necessarily straightforward to figure them out.

    • Some manual tuning is obviously in place (mapping reads with different k sizes, manual creation of the peGrads file). What are the criteria behind the parameters used for these manual steps?

    • Mapping for scaffolding on the v1 YH data has been done on k-mer=45 but gap closing is done on overlap=31:

    • this means that gap closing is done in a more permissive way than scaffolding, which makes sense, but then read mapping on the new reads is done with k=31, why is this?
    • Is there any rule to set values for this parameters?
    • Are the tools used for combination of the mapping being distributed as part of SOAP2.
    • Are the structural errors to the lower "contig/scaffold path NG50" vs. "contig/scaffold N50" ratio compared with the results obtained with ALL-PATHS?

    Level of interest: An article of outstanding merit and interest in its field

    Quality of written English: Acceptable

    Statistical review: No, the manuscript does not need to be seen by a statistician.

    Declaration of competing interests: We declare we have no competing interests. We have ongoing informal collaboration with BGI but not directly working on assembly tools.

    Bernardo Clavijo and Mario Caccamo
    The Genome Analysis Centre
    Norwich Research Park
    Norwich NR4 7UH
    UK

    Published in
    Reviewed by
    Ongoing discussion
  • This paper describes improvements to the original SOAPdenovo genome assembly software by BGI. The improvements are rather minor and I am not convinced that they will result in singificant improvements when the assembler is used on real-life data sets.

    Based on my comments listed below, the paper in its current shape requires much more than a major revision. The authors need to present more and different kinds of results to merit the publication, as well as clarify the presentation. The paper needs to be rewritten and resubmitted. Therefore I recommend that the paper is rejected at this time.

    Major compulsory revisions.

    1. Authors compare the performance of the SOAPdenovo2 to SOAPdenovo1 and Allpaths-LG on faux data set from Assemblathon 1. Faux data usually is much easier to assemble than the real data and the evaluation presented in the subsequent works (e.g. GAGE assembly competition by Salzberg et al., 2011) painted completely different picture. The compariosons of assemblers using faux data have no practical value for determining real-life usability and performance in de novo genome assembly projects. Papers that present genome assembly software must demonstrate the performance of the software on real-life data sets for which finished sequence exists. For example one can use data sets from the GAGE project (Salzberg et al., 2011). The data sets are available at http://gage.cbcb.umd.edu. Alternatively, one can download data for mouse B6 genome available at SRA. Then authors can create an assembly, compate it to the finished sequence and clearly comment on the contiguity, connectivity and correctness of the assembly. For example one can split the assembly at the locatins of all misassemblies and compare the resulting N50 size to the N50 size of the original assembly for both contigs and scaffolds. A comparison to another major software package (such as Allpaths-LG) on the same data set would be a big plus.

    Therefore I ask that authors demonstrate the performance of the SOAPdenovo2 on a real mammalian (one chromosome is sufficient) genome data set and compare the resulting assembly to the finished sequence to evaluate the performance of the assembler.

    1. The performance of SOAPdenovo2 was evaluated against SOAPdenovo1 using YH chomorome data. However, the data used for the two assemblies was different, the new assembly with SOAPdenovo2 has more data in it. Yes, the assembly is better, but would SOAPdenovo1 generate the same improvements with the additional data?

    Minor essential revisions

    1. This paper is written in a style of more of a technical report than a structured scientific publication. Material is presented in haphazard fashion. I would call this paper an initial draft that would require more work to be worthy of a scientific publication. I suggest that the authors divide their paper into more common Introduction -- Results -- Methods -- Discussion sections, or alike.

    2. Authors should clarify what improvements were made to the GapCloser module described on page 4. Are the improvements described implemented in the main assembler code of in the GapCloser? It is not clear from the text.

    Level of interest: An article of limited interest

    Quality of written English: Needs some language corrections before being published

    Statistical review: No, the manuscript does not need to be seen by a statistician.

    Declaration of competing interests: No competing interests.

    Published in
    Reviewed by
    Ongoing discussion
  • This paper introduces an update to the SOAPdenovo program, describes its major improvements and shows improved results on two datasets. We reviewed this paper with a group of four people from the same research group.

    Major Compulsory Revisions

    We have the following major issues with this paper:

    1. The explanation on the improvements in SOAPdenovo2 lack sufficient detail to be able to fully understand them. Papers of this kind usually explain approaches and algorithms used in much more detail. The authors should look at other papers describing new versions of existing software, such as the recent ALLPATHS_LG paper (Ribeiro et al, 2012, http://genome.cshlp.org/content/early/2012/07/24/gr.141515.112.abstract), or even the article describing the first version of SOAP (Li et al, 2009). Improvements of the text are needed so that the reader can understand what changes were implemented and exactly how that improved the program.

    2. Even though we were given access to the underlying raw data, and obtained a pre release versions of SOAPdenovo2 from the authors, we could not replicate the results described in the paper due to a lack of detail in the section on 'Testing and Assessment': the exact commands used for the assemblies are not given.

    3. The article is very biased towards assembly of human genomes. However, SOAPdenovo can be, and often is, used for the assembly of bacterial genomes. The authors use the Assemblathon1 data for their analyses of SOAPdenovo2. In the 'Background' section, the GAGE assembly competition is mentioned, which focusses on comparing programs for assembly of bacterial-sized genomes. However, SOAPdenovo2 was not evaluated against the GAGE data, something we feel is an omission.

    One of us tested SOAPdenovo2 on the Rhodobacter sphaeroides dataset from GAGE, and ran the same analysis script as was used for the GAGE publication (http://gage.cbcb.umd.edu/results/index.html). We have included a summary of this analysis as a PDF attached to this report. From the results, we find the following:

    • SOAPdenovo2, as the first version of the program, still results in many errors in contigs and scaffolds ('corrected' N50's are much lower then N50' values of the sequences generated by SOAPdenovo2)

    • In our tests of the 'sparse assembly graph' approach, a better assembly was obtained by providing a larger estimated genome size then the real size. Do the authors have an explanation for this effect?

    • The 'sparse assembly graph' runs improved uncorrected scaffold sizes, however they resulted in a larger number of scaffolds. Also, the corrected scaffolds N50 of these assemblies were in fact lower than reported in the GAGE article for SOAPdenovo1.

    • We did see an improvement in the contigs from SOAPdenovo2 relative to the first version: fewer errors and higher corrected N50 values, but at the cost of higher contig numbers.

    In conclusion, we do not see significant improvements using SOAPdenovo2 versus the first version of the program on the Rhodobacter dataset. We feel the authors should document the performance of SOAPdenovo2 on small genomes with an available reference genome, for example using the data that was the basis of the GAGE competition.

    1. We also tried SOAPdenovo2 on data from one of our own large eukaryotic genomes. The 'default' version of the program crashed, only when we used the sparse assembly graph version did we get the program running. This may have been due to the fact that we were not able to compile the program on our system, and only could use the provided binaries.

    2. GigaScience's description of a technical note requires 'the code described be documented and tested to high standards.' We did not have access to the source code and can therefore not judge whether the code was well documented. Also, we feel the few tests reported in the paper make us uncertain whether the code can be considered 'tested to high standards' (see also above).

    3. The paper makes many claims that are not referring to any articles or actual data. For example, it is written "Scaffold construction is another area that needs improvement in NGS de novo assembly programs." Can the authors point to some references to back up this claim? Similarly, when discussing the original SOAPdenovo program, the authors give three problematic areas as examples -improperly handling of heterozygous contigs, chimeric scaffolds, false contig relationships. However, no documentation of these problems is provided - real tests of assemblies of datasets with a reference genome where these problems can be shown.

    4. The authors tested new YH 2x100 illumina data with SOAPdenovo2 but failed to show comparable analyses of the same data with the original SOAPdenovo program. To fully elucidate the improvements made from the upgrade to SOAPdenovo2, the authors should report on the analysis of these new YH data with both versions of the program.

    5. The authors used analyses from the assemblathon1 (published February 2011) in their comparison of SOAPdenovo2 with the ALLPATHS_LG program. However, new versions of ALLPATHS_LG have been released since February 2011. As such, we feel that the authors should test the most recent version of ALLPATHS_LG against SOAPdenovo2 (using the same data) to ensure a fair comparison between the two programs.

    Minor Essential Revisions

    1. There is no reference to table 2 in main text

    2. The doi link for reference reference 11 (http://dx.doi.org/10.5524/100038) was not resolving at the time this manuscript was submitted for review.

    Discretionary Revisions

    None

    Level of interest: An article of importance in its field

    Quality of written English: Needs some language corrections before being published

    Statistical review: No, the manuscript does not need to be seen by a statistician.

    Declaration of competing interests: I declare that I have no competing interests.

    Names and affiliations of the reviewers of this report:

    Lex Nederbragt, Ole Kristian Tørresen and Karin Lagesen: Centre for Ecological and Evolutionary Synthesis (CEES), Dept. of Biology, University of Oslo, Oslo, Norway

    Jeremy Chase Crawford (currently guest researcher at CEES): Dept. of Integrative Biology & Museum of Vertebrate Zoology, University of California, Berkeley, USA

    Published in
    Ongoing discussion (81 comments - click to toggle)
    • Mohan.A.Sunkad | 2 years, 7 months ago

      We have read the "Review". Really extensive and possibly tearing the entire manuscript from beginning to end, of course a through check. At the same time "authors" have made a splendid revision work of their manuscript.

    • Felix Jose Gonzalez Gonzalez | 2 years, 3 months ago

      I am a neophyte in programming. However, I have noticed that the corrections are clear and the reasons for these corrections are established in an educated manner.

    • Ramesh Desikan | 1 year, 10 months ago

      This review covers both technical and language aspects in acceptable manner.

    • Prem Kumar Seelam | 1 year, 9 months ago

      Both reviews includes different aspects i.e. language corrections to technical and research part. Useful template for the new peer-reviewers. Thanks to Publons and the reviewers.

    • Sheila Hadden Hinton | 1 year, 9 months ago

      Extremely helpful for me as a new reviewer. The feedback was detailed and easily incorporated as stated in the second review. Publons continues to consistently lay the foundation for objective review processes.

    • Mohamed Hesien | 1 year, 8 months ago

      I really get more informed about the way of pre-puplication review thanks for the examples

    • Noreen Mirza | 1 year, 7 months ago

      very useful to understand the pre publication review process. The reviewers have given thorough comments about content and language. The authors have responded responsibly/.

    • Navnath Bhagchand Tupe | 1 year, 5 months ago

      The pre-publication review is a good and effective lesson to a new reviewer. Detail comments and suggestions are systematically provided.

    • Dr. Muhammad Ujudud Musa | 1 year, 5 months ago

      This is clearly defined and elaborate and is an eye opener to me, thank you very much.

    • Usman Atique | 1 year, 5 months ago

      The review has been made in a splendid way and it can be used as one of the good examples to educate the new reviewers about the rigorous review process. The authors' responses also genuinely address the comments. Indeed a good way to conduct a pre-publication review.

    • Md. Monjurul Hasan | 1 year, 5 months ago

      This is an excellent review done by the reviewers. A lot of things can be learnt from this review.

    • Mohammed, Hafsah Laraba | 1 year, 5 months ago

      This is a very thorough review work. It is a good model.

    • Nzooma Shimaponda-Mataa | 1 year, 5 months ago

      Great review and thorough to ensure quality work

    • RAIMI MORUFU OLALEKAN | 1 year, 5 months ago

      An in-debt review help to bring out the best in any article. Thanks to all reviewer. You are all making an outstanding impact in the global environment. Thanks to all your effort. Keep it up.

    • Habeeb Shuhaib | 1 year, 3 months ago

      I found this example is so beneficial, although it is not in my scientific field . Thanks for publons team .

    • Polinpapilinho F. Katina | 1 year, 3 months ago

      Thanks for sharing

    • Ronaldo da Silva Cruz | 1 year, 3 months ago

      Exemplo excelente!

    • Olha Nesterova | 1 year, 2 months ago

      The editor of a journal and journal team did quite hard job to find so relevant reviewers

    • Aissa Boulmerka | 1 year, 2 months ago

      Thank you!

    • Felix Lirio-Loli | 1 year, 2 months ago

      I have learned a lot from the reviewers

    • Patrick Okon | 1 year, 2 months ago

      The reviewers' comments are detailed and recommend areas that can improve the paper quality. There are many things I have learnt from this.

    • Ammcise Apply | 1 year, 1 month ago

      I have learned a lot from the reviewers' comments although it is not in my scientific field.

    • Oluwafunke O Akinbule | 1 year, 1 month ago

      The reviewer's comments are clear and detailed. A very good example to learn from

    • Ali H A Alwaeli | 1 year ago

      Thank you for this good example to learn from. Your review is valuable and well-structured.

    • Maria Jesús Alonso | 1 year ago

      Me ha gustado que al final sintetice y de respuestas concretas a diversos campos, como el nivel de interés, la calidad del lenguaje, etc.

    • Adnan Qasim | 1 year ago

      a bit extensive and detailed review example to highlight basic points which a pre-publication reviewer should take in account before writing about articles. quite helpful in understanding the points. thanks publons for well written example.

    • Adarshlata Singh | 1 year ago

      Pre publication review by all the reviewers has been done in detail and very precise manner. Clear statements with no ambiguity .

    • Thompson Bolarinwa | 1 year ago

      A well-detailed review. Regards.

    • Fatma Abubaker | 1 year ago

      A good example showing a well constructed review. It covers the Linguistic as well as the technical issues and provides constructive feedback in a clear way. Thank you for this example. As a new peer reviewer, This found it useful.

    • PETER DAVID KULYAKWAVE | 10 months, 3 weeks ago

      For sure I found it very detailed and extremely helpful to me.

    • Hanan M Fathi | 9 months, 3 weeks ago

      Reviewers comments' were clear and focused; point by point. A useful template to learn from it.

    • Ali Raza | 8 months, 2 weeks ago

      Very interesting and fruitful to me. I would like to say thanks for sharing this excellent pre-pub review example. Looking forward to learning more and more.

    • TABASSUM NAWAB | 8 months, 2 weeks ago

      A very good and thorough review with clear instructions to authors...I have learnt a lot from it! thanks for sharing!

    • Soumayya Aib | 8 months, 1 week ago

      The article is reviewed well. This review shows that the reviewer has possessed an epitome of effort for reviewing this article, along with the other reviewer, of course!

    • Kamaliah Mohamad Noh | 7 months, 1 week ago

      Brings to mind the importance of being a subject matter expert when reviewing. As a reviewer, we need to know our limits so that we can give professional reviews which can help the authors improve their publication.

    • Yannick Luther AGBANA | 7 months ago

      These are definitely good examples as teaching materials. I found them very helpful. I want to say thank you for providing them.

    • Mohammed Khayri Aboubeirah | 6 months, 4 weeks ago

      its very useful and This review covers both technical and language aspects in acceptable manner.

    • Pravin Kumar | 6 months, 3 weeks ago

      Although the topic is not from my field of interest but the way reviewer has commented each section is extremely helpful for me as a new reviewer.

    • Ahmed Alqatub | 6 months, 3 weeks ago

      That was very hard and detailed review... I did not face such reviewer before. Good luck

    • Sarla Achuthan | 6 months, 2 weeks ago

      The subject of the research is unknown to me. But i found review very interesting. I could understand the issues in the paper. Detailed and elaborate comments are given in the review. Very useful and valuable review. Good learning.

    • Muhamad Azhar Abdilatef | 6 months, 1 week ago

      Although the field of example's paper is far from me, but I really gathered a good tips from it, thank you very much.

    • Valentine Joseph Owan | 6 months ago

      A wonderful way of learning pre publication review

    • Wisal Hashim Abdulsalam | 6 months ago

      It's a good example to show how a reviewer works as a mentor to the authors.

    • MANUEL SALVADOR MACHADO VILORIA | 6 months ago

      Es una revisión a profundidad que va más allá del simple escrito de los investigadores, con orientaciones y aseveraciones exactas para el fortalecimiento del artículo d investigación. Me han aclarado ideas que tenía en el momento de evaluar y dar opiniones para mejorar en la complejidad investigativa.-

    • ALI ABBAS HASHIM ALMUSAWI | 5 months, 4 weeks ago

      I really get more informed about the way of pre-puplication review thanks for the examples

    • Victor Ighariemu | 5 months, 3 weeks ago

      it was a good review

    • Agbortoko Ashu | 5 months, 3 weeks ago

      Great review

    • Rabea Jamil Mahfoud | 5 months, 2 weeks ago

      I admire the reviewer who really takes care of the paper's details regarding structure, methodologies, format, and language. It's not easy at all to review a scientific article with such professional way. Great review !

    • Asha Durafe | 5 months, 2 weeks ago

      Excellent review!

    • PREMA D'CUNHA | 5 months, 1 week ago

      A real detailed review which showed me how to do a pre Publication review. Very helpful.

    • Dr. Balasubramani R | 5 months, 1 week ago

      Very detailed pre-publication review example. Learnt many useful tips. Will be helpful to write reviews for papers in my area.

    • Mohammed Al-Rawi | 5 months, 1 week ago

      Although the topic is outside my scientific field, but I think the two reviews cover most pre-publication considerations..it is a wonderful example..thank you..

    • Tarek Mami | 5 months, 1 week ago

      Good example

    • Dr. Saima Eman | 5 months, 1 week ago

      I think knowing the names and affiliations of reviewers might help reduce bias but it may also be problematic for the reviewers.

    • Maaz Alata | 5 months, 1 week ago

      This is an excellent model for teaching the pre publication review. It was extensive reviews affects the main stem of the manuscript, however the authors did a hard job to correct and responds in a very scientific way.

    • Dr. Chetan Panchal | 5 months, 1 week ago

      Good. Point to point remarks can ease author for further action

    • Tetiana Hranchak | 5 months, 1 week ago

      Thanks for providing good examples of pre-publication reviews. Clear approaches to the evaluation of the manuscript, the structure of reviews, detailing comments.

    • Francianne Mourão | 5 months, 1 week ago

      Very useful to understand the pre publication review process. The reviewers have given thorough comments about content and language. The authors have responded responsibly

    • Sunil Kumar Gupta | 5 months, 1 week ago

      A well-framed pre-publication review for all.

    • Olga Predushchenko | 5 months, 1 week ago

      ​There are effective pre-publication reviews with clear structure and strong arguments. The authors have a good chance to improve their article.

    • Reem Abou Assi | 5 months, 1 week ago

      This is an ideal per-reviewing! Even if the authors got a rejection, they will still benefit from the "in details" and "point to point" discussion to improve their paper quality and increase its impact factor. Usually, such reviews are received from high impact factor journals. Thanks for sharing the knowledge.

    • zinah wA | 5 months, 1 week ago

      This is an excellent review done by the reviewers. It's not easy at all to review a scientific article with such professional way.

    • tarek el-desouky | 5 months, 1 week ago

      help the authors to publish the article in the best image

    • DENNIS ZAMI ATIBUNI | 5 months, 1 week ago

      The reviewers did a very thorough job, the authors made corrections as recommended, the re-reviewed work exhibited a far better quality. One learns the need for thoroughness in content mastery and doing the job in a candid but respectful manner.

    • Laly Antoney | 5 months, 1 week ago

      A peer-review of its kind and model to be used as a reference to those who wish to be trained as reviewers. The review is conclusive and critical in technically, contentwise, and linguistically. Reviewers posit their arguments/observations in a highly polite and professional way. The review is a demonstration of thorough knowledge of reviewers in the research area.

    • Ramkrishna Kadam | 5 months, 1 week ago

      Lot of things which I have learned from this review.

    • Morolake Dairo | 5 months, 1 week ago

      This is a very detailed review, I admire the use of points and references to the main text. It helps the author identify what is being discussed.

      Rather than focus on other editing issues like grammar or spelling; the reviewers focus on the idea of the paper and tackle that.

      Alot of work has gone into this review and it shows interest in the work of the author.

    • Mudassar Hussain | 5 months, 1 week ago

      A detailed review almost cover each part which certainly helpful for the novel reviewers.

    • Walter Lintangah | 5 months ago

      A good example of a review.

    • Gutierrez Alejandra C. | 5 months ago

      The review is very complete, I like the responses of the reviewer to improve the paper.

    • Pablo Vieira Rego | 5 months ago

      Very helpful. Thanks.

    • Hazim Abdul Rahman Alhiti | 5 months ago

      EXCELLENT AND INFORMATIVE EXAMPLE

    • Joshua Offe Berkoh | 5 months ago

      This review covers both technical and language aspects in an acceptable manner and this is clearly defined and elaborate and is an eye opener to me, thank you very much.

    • Mónica Montaño Garcés | 5 months ago

      Very detailed review, offering the authors the weak points of their work, which will allow them to correct and present research with greater scientific solvency. Thank you so much.

    • OGUNTUASE Mary Aderonke | 5 months ago

      I can now understand that accepting the invitation to be a peer reviewer requires high sense of responsibility and to be very vast and versatile in one's field of expertise. I now understand the difference between pre and post publication reviews. I appreciate Publons Academy.

    • Bernadine Nsa Ekpenyong | 5 months ago

      detailed and constructive review

    • Mohamed Eldeeb | 4 months, 3 weeks ago

      the review is very clear and to the point

    • Onuoha Chidiebere (PhD) | 4 months, 3 weeks ago

      The review has taught me that all aspect of an article should be constructively assessed. With the singular aim to strengthen it and make it more appealing in its field.This is insightful. Thanks Publons for the good job.

    • P. Senthil Kumari | 4 months, 3 weeks ago

      Good examples are shown for us to learn and improve a lot. Thank you publon for your quality work while considering new reveiewers like me.

    • Emmanuel N Barthalomew | 4 months, 3 weeks ago

      A very constructive, detailed and thorough review covering all needed aspects with subsequent suggestions on how to address the issues identified.

    • Sugianto | 4 months, 3 weeks ago

      Very good review to be used as an example, even though it's not in my field. Thank you

All peer review content displayed here is covered by a Creative Commons CC BY 4.0 license.