Content of review 1, reviewed on August 15, 2012

This paper introduces an update to the SOAPdenovo program, describes its major improvements and shows improved results on two datasets. We reviewed this paper with a group of four people from the same research group.

Major Compulsory Revisions

We have the following major issues with this paper:

  1. The explanation on the improvements in SOAPdenovo2 lack sufficient detail to be able to fully understand them. Papers of this kind usually explain approaches and algorithms used in much more detail. The authors should look at other papers describing new versions of existing software, such as the recent ALLPATHS_LG paper (Ribeiro et al, 2012,, or even the article describing the first version of SOAP (Li et al, 2009). Improvements of the text are needed so that the reader can understand what changes were implemented and exactly how that improved the program.

  2. Even though we were given access to the underlying raw data, and obtained a pre release versions of SOAPdenovo2 from the authors, we could not replicate the results described in the paper due to a lack of detail in the section on 'Testing and Assessment': the exact commands used for the assemblies are not given.

  3. The article is very biased towards assembly of human genomes. However, SOAPdenovo can be, and often is, used for the assembly of bacterial genomes. The authors use the Assemblathon1 data for their analyses of SOAPdenovo2. In the 'Background' section, the GAGE assembly competition is mentioned, which focusses on comparing programs for assembly of bacterial-sized genomes. However, SOAPdenovo2 was not evaluated against the GAGE data, something we feel is an omission.

One of us tested SOAPdenovo2 on the Rhodobacter sphaeroides dataset from GAGE, and ran the same analysis script as was used for the GAGE publication ( We have included a summary of this analysis as a PDF attached to this report. From the results, we find the following:

  • SOAPdenovo2, as the first version of the program, still results in many errors in contigs and scaffolds ('corrected' N50's are much lower then N50' values of the sequences generated by SOAPdenovo2)

  • In our tests of the 'sparse assembly graph' approach, a better assembly was obtained by providing a larger estimated genome size then the real size. Do the authors have an explanation for this effect?

  • The 'sparse assembly graph' runs improved uncorrected scaffold sizes, however they resulted in a larger number of scaffolds. Also, the corrected scaffolds N50 of these assemblies were in fact lower than reported in the GAGE article for SOAPdenovo1.

  • We did see an improvement in the contigs from SOAPdenovo2 relative to the first version: fewer errors and higher corrected N50 values, but at the cost of higher contig numbers.

In conclusion, we do not see significant improvements using SOAPdenovo2 versus the first version of the program on the Rhodobacter dataset. We feel the authors should document the performance of SOAPdenovo2 on small genomes with an available reference genome, for example using the data that was the basis of the GAGE competition.

  1. We also tried SOAPdenovo2 on data from one of our own large eukaryotic genomes. The 'default' version of the program crashed, only when we used the sparse assembly graph version did we get the program running. This may have been due to the fact that we were not able to compile the program on our system, and only could use the provided binaries.

  2. GigaScience's description of a technical note requires 'the code described be documented and tested to high standards.' We did not have access to the source code and can therefore not judge whether the code was well documented. Also, we feel the few tests reported in the paper make us uncertain whether the code can be considered 'tested to high standards' (see also above).

  3. The paper makes many claims that are not referring to any articles or actual data. For example, it is written "Scaffold construction is another area that needs improvement in NGS de novo assembly programs." Can the authors point to some references to back up this claim? Similarly, when discussing the original SOAPdenovo program, the authors give three problematic areas as examples -improperly handling of heterozygous contigs, chimeric scaffolds, false contig relationships. However, no documentation of these problems is provided - real tests of assemblies of datasets with a reference genome where these problems can be shown.

  4. The authors tested new YH 2x100 illumina data with SOAPdenovo2 but failed to show comparable analyses of the same data with the original SOAPdenovo program. To fully elucidate the improvements made from the upgrade to SOAPdenovo2, the authors should report on the analysis of these new YH data with both versions of the program.

  5. The authors used analyses from the assemblathon1 (published February 2011) in their comparison of SOAPdenovo2 with the ALLPATHS_LG program. However, new versions of ALLPATHS_LG have been released since February 2011. As such, we feel that the authors should test the most recent version of ALLPATHS_LG against SOAPdenovo2 (using the same data) to ensure a fair comparison between the two programs.

Minor Essential Revisions

  1. There is no reference to table 2 in main text

  2. The doi link for reference reference 11 ( was not resolving at the time this manuscript was submitted for review.

Discretionary Revisions


Level of interest: An article of importance in its field

Quality of written English: Needs some language corrections before being published

Statistical review: No, the manuscript does not need to be seen by a statistician.

Declaration of competing interests: I declare that I have no competing interests.

Names and affiliations of the reviewers of this report:

Lex Nederbragt, Ole Kristian Tørresen and Karin Lagesen: Centre for Ecological and Evolutionary Synthesis (CEES), Dept. of Biology, University of Oslo, Oslo, Norway

Jeremy Chase Crawford (currently guest researcher at CEES): Dept. of Integrative Biology & Museum of Vertebrate Zoology, University of California, Berkeley, USA


    © 2012 the Reviewer (CC-BY 4.0 - source).

Content of review 2, reviewed on November 30, 2012

The authors have done a good job responding to all the criticism. We thank them for the changes and additions they made to the manuscript. We particularly appreciate the effort that went into the supplementary material.

We therefore recommend to accept the manuscript for publication, pending some minor revisions.

Major remarks:

The authors write in their response "We will release the source code of SOAPdenovo2 as soon as the paper is accepted." We still feel strongly that the reviewers should have been given access to the source code of the program(s). However, we decided not to reject the paper because of this omission.

The authors write on page 5 "Notably, SOAPdenovo1.05 was released two years after SOAPdenovo1 and already included several improvements and new features from SOAPdenovo2, including the new contig and scaffold construction improvements, but without the new error correction and gap closure modules. " We are wondering whether it then is fair to compare SOAPdenovo2 only with SOAPdenovo1. The authors should have included SOAPdenovo1.05 as well for testing the new 100bp PE reads. At the very least they should acknowledge the limits of their comparison.

Minor Essential Revisions

Main text

  • page 3, in the sentence "However, the error correction module in SOAPdenovo was designed for short Illumina reads (35-50bp), which consumes excessive amount of computational time and memory on longer reads, say over 150GB memory running for two days using 40x 100bp paired-end Illumina HiSeq 2000 reads." It is not clear what is referred to for the 40x. Is this a genome the size of the human genome?

  • page 5: "The SOAPdenovo2 assembly also had a much lower amount of copy number errors, but did have more substitution errors." Please include a description on how these measures are calculated

  • please be careful to keep the colors consistent among figure 1 and 2, to avoid confusion

  • table 1: Copy Number Error-rate --> how is this defined, what is the unit?

  • table 2: please indicate v1 or v2 for the 'version' column of each row; please explain 'scaffold size'

  • both tables: what are the units? (usually bp)


  • could page numbers be included in the table of contents of the supplementary material?

  • section 1: the text mentions 'SOAPec-1.0' and 'SOAPec-2.0', however, these terms are used nowhere else. Please be consistent

  • section 1: "The algorithm is based on k-mer frequency spectrums (i.e. KFS), but the algorithm is quite different from other KFS tools" Please reference these other tools.

  • section 7: "As shown in Table 2, the scaffold N50 of SOAPdenovo2 overwhelmed ALLPATHS-LG and increased more than 4-fold compare to SOAPdenovo. But the contig N50 of ALLPATHS-LG is the longest" We object to the word 'overwhelmed'. There is a difference, but it is not overwhelming. In addition, the larger contig N50 of Allpaths is equally 'overwhelming' soapdenovo2… Please rephrase.

  • section 7: "However, the contig N50 could be further improved for SOAPdenovo2 by using 3’-end connected reads and larger k-mer size as ALLPATHS-LG do." Why didn't the authors try this then?

Discretionary Revisions

  • we recommend the authors to ask a native English speaker to have a look at the text, especially of the Supplementary material

Level of interest: An article of importance in its field

Quality of written English: Needs some language corrections before being published

Statistical review: No, the manuscript does not need to be seen by a statistician.

Declaration of competing interests: I declare that I have no competing interests


    © 2012 the Reviewer (CC-BY 4.0 - source).

Comments   (Guidelines)


8:23 a.m., 8 Mar 18 (UTC) | Link

We have read the "Review". Really extensive and possibly tearing the entire manuscript from beginning to end, of course a through check. At the same time "authors" have made a splendid revision work of their manuscript.

Felix Jose Gonzalez Gonzalez

3:56 a.m., 8 Jul 18 (UTC) | Link

I am a neophyte in programming. However, I have noticed that the corrections are clear and the reasons for these corrections are established in an educated manner.

Ramesh Desikan

4:16 p.m., 15 Dec 18 (UTC) | Link

This review covers both technical and language aspects in acceptable manner.

Prem Kumar Seelam

10:07 a.m., 2 Jan 19 (UTC) | Link

Both reviews includes different aspects i.e. language corrections to technical and research part. Useful template for the new peer-reviewers. Thanks to Publons and the reviewers.

Sheila Hadden Hinton

11:28 p.m., 5 Jan 19 (UTC) | Link

Extremely helpful for me as a new reviewer. The feedback was detailed and easily incorporated as stated in the second review. Publons continues to consistently lay the foundation for objective review processes.

Mohamed Hesien

8:52 a.m., 8 Feb 19 (UTC) | Link

I really get more informed about the way of pre-puplication review thanks for the examples

Noreen Mirza

10:05 a.m., 1 Mar 19 (UTC) | Link

very useful to understand the pre publication review process. The reviewers have given thorough comments about content and language. The authors have responded responsibly/.

Navnath Bhagchand Tupe

3:40 p.m., 30 Apr 19 (UTC) | Link

The pre-publication review is a good and effective lesson to a new reviewer. Detail comments and suggestions are systematically provided.

Dr. Muhammad Ujudud Musa

5:48 p.m., 2 May 19 (UTC) | Link

This is clearly defined and elaborate and is an eye opener to me, thank you very much.

Usman Atique

2:31 a.m., 3 May 19 (UTC) | Link

The review has been made in a splendid way and it can be used as one of the good examples to educate the new reviewers about the rigorous review process. The authors' responses also genuinely address the comments. Indeed a good way to conduct a pre-publication review.

Md. Monjurul Hasan

10:40 a.m., 4 May 19 (UTC) | Link

This is an excellent review done by the reviewers. A lot of things can be learnt from this review.

Mohammed, Hafsah Laraba

5:21 p.m., 5 May 19 (UTC) | Link

This is a very thorough review work. It is a good model.

Nzooma Shimaponda-Mataa

2:15 p.m., 10 May 19 (UTC) | Link

Great review and thorough to ensure quality work


9:23 a.m., 14 May 19 (UTC) | Link

An in-debt review help to bring out the best in any article. Thanks to all reviewer. You are all making an outstanding impact in the global environment. Thanks to all your effort. Keep it up.

Habeeb Shuhaib

5:32 a.m., 28 Jun 19 (UTC) | Link

I found this example is so beneficial, although it is not in my scientific field . Thanks for publons team .

Polinpapilinho F. Katina

4:40 p.m., 3 Jul 19 (UTC) | Link

Thanks for sharing

Ronaldo da Silva Cruz

11:35 a.m., 22 Jul 19 (UTC) | Link

Exemplo excelente!

Olha Nesterova

11:49 a.m., 4 Aug 19 (UTC) | Link

The editor of a journal and journal team did quite hard job to find so relevant reviewers

Aissa Boulmerka

3:50 p.m., 5 Aug 19 (UTC) | Link

Thank you!

Felix Lirio-Loli

1:47 a.m., 15 Aug 19 (UTC) | Link

I have learned a lot from the reviewers

Patrick Okon

5:52 p.m., 17 Aug 19 (UTC) | Link

The reviewers' comments are detailed and recommend areas that can improve the paper quality. There are many things I have learnt from this.

Ammcise Apply

1:40 a.m., 19 Sep 19 (UTC) | Link

I have learned a lot from the reviewers' comments although it is not in my scientific field.

Oluwafunke O Akinbule

12:47 p.m., 19 Sep 19 (UTC) | Link

The reviewer's comments are clear and detailed. A very good example to learn from

Ali H A Alwaeli

10:04 a.m., 25 Sep 19 (UTC) | Link

Thank you for this good example to learn from. Your review is valuable and well-structured.

Maria Jesús Alonso

10 p.m., 29 Sep 19 (UTC) | Link

Me ha gustado que al final sintetice y de respuestas concretas a diversos campos, como el nivel de interés, la calidad del lenguaje, etc.

Adnan Qasim

3:11 p.m., 11 Oct 19 (UTC) | Link

a bit extensive and detailed review example to highlight basic points which a pre-publication reviewer should take in account before writing about articles. quite helpful in understanding the points. thanks publons for well written example.

Adarshlata Singh

5:28 p.m., 12 Oct 19 (UTC) | Link

Pre publication review by all the reviewers has been done in detail and very precise manner. Clear statements with no ambiguity .

Thompson Bolarinwa

8:06 a.m., 17 Oct 19 (UTC) | Link

A well-detailed review. Regards.

Fatma Abubaker

9:18 a.m., 17 Oct 19 (UTC) | Link

A good example showing a well constructed review. It covers the Linguistic as well as the technical issues and provides constructive feedback in a clear way. Thank you for this example. As a new peer reviewer, This found it useful.


7:25 a.m., 29 Nov 19 (UTC) | Link

For sure I found it very detailed and extremely helpful to me.

Hanan M Fathi

6:18 a.m., 1 Jan 20 (UTC) | Link

Reviewers comments' were clear and focused; point by point. A useful template to learn from it.

Ali Raza

7:08 a.m., 6 Feb 20 (UTC) | Link

Very interesting and fruitful to me. I would like to say thanks for sharing this excellent pre-pub review example. Looking forward to learning more and more.


6:26 a.m., 11 Feb 20 (UTC) | Link

A very good and thorough review with clear instructions to authors...I have learnt a lot from it! thanks for sharing!

Soumayya Aib

6:43 a.m., 13 Feb 20 (UTC) | Link

The article is reviewed well. This review shows that the reviewer has possessed an epitome of effort for reviewing this article, along with the other reviewer, of course!

Kamaliah Mohamad Noh

8:26 a.m., 18 Mar 20 (UTC) | Link

Brings to mind the importance of being a subject matter expert when reviewing. As a reviewer, we need to know our limits so that we can give professional reviews which can help the authors improve their publication.

Yannick Luther AGBANA

12:41 p.m., 23 Mar 20 (UTC) | Link

These are definitely good examples as teaching materials. I found them very helpful. I want to say thank you for providing them.

Mohammed Khayri Aboubeirah

8:25 p.m., 27 Mar 20 (UTC) | Link

its very useful and This review covers both technical and language aspects in acceptable manner.

Pravin Kumar

4:25 p.m., 29 Mar 20 (UTC) | Link

Although the topic is not from my field of interest but the way reviewer has commented each section is extremely helpful for me as a new reviewer.

Ahmed Alqatub

9:29 p.m., 30 Mar 20 (UTC) | Link

That was very hard and detailed review... I did not face such reviewer before. Good luck

Sarla Achuthan

10:35 a.m., 8 Apr 20 (UTC) | Link

The subject of the research is unknown to me. But i found review very interesting. I could understand the issues in the paper. Detailed and elaborate comments are given in the review. Very useful and valuable review. Good learning.

Muhamad Azhar Abdilatef

6:21 p.m., 17 Apr 20 (UTC) | Link

Although the field of example's paper is far from me, but I really gathered a good tips from it, thank you very much.

Valentine Joseph Owan

7:59 a.m., 20 Apr 20 (UTC) | Link

A wonderful way of learning pre publication review

Wisal Hashim Abdulsalam

1:04 p.m., 22 Apr 20 (UTC) | Link

It's a good example to show how a reviewer works as a mentor to the authors.


11:43 p.m., 22 Apr 20 (UTC) | Link

Es una revisión a profundidad que va más allá del simple escrito de los investigadores, con orientaciones y aseveraciones exactas para el fortalecimiento del artículo d investigación. Me han aclarado ideas que tenía en el momento de evaluar y dar opiniones para mejorar en la complejidad investigativa.-


2:37 p.m., 25 Apr 20 (UTC) | Link

I really get more informed about the way of pre-puplication review thanks for the examples

Victor Ighariemu

9:39 p.m., 29 Apr 20 (UTC) | Link

it was a good review

Agbortoko Ashu

5:57 a.m., 4 May 20 (UTC) | Link

Great review

Rabea Jamil Mahfoud

4:38 p.m., 7 May 20 (UTC) | Link

I admire the reviewer who really takes care of the paper's details regarding structure, methodologies, format, and language. It's not easy at all to review a scientific article with such professional way. Great review !

Asha Durafe

3:20 p.m., 8 May 20 (UTC) | Link

Excellent review!


8:57 a.m., 12 May 20 (UTC) | Link

A real detailed review which showed me how to do a pre Publication review. Very helpful.

Dr. Balasubramani R

10:02 a.m., 12 May 20 (UTC) | Link

Very detailed pre-publication review example. Learnt many useful tips. Will be helpful to write reviews for papers in my area.

Mohammed Al-Rawi

12:49 p.m., 12 May 20 (UTC) | Link

Although the topic is outside my scientific field, but I think the two reviews cover most pre-publication is a wonderful example..thank you..

Tarek Mami

4:53 p.m., 12 May 20 (UTC) | Link

Good example

Dr. Saima Eman

6:53 p.m., 12 May 20 (UTC) | Link

I think knowing the names and affiliations of reviewers might help reduce bias but it may also be problematic for the reviewers.

Maaz Alata

9:56 p.m., 12 May 20 (UTC) | Link

This is an excellent model for teaching the pre publication review. It was extensive reviews affects the main stem of the manuscript, however the authors did a hard job to correct and responds in a very scientific way.

Dr. Chetan Panchal

5:58 a.m., 13 May 20 (UTC) | Link

Good. Point to point remarks can ease author for further action

Tetiana Hranchak

8:16 a.m., 13 May 20 (UTC) | Link

Thanks for providing good examples of pre-publication reviews. Clear approaches to the evaluation of the manuscript, the structure of reviews, detailing comments.

Francianne Mourão

1:05 p.m., 13 May 20 (UTC) | Link

Very useful to understand the pre publication review process. The reviewers have given thorough comments about content and language. The authors have responded responsibly

Sunil Kumar Gupta

3:19 p.m., 13 May 20 (UTC) | Link

A well-framed pre-publication review for all.

Olga Predushchenko

3:45 p.m., 13 May 20 (UTC) | Link

​There are effective pre-publication reviews with clear structure and strong arguments. The authors have a good chance to improve their article.

Reem Abou Assi

6:46 a.m., 14 May 20 (UTC) | Link

This is an ideal per-reviewing! Even if the authors got a rejection, they will still benefit from the "in details" and "point to point" discussion to improve their paper quality and increase its impact factor. Usually, such reviews are received from high impact factor journals. Thanks for sharing the knowledge.

zinah wA

12:32 p.m., 14 May 20 (UTC) | Link

This is an excellent review done by the reviewers. It's not easy at all to review a scientific article with such professional way.

tarek el-desouky

3:53 a.m., 15 May 20 (UTC) | Link

help the authors to publish the article in the best image


8:58 a.m., 16 May 20 (UTC) | Link

The reviewers did a very thorough job, the authors made corrections as recommended, the re-reviewed work exhibited a far better quality. One learns the need for thoroughness in content mastery and doing the job in a candid but respectful manner.

Laly Antoney

1:35 p.m., 16 May 20 (UTC) | Link

A peer-review of its kind and model to be used as a reference to those who wish to be trained as reviewers. The review is conclusive and critical in technically, contentwise, and linguistically. Reviewers posit their arguments/observations in a highly polite and professional way. The review is a demonstration of thorough knowledge of reviewers in the research area.

Ramkrishna Kadam

3:19 p.m., 16 May 20 (UTC) | Link

Lot of things which I have learned from this review.

Morolake Dairo

4:58 p.m., 16 May 20 (UTC) | Link

This is a very detailed review, I admire the use of points and references to the main text. It helps the author identify what is being discussed.

Rather than focus on other editing issues like grammar or spelling; the reviewers focus on the idea of the paper and tackle that.

Alot of work has gone into this review and it shows interest in the work of the author.

Mudassar Hussain

3:31 p.m., 17 May 20 (UTC) | Link

A detailed review almost cover each part which certainly helpful for the novel reviewers.

Walter Lintangah

7:23 a.m., 18 May 20 (UTC) | Link

A good example of a review.

Gutierrez Alejandra C.

7:11 p.m., 18 May 20 (UTC) | Link

The review is very complete, I like the responses of the reviewer to improve the paper.

Pablo Vieira Rego

2:45 p.m., 20 May 20 (UTC) | Link

Very helpful. Thanks.

Hazim Abdul Rahman Alhiti

1:28 a.m., 21 May 20 (UTC) | Link


Joshua Offe Berkoh

7:20 a.m., 22 May 20 (UTC) | Link

This review covers both technical and language aspects in an acceptable manner and this is clearly defined and elaborate and is an eye opener to me, thank you very much.

Mónica Montaño Garcés

8:02 p.m., 23 May 20 (UTC) | Link

Very detailed review, offering the authors the weak points of their work, which will allow them to correct and present research with greater scientific solvency. Thank you so much.

OGUNTUASE Mary Aderonke

8:42 p.m., 23 May 20 (UTC) | Link

I can now understand that accepting the invitation to be a peer reviewer requires high sense of responsibility and to be very vast and versatile in one's field of expertise. I now understand the difference between pre and post publication reviews. I appreciate Publons Academy.

Bernadine Nsa Ekpenyong

10:32 p.m., 24 May 20 (UTC) | Link

detailed and constructive review

Mohamed Eldeeb

1:29 p.m., 27 May 20 (UTC) | Link

the review is very clear and to the point

Onuoha Chidiebere (PhD)

12:59 p.m., 30 May 20 (UTC) | Link

The review has taught me that all aspect of an article should be constructively assessed. With the singular aim to strengthen it and make it more appealing in its field.This is insightful. Thanks Publons for the good job.

P. Senthil Kumari

1:48 p.m., 30 May 20 (UTC) | Link

Good examples are shown for us to learn and improve a lot. Thank you publon for your quality work while considering new reveiewers like me.

Emmanuel N Barthalomew

5:39 a.m., 31 May 20 (UTC) | Link

A very constructive, detailed and thorough review covering all needed aspects with subsequent suggestions on how to address the issues identified.


2:54 a.m., 2 Jun 20 (UTC) | Link

Very good review to be used as an example, even though it's not in my field. Thank you

Please log in to leave a comment.


    Ruibang, L., Binghang, L., Yinlong, X., Zhenyu, L., Weihua, H., Jianying, Y., Guangzhu, H., Yanxiang, C., Qi, P., Yunjie, L., Jingbo, T., Gengxiong, W., Hao, Z., Yujian, S., Yong, L., Chang, Y., Bo, W., Yao, L., Changlei, H., W., C. D., Siu-Ming, Y., Shaoliang, P., Xiaoqian, Z., Guangming, L., Xiangke, L., Yingrui, L., Huanming, Y., Jian, W., Tak-Wah, L., Jun, W. 2012. SOAPdenovo2: an empirically improved memory-efficient short-read de novo assembler. GigaScience.