Content of review 1, reviewed on September 07, 2019

This paper examined the effectiveness of an online argumentation model for improving undergraduate students’ argumentation ability and related content knowledge. This manuscript presents an interesting study. However, there are a number of theoretical and methodological issues that need attention. The following comments summarize my concerns regarding this manuscript.

Introduction and literature review:
1.The most serious issue in this paper is that the research questions (RQs) and independent variable were unclear. The authors stated that “In the experimental group, the IPadE model with the WIAS was adopted, and the control group used a face-to-face traditional method with the IPadE training model.” (p.9). I agree with the comment of Reviewer 1 “Since both groups followed the IPadE learning paradigm, it seems to me then that the reason for the pattern of the results is confounded …” in last review. This comment showed that the difference between experimental and comparison groups was the factor that students used online learning (WIAS) instead of the IPadE model. The fact that both groups followed the IPadE learning made RQ1 and RQ2 invalid. This problem remains in this version of manuscript.
2.The idea of “IPadE model” and “reasoning modes” were not mentioned and discussed in the Introduction but appeared in the RQs. This is an abrupt writing style for readers.
3.The research gap in this domain should be elaborated in Introduction section (p.3). That is to say, the authors should explain what is lacking in previous studies.
4.There have been many argumentation studies in recent years. Authors should update the literature in the last three years. Moreover, the authors stated that “We reviewed and compared ... see Table 1” (p.5). The authors are encouraged to discuss the Table 1 in more detail.
5.Some abbreviations were not defined clearly, such as SSI, PH, NH, NHI… Abbreviations should list complete terms in the first use.
6.The statement of “Many researchers have …” (p.6) should cite the source(s).
7.The sentences of “… and clarifying their perspectives Questions raised during the discussion process” (p.7) need the period.
8.Table 2, Figure 1, and Figure 2 were just listed but not discussed adequately.
9.The IPadE model was not discussed clearly in the Literature review section. The process of this model is similar to the proposed model of “argument-critique-argument approach” proposed by Yu and Yore (2013). The authors are encouraged to provide more explanation for the insight which this model had for the argumentation research. Moreover, the authors stated that “The IPadE model was based on the fundamental framework of the argumentation model proposed by Lakatos” (p.8). As far as I know, IPadE model was designed for the guiding process of the argumentation while Lakatos framework was used to justify the quality of arguments. The authors should clarify these two models clearly in the manuscript.

Methodology:
1.The authors stated that “Cronbach alpha for the HKT was 0.65” (p.9). The measure of internal consistency reliability for measures with dichotomous choices should be KR20.
2.The authors stated that “Lakatos as a framework … including the HC (the claim and supporting reasons), PH (the alternative idea to expand the HC), and NH (the statement mentioning the rebuttal or limitation of the HC)” (p.9). According to authors’ definition, the rebuttal is one form of negative heuristic (NH) (also noted in Figure 3). However, there were five kinds of scores in the Argumentation Abilities Checklist (AAC) (Table 3), which were the HC (claim and supporting reasons), PH, NH, and rebuttal. This is an inconsistency between framework and scoring of the AAC. Moreover, according to Table 3, if the participant responses “I agree” or “I disagree”, he or she can get the 9 points for “claim” part. It would be more difficult to get 9 points for the other four parts. The scoring weight for the five parts in the AAC was not equal.
3.The System 1 and 2 in Table 4 should be noted.

Results:
1.Repeated Measures ANOVA is often used to test differences in mean scores under three or more different conditions while Paired Sample t-test is used under two different conditions within a single sample (https://statistics.laerd.com/statistical-guides/repeated-measures-anova-statistical-guide.php). One the other hand, in experimental designs, analysis of covariance (ANCOVA) is often used to test the main and interaction effects (treatment) between two or more groups. Therefore, Repeated Measures ANOVA and Paired Sample t-test are used to compare the same group while ANCOVA is used to compare different groups. The authors are encouraged to consider again if their data analyses were suitable.
2.The statement of “To assess the ... from 0.90–0.93 (p < 0.01)” was repeated.
3.The analyses of “changes in students’ use of arguments modes” should have the theoretical foundation discussed in the Literature review section. How to categorize and define the “arguments modes” may be discussed in more detail.

Conclusion:
1.This subtitle should be “Conclusions and discussions”.
2.I think the most different part of IPadE model, comparing with the previous argumentation learning approach, is the use of peer-assessment. However, the effect of peer-assessment was not discussed adequately.

Source

    © 2019 the Reviewer.

References

    Ya-Ching, F., Tzu-Hua, W., Kuo-Hua, W. 2020. Studying the effectiveness of an online argumentation model for improving undergraduate students' argumentation ability. Journal of Computer Assisted Learning.