Content of review 1, reviewed on May 14, 2020
The manuscript reports on a study in which the impact of virtual reality for foreign language learning was tested. Participants studied Finnish words in three encoding conditions: Motor manipulation of the virtual object, Irrelevant motor action towards the object, and only watching at the virtual object in its real-like context.
The experimental protocol and the virtual task are interesting and well designed, however the manuscript needs substantial changes in order to provide a clear and coherent view of the findings.
I will make some recommendations here below.
- ABSTRACT: it is missing in my pdf, may be it was an error inside the system
-INTRODUCTION: the first part is clear, but later authors lost a logical thread. When VR comes into play, they started going back and forth with similar argumentation describing the features that make VR a potential useful tool in this learning task. The VR section is quite confused, as it seems that authors aimed to summarize all the features that in literature have been reported as potentially useful for learning, without a clear goal. Instead, in my opinion, authors should discuss the features that make VR a useful tool for learning within an embodied approach, which was the one that framed their research.
On page 5 line 26, please replace the unpublished work by Repetto (which seems to have inspired a lot the introduction section…) with a published one, reporting the same concept (Repetto, C., Colombo, B., & Riva, G. (2015). Is motor simulation involved during foreign language learning? A virtual reality experiment. SAGE Open, 5(4), 2158244015609964.).
Later one, the concepts of presence and multimodal processing are introduced, but without any definition or explanation. Please elaborate more on these.
- PARTICIPANT: 22+22= forty-four, not forty-six as reported
- page 11 line 50: FIG2B doesn’t match with the text
- PROCEDURE: only 2 repetitions of the same item during encoding are very little for leanring a new vocabulary. This choice should be motivated considering that usually in previous studies on enactment repetitions were much more (and training much longer). This could also explain the low learning rate and low retention after 1 week.
- STATISTICAL APPROACH: unfortunately the statistical approach is not suited for this paradigm. As suggested by Jannsen (Janssen, D. P. (2012). Twice random, once mixed: Applying mixed models to simultaneously analyze random effects of language and participants. Behavior Research Methods, 44(1), 232-247.), in language studies there are 2 sources of random effects, participants and words, and as such the correct approach is to run Linear Mixed Models . I think it is necessary to re-run the analyses with this approach.
-DISCUSSION: comments on the present discussion might be irrelevant since new results could be different and require to review substantially also this section. For what I read now, authors tended to overstate some points: i.e. page 20, “VR significantly improves vocabulary learning” → actually according to the present findings you can only say that Motor interaction is better than irrelevant movement, but not that it is better than watching (results on this comparison are not reported). Importantly, this was a recognition task, not a recall. More: “manipulation was favourable to Watch-only condition in step 2”, this statement is not statistically supported as the difference was not significant; again “A week after the learning session, a true to-life simulation of object manipulation promoted better memory of object names in a foreign language, compared to an impoverished simulation”, this is overstated since in the end participants recalled less than 1 word in the follow-up (which means 0 words, being words counted as integer values). Once the new analyses will be performed I could make a more accurate judgment on the discussion.
Source
© 2020 the Reviewer.
Content of review 2, reviewed on October 06, 2020
I thank the authors for having considered all the suggestions I made.
The manuscript improved upon revision, but still there are few other points to address, in order to clarify the methods and to make the discussion more coherent with the findings.
Major points:
1. in the Methods section, more should be said about the testing procedure. Authors state that after the encoding they administered a picture-word matching test, but no information is provided about this task. Please describe the task in order to make it replicable. In addition, why the recall task was only administered during the follow-up? A justification for this choice should be provided.
2. when authors try to explain the lack of advantage of the manipulation condition over the observation condition, in the discussion, they basically rephrased the results instead of offering a possible interpretation (page 21, lines 38-49). Please try to elaborate more on these findings, also connecting them with literature
3. the results include several post-hoc contrasts: did authors correct for multiple comparisons? Also, about this topic: page 22 lines 22-28 the contrast Irrelevant movement vs Observation is not reported within the result or I missed it?
4. page 19, line 15: “marginal significance” is meaningless in statistics. Researchers should avoid grasping around the significance and should acknowledge that a p=0.06 is NOT significant, indicating that the two conditions are NOT different from each other.
5. the sentence in page 20 lines 56-56 should be restated indicating that the accuracy is better for the Manipulation condition compared to irrelevant movement condition (and not overall).
6. page 21 lines 28-30: see above
7. pages 22-23: if a result is NOT significant, the difference between condition doesn’t exist, therefore it can not be presented to support a hypothesis.
Minor:
- Page 19: the commas after “from” and after “interaction” should be removed.
Source
© 2020 the Reviewer.
References
Orly, F., Anabel, E., Naama, F., Ricardo, T., Gal, R. 2021. The moving learner: Object manipulation in virtual reality improves vocabulary learning. Journal of Computer Assisted Learning.