Content of review 1, reviewed on December 18, 2023

This manuscript describes a proposed experiment wherein narrative will be manipulated (narrative or non-narrative) without manipulating the persuasiveness of the narrative, with recall, comprehension, and perceptions of how engaging the information is as dependent measures. An equivalence test will be used to test whether persuasiveness is manipulated, while t tests will be used to test group differences for outcome measures.

The study seems justified in its claim that narrative persuasion has been usually studied in ways that omit study on the effects of narrative regardless of persuasion, and the study seems a simple way to test as much. There is probably room for some subjectivity regarding the extent to which the literature leaves the study's main question unanswered so far, as well as to the extent to which the study is mostly manipulation check vs. new understanding of the psychology of narrative, but my sense from the manuscript is that the authors have made an adequate case that this study is needed to advance research on narratives not rooted in persuasion. Further, they are certainly going about it the right way with a registered report to instill confidence in any further research using these materials, measures, or broader concepts.

While I don't see concerns with the design (the reliability threshold for the multi-item index is not particularly high, but in any case readers will be able to make of the actual reliability observed what they choose), I am inclined to suggest there is an opportunity to add relevant outcome measures often studied in other narrative research. There are a lot of options there, from behavioral intent to perceived relevance to feelings of transportation, but I do not want the authors to be pressured to change the measures in their study design if they have already considered additional outcome measures (even for exploratory purposes) and ruled them out.

In the final manuscript, there may be a need to address the possibility that a message may be persuasive even without the recipient perceiving it to be persuasive, but I do not see this as an issue with the design so much as a limitation of the approach and thus don't think it needed to be addressed in a revision before data collection--just something to keep in mind for the final interpretation.

Source

    © 2023 the Reviewer.

Content of review 2, reviewed on May 04, 2024

The authors have thoughtfully incorporated feedback from the Stage 1 in-principle acceptance and produced simple, but clear analyses of results that appear to be consistent with the pre-registered plan approved in Stage 1. The manuscript is clear in presenting results of statistical tests, as well as an effective visualization of the data, and supplemental information is appropriately included in online appendices. Unless I am in error, the analyses and results presented follow the procedures reached during the review of the proposed study, and the results and discussion are well-written. The discussion does not appear to overreach based on the findings.

I feel that this second segment of the manuscript is appropriate for publication based on the previous in-principle acceptance, though I do have one organizational recommendation. The first section of the results describes the participants. I would recommend moving this to a section at the beginning of the methods section titled "Participants," "Participants and Sampling," or some such to make it consistent with similar empirical manuscripts that typically describe the sample of participants in the methods section (even though the information about them is collected in the study). I don't think there is any reason to move this material to the results section rather than the methods section just because the manuscript is a registered report. Otherwise, I find the study and its results interesting and applaud the authors for effective use of the registered report format to make a useful and credible contribution to knowledge in this area.

Source

    © 2024 the Reviewer.

References

    J., F. A. L., Lisa-Maria, T., R., S. C., John, K. 2024. Can narrative help people engage with and understand information without being persuasive? An empirical study. Royal Society Open Science.