The logic of significance testing

Statistical misunderstandings due to poor heuristics, especially regarding hypothesis and data testing, need to be corrected, else we are going to spend another century advancing misconceptions of old while, at the same time, debating those same misconceptions in a never-ending cycle of academic fighting leading to nowhere.

In above article ( https://doi.org/10.3389/fpsyg.2017.01434 ), I took the opportunity to address misunderstandings regarding the falsificationist logic of frequentist data testing by Bayesians. It all boils down to how you construct your Modus Tollens.

• Frequentists start from a certain antecedent [p(H0)= 1] and link to a probabilistic consequent [p(D|H0) > 5%, the 5% being the significance level, not an actual p-value]. Thus, denying the consequent also denies the antecedent (i.e., denies it = no H0, which means it does not make it probabilistic, thus, probably no H0 is an illogical conclusion).

• Bayesians start from a probabilistic antecedent [p(H0) < 1, or probably H0] and link to a certain, because observed, consequent [p(D|H0) = x]. Thus, denying the consequent also denies the probabilistic antecedent, which can be read as making it less probable (i.e., no x = probably no H0). This article follows on the steps of a previous "cognitive ergonomics" approach to re-conceptualize significance testing in order to minimize confusions when testing data and inferring scientific evidence for hypotheses ( see https://doi.org/10.1177/0959354314546157 ). It also links to another attempt at improving out heuristics here ( https://doi.org/10.3389/fpsyg.2017.01715 )

Wagenmakers, E. J., Verhagen, J., Ly, A., Matzke, D., Steingroever, H., Rouder, J. N. and Morey, R. D. (2017). “The need for Bayesian hypothesis testing in psychological science,” in Psychological Science under Scrutiny: Recent Challenges and Proposed Solutions, ed., S. O. Lilienfeld and I. D. Waldman (Chichester: John Wiley & Sons), 123-138.

Perezgonzalez JD (2017) Commentary: The Need for Bayesian Hypothesis Testing in Psychological Science. Front. Psychol. 8:1434. doi: 10.3389/fpsyg.2017.01434