Discuss practical importance of results based on interval estimates and p-value functions, not only on point estimates and null p-values

Authors

Valentin Amrhein, University of Basel, Basel, Switzerland
Sander Greenland, University of California, Los Angeles, CA, USA

Document Type

Other

Abstract

It has long been argued that we need to consider much more than an observed point estimate and a p-value to understand statistical results. One of the most persistent misconceptions about p-values is that they are necessarily calculated assuming a null hypothesis of no effect is true. Instead, p-values can and should be calculated for multiple hypothesized values for the effect size. For example, a p-value function allows us to visualize results continuously by examining how the p-value varies as we move across possible effect sizes. For more focused discussions, a 95% confidence interval shows the subset of possible effect sizes that have p-values larger than 0.05 as calculated from the same data and the same background statistical assumptions. In this sense a confidence interval can be taken as showing the effect sizes that are most compatible with the data, given the assumptions, and thus may be better termed a compatibility interval. The question that should then be asked is whether any or all of the effect sizes within the interval are substantial enough to be of practical importance.

DOI

/10.1177/02683962221105904

Recommended Citation

Amrhein, Valentin and Greenland, Sander (2022) "Discuss practical importance of results based on interval estimates and p-value functions, not only on point estimates and null p-values," Journal of Information Technology: Vol. 37: Iss. 3, Article 8.
DOI: /10.1177/02683962221105904
Available at: https://aisel.aisnet.org/jit/vol37/iss3/8