New perspectives on statistical data analysis: challenges and possibilities of digitalization for hypothesis testing in quantitative research

Abstract

p-values, the 'gold standard' of statistical validity are not as reliable as many scientists assume. In the last decade, severe problems have been observed regarding the validity of highly reputable research. Additionally, the growing availability of big data challenges the design and statistical analysis of studies and experiments across science. Therefore, it is more important than ever to make the best use of available computational tools, software and possibilities digitalization offers to improve the validity of research results. In this paper, we focus on an essential procedure often carried out in quantitative research, which is directly related to the experienced problems: Statistical hypothesis testing. First, we show that the traditional way of hypothesis testing has severe logical problems. Second, it is shown that due to the increasing availability of computational resources, highly sophisticated methods from the area of computational statistics - namely Bayesian data analysis - can complement and even replace traditional hypothesis testing. Third, we highlight how digitalization helps in making these technologies available to a vast range of researchers in the form of the novel and free software package JASP. Together, this paper shows that considering a change in perspective on statistical data analysis, in particular on hypothesis testing, provides the possibility to improve the transparency and reliability of research in the medical, social and natural sciences.