From black box to clear box: A hypothesis testing framework for scalar regression problems using deep artificial neural networks

Abstract

Despite the impressive predictive performance exhibited by deep learning across various domains, its application in research models within the social and behavioral sciences has been limited. Deep learning lacks human-accessible interpretability and does not provide statistical inferences. To address these limitations, this article presents a novel model-agnostic hypothesis testing framework tailored to scalar regression problems using deep artificial neural networks. The new framework not only determines each input variable’s direction of influence and statistical significance, but computes effect size measures akin to Cohen’s f2 in traditional ordinary least squares (OLS) regression models. Effect sizes are an important complement to null hypothesis significance testing by providing a practical significance measure that is independent of sample size considerations. To showcase the usefulness of the new framework, its application is demonstrated on both an artificial data set and a social survey using a Python sandbox implementation.