A comparison of statistical tests for the adequacy of a neural network regression model

Abstract

An integral part of econometric practice is to test the adequacy of model specifications. If a model is adequately specified, it should not leave interesting features of the data-generating process in the errors. Despite the common tradition, the importance of diagnostic checking as a safeguard against mis-specification has only recently been recognized by neural network (NN) practitioners, possibly because this type of semi-parametric methodology was not originally designed for economic and financial applications. The purpose of this paper is to compare a number of analytical statistical testing procedures suitable to diagnostic checking on a neural network regression model. We present the standard Lagrange multiplier (LM) testing framework designed under the assumption of identically distributed disturbances and also examine two modifications that are robust to heteroskedasticity in errors. One modification also gives the researcher an opportunity to incorporate information concerning the volatility structure of the data-generating process in the testing procedure. By means of a Monte Carlo simulation, we investigate the performance of these tests under GARCH-type heteroskedasticity in errors and various distributional assumptions. The results show that although the primary concern of the researcher may be to design a regression model that accurately captures relations in the mean of the conditional distribution, developing a good approximation of the underlying volatility structure generally increases the efficiency of tests in detecting non-adequacy of a NN model. †http://fidelity.fme.aegean.gr/decision