A Meta-Learning Approach to Predicting Financial Statement Fraud

Abstract

ABSTRACT: An “ultimate learning algorithm” is one that produces models that closely match the real world’s underlying distribution of functions. To try to create such an algorithm, researchers typically employ manual algorithm design with cross-validation. It has been shown that cross-validation is not a viable way to construct an ultimate learning algorithm. For machine learning researchers, “meta-learning” should be more desirable than manual algorithm design with cross-validation. Meta-learning is concerned with gaining knowledge about learning methodologies. One meta-learning approach involves evaluating the suitability of various algorithms for a learning task in order to select an appropriate algorithm. An alternative approach is to incorporate predictions from base algorithms as features to be evaluated by subsequent algorithms. This paper reports on exploratory research that implemented the latter approach as a three-layer stacked generalization model using neural networks, logistic regression, and classification tree algorithms to predict all categories of financial fraud. The purpose was to see if this form of meta-learning offered significant benefits for financial fraud prediction. Fifteen possible financial fraud predictors were identified based on a theoretical fraud model from prior research. Only public data for these possible predictors were obtained from U.S. Securities and Exchange Commission filings from the period 1995–2002 for a sample of 50 fraud and 50 non-fraud companies. These data were selected for the year prior to when the fraud was initiated. These variables were used to create a variety of neural network, logistic regression, and classification tree models while using holdout sample and cross-validation techniques. A 71.4 percent accurate neural network model was then stacked into a logistic regression model, increasing the prediction accuracy to 76.5 percent. The logistic regression model was subsequently stacked into a classification tree model to achieve an 83 percent accuracy rate. These results compared favorably to two prior neural network studies, also employing only public data, which achieved 63 percent accuracy rates. Model results were also analyzed via probability-adjusted overall error rates, relative misclassification costs, and receiver operating characteristics. The increase in classification accuracy from 71 percent to 83 percent, the decline in estimated overall error rate from 0.0057 to 0.0035, and the decline in relative misclassification costs from 2.79 to 0.58 suggest that benefits were achieved by the meta-learning stacking approach. Further research into the meta-learning stacking approach appears warranted.