Untangling Neural Nets

Abstract

Beck, King, and Zeng (2000) offer both a sweeping critique of the quantitative security studies field and a bold new direction for future research. Despite important strengths in their work, we take issue with three aspects of their research: (1) the substance of the logit model they compare to their neural network, (2) the standards they use for assessing forecasts, and (3) the theoretical and model-building implications of the nonparametric approach represented by neural networks. We replicate and extend their analysis by estimating a more complete logit model and comparing it both to a neural network and to a linear discriminant analysis. Our work reveals that neural networks do not perform substantially better than either the logit or the linear discriminant estimators. Given this result, we argue that more traditional approaches should be relied upon due to their enhanced ability to test hypotheses.