Parametric Statistical Modeling by Minimum Integrated Square Error

Abstract

The likelihood function plays a central role in parametric and Bayesian estimation, as well as in nonparametric function estimation via local polynomial modeling. However, integrated square error has enjoyed a long tradition as the goodness-of-fit criterion of choice in nonparametric density estimation. In this article, I investigate the use of integrated square error, or L2 distance, as a theoretical and practical estimation tool for a variety of parametric statistical models. I show that the asymptotic inefficiency of the parameters estimated by minimizing the integrated square error or L2estimation (L2E) criterion versus the maximum likelihood estimator is roughly that of the median versus the mean. I demonstrate by example the well-known result that minimum distance estimators, including L2E, are inherently robust; however, L2E does not require specification of any tuning factors found in robust likelihood algorithms. L2E is particularly appropriate for analyzing massive datasets in which data cleaning is impractical and statistical efficiency is a secondary concern. Setting up the L2E criterion is relatively simple even with some very complex model specifications. Specific problems studied in this article include univariate density estimation, mixture density estimation, multivariate regression estimation, and robust estimation of the mean and covariance. John Tukey had a pivotal role in both nonparametric and robust estimation. This article is dedicated to his memory.