Three Sides of Smoothing: Categorical Data Smoothing, Nonparametric Regression, and Density Estimation

Abstract

The past forty years have seen a great deal of research into the construction and properties of nonparametricestimates of smooth functions. This research has focused primarily on two sides of the smoothingproblem: nonparametric regression and density estimation. Theoretical results for these two situationsare similar, and multivariate density estimation was an early justification for the Nadaraya-Watsonkernel regression estimator.A third, less well-explored, strand of applications of smoothing is to the estimation of probabilities incategorical data. In this paper the position of categorical data smoothing as a bridge between nonparametricregression and density estimation is explored. Nonparametric regression provides a paradigmfor the construction of effective categorical smoothing estimates, and use of an appropriate likelihoodfunction yields cell probability estimates with many desirable properties. Such estimates can be usedto construct regression estimates when one or more of the categorical variables are viewed as responsevariables. They also lead naturally to the construction of well-behaved density estimates using local orpenalized likelihood estimation, which can then be used in a regression context. Several real data sets areused to illustrate these points.