Penalized-likelihood Smoothing Research Articles

Non-parametric log-concave mixtures

Finite mixtures of parametric distributions are often used to model data of which it is known or suspected that there are sub-populations. Instead of a parametric model, a penalized likelihood smoothing algorithm is developed. The penalty is chosen to favor a log-concave result. The standard EM algorithm (“split and fit”) can be used. Theoretical results and applications are presented.

Smoothing spline ANOVA models for large data sets with Bernoulli observations and the randomized GACV

We propose the randomized Generalized Approximate Cross Validation (ranGACV) method for choosing multiple smoothing parameters in penalized likelihood estimates for Bernoulli data. The method is intended for application with penalized likelihood smoothing spline ANOVA models. In addition we propose a class of approximate numerical methods for solving the penalized likelihood variational problem which, in conjunction with the ranGACV method allows the application of smoothing spline ANOVA models with Bernoulli data to much larger data sets than previously possible. These methods are based on choosing an approximating subset of the natural (representer) basis functions for the variational problem. Simulation studies with synthetic data, including synthetic data mimicking demographic risk factor data sets is used to examine the properties of the method and to compare the approach with the GRKPACK code of Wang (1997c). Bayesian “confidence intervals” are obtained for the fits and are shown in the simulation studies to have the “across the function” property usually claimed for these confidence intervals. Finally the method is applied to an observational data set from the Beaver Dam Eye study, with scientifically interesting results.

Penalized likelihood smoothing in robust state space models

In likelihood-based approaches to robustify state space models, Gaussian error distributions are replaced by non-normal alternatives with heavier tails. Robustified observation models are appropriate for time series with additive outliers, while state or transition equations with heavy-tailed error distributions lead to filters and smoothers that can cope with structural changes in trend or slope caused by innovations outliers. As a consequence, however, conditional filtering and smoothing densities become analytically intractable. Various attempts have been made to deal with this problem, reaching from approximate conditional mean type estimation to fully Bayesian analysis using MCMC simulation. In this article we consider penalized likelihood smoothers, this means estimators which maximize penalized likelihoods or, equivalently, posterior densities. Filtering and smoothing for additive and innovations outlier models can be carried out by computationally efficient Fisher scoring steps or iterative Kalman-type filters. Special emphasis is on the Student family, for which EM-type algorithms to estimate unknown hyperparameters are developed. Operational behaviour is illustrated by simulation experiments and by real data applications.

Bootstrap confidence intervals for smoothing splines and their comparison to bayesian confidence intervals

We construct bootstrap confidence intervals for smoothing spline estimates based on Gaussian data, and penalized likelihood smoothing spline estimates based on data from .exponential families. Several vari- ations of bootstrap confidence intervals are considered and compared. We find that the commonly used ootstrap percentile intervals are inferior to the T intervals and to intervals based on bootstrap estimation of mean squared errors. The best variations of the bootstrap confidence intervals behave similar to the well known Bayesian confidence intervals. These bootstrap confidence intervals have an average coverage probability across the function being estimated, as opposed to a pointwise property.

Adaptive Spline Smoothing in Non-Gaussian Regression Models

Abstract A convenient and efficient algorithm for penalized likelihood smoothing spline regression is proposed and illustrated. The amount of smoothing is tuned adaptively via the generalized cross-validation method. Under certain conditions, the method is shown to approximately minimize the expected Kullback-Leibler discrepancy in modeling one-parameter exponential family observations through the canonical parameter. The algorithm is applied to fit logistic models using interaction splines. This research extends the recent algorithmic development of computing smoothing splines, provides further justification for the heuristics of O'Sullivan, Yandell, and Raynor (1986) on the property of generalized cross-validation in this setting, and experiments with interaction splines.

Probability estimation via smoothing in sparse contingency tables with ordered categories

Probability estimation in sparse two-dimensional contingency tables with ordered categories is examined. Several smoothing procedures are compared to analysis of the unsmoothed table. It is shown that probability estimates obtained via maximum penalized likelihood smoothing are consistent under a sparse asymptotic framework if the underlying probability matrix is smooth, and are more accurate than kernel-based and other smoothing techniques. In fact, computer simulations indicate that smoothing based on a product kernel is less effective than no smoothing at all. An example is given to illustrate the smoothing technique. Possible extensions to model building and higher dimensional tables are discussed.