Nonparametric Curve Estimation Research Articles

Efficient Semiparametric Marginal Estimation for Longitudinal/Clustered Data

We consider marginal generalized semiparametric partially linear models for clustered data. Lin and Carroll derived the semiparametric efficient score function for this problem in the multivariate Gaussian case, but they were unable to construct a semiparametric efficient estimator that actually achieved the semiparametric information bound. Here we propose such an estimator and generalize the work to marginal generalized partially linear models. We investigate asymptotic relative efficiencies of the estimators that ignore the within-cluster correlation structure either in nonparametric curve estimation or throughout. We evaluate the finite-sample performance of these estimators through simulations and illustrate it using a longitudinal CD4 cell count dataset. Both theoretical and numerical results indicate that properly taking into account the within-subject correlation among the responses can substantially improve efficiency.

On the limit in the equivalence between heteroscedastic regression and filtering model

The fact that signal-in-noise filtering model can serve as a prototype for nonparametric regression is well known and it is widely used in the nonparametric curve estimation theory. So far the only known examples showing that the equivalence may not hold have been about homoscedastic regressions where regression functions have the smoothness index at most 1 2 . As a result, the possibility of a nonequivalence and its consequences have been widely ignored in the literature by assuming that an underlying regression function is smooth enough, for instance, it is differentiable. This note presents another interesting example of a nonequivalence. It is shown that, regardless of how many times the regression function is assumed to be differentiable, it is always possible to find a heteroscedastic regression model that is nonequivalent to the corresponding filtering model. As a result, the statistician should be vigilant in using filtering models for the analysis of heteroscedastic regressions.

On Gauss quadrature and partial cross validation

New estimators of expected values Ew( X) of functions of a random variable X are introduced. The new estimators are based on Gauss quadrature, a numerical method frequently used to approximate integrals over finite intervals. The estimators need a small number of numerical evaluations and hence are useful in partial cross validation (PCV) a numerical method for finding optimal smoothing parameters in nonparametric curve estimation. The PCV can considerably reduce the computational cost of the generalized cross validation method typically used to determine the optimal smoothing parameter.

Wavelet designs for estimating nonparametric curves with heteroscedastic error

In this paper, we discuss the problem of constructing designs in order to maximize the accuracy of nonparametric curve estimation in the possible presence of heteroscedastic errors. Our approach is to exploit the flexibility of wavelet approximations to approximate the unknown response curve by its wavelet expansion thereby eliminating the mathematical difficulty associated with the unknown structure. It is expected that only finitely many parameters in the resulting wavelet response can be estimated by weighted least squares. The bias arising from this, compounds the natural variation of the estimates. Robust minimax designs and weights are then constructed to minimize mean-squared-error-based loss functions of the estimates. We find the periodic and symmetric properties of the Euclidean norm of the multiwavelet system useful in eliminating some of the mathematical difficulties involved. These properties lead us to restrict the search for robust minimax designs to a specific class of symmetric designs. We also construct minimum variance unbiased designs and weights which minimize the loss functions subject to a side condition of unbiasedness. We discuss an example from the nonparametric literature.

Nonparametric econometrics

In recent years several economic data have been analyzed by nonparametric approaches. This paper is a review of a few of the most useful procedures in the nonparametric econometric field. In particular, it describes the theory and the applications of nonparametric curve estimation (density and regression) problems with emphasis in kernel, nearest neighbor, orthogonal series, smoothing splines, logsplines and H-splines methods.

Nonparametric methods for clutter removal

Methods of clutter rejection are discussed which furnish an inherent counterpart of target tracking and detection algorithms. We describe how nonparametric curve estimation methods reduce the original sensor data to a "signal-plus-noise" model which is well suited for various hypotheses testing and dynamical filtering algorithms. We also verify a "white noise" assumption for the model of residuals.

Model Selection and the Principle of Minimum Description Length

This article reviews the principle of minimum description length (MDL) for problems of model selection. By viewing statistical modeling as a means of generating descriptions of observed data, the MDL framework discriminates between competing models based on the complexity of each description. This approach began with Kolmogorov's theory of algorithmic complexity, matured in the literature on information theory, and has recently received renewed attention within the statistics community. Here we review both the practical and the theoretical aspects of MDL as a tool for model selection, emphasizing the rich connections between information theory and statistics. At the boundary between these two disciplines we find many interesting interpretations of popular frequentist and Bayesian procedures. As we show, MDL provides an objective umbrella under which rather disparate approaches to statistical modeling can coexist and be compared. We illustrate the MDL principle by considering problems in regression, nonparametric curve estimation, cluster analysis, and time series analysis. Because model selection in linear regression is an extremely common problem that arises in many applications, we present detailed derivations of several MDL criteria in this context and discuss their properties through a number of examples. Our emphasis is on the practical application of MDL, and hence we make extensive use of real datasets. In writing this review, we tried to make the descriptive philosophy of MDL natural to a statistics audience by examining classical problems in model selection. In the engineering literature, however, MDL is being applied to ever more exotic modeling situations. As a principle for statistical modeling in general, one strength of MDL is that it can be intuitively extended to provide useful tools for new problems.

Adaptive wavelet series estimation in separable nonparametric regression models

It is well-known that multivariate curve estimation suffers from the “curse of dimensionality.” However, reasonable estimators are possible, even in several dimensions, under appropriate restrictions on the complexity of the curve. In the present paper we explore how much appropriate wavelet estimators can exploit a typical restriction on the curve such as additivity. We first propose an adaptive and simultaneous estimation procedure for all additive components in additive regression models and discuss rate of convergence results and data-dependent truncation rules for wavelet series estimators. To speed up computation we then introduce a wavelet version of functional ANOVA algorithm for additive regression models and propose a regularization algorithm which guarantees an adaptive solution to the multivariate estimation problem. Some simulations indicate that wavelets methods complement nicely the existing methodology for nonparametric multivariate curve estimation.

Scale space view of curve estimation

Scale space theory from computer vision leads to an interesting and novel approach to nonparametric curve estimation. The family of smooth curve estimates indexed by the smoothing parameter can be represented as a surface called the scale space surface. The smoothing parameter here plays the same role as that played by the scale of resolution in a visual system. In this paper, we study in detail various features of that surface from a statistical viewpoint. Weak convergence of the empirical scale space surface to its theoretical counterpart and some related asymptotic results have been established under appropriate regularity conditions. Our theoretical analysis provides new insights into nonparametric smoothing procedures and yields useful techniques for statistical exploration of features in the data. In particular, we have used the scale space approach for the development of an effective exploratory data analytic tool called SiZer.

Minimax kernels for nonparametric curve estimation

Minimax kernels for nonparametric curve estimation are investigated. They are defined to be the solutions to the kernel variational problem arising from the asymptotic maximum risk of the kernel density or derivative estimation established in this paper. A δ-perturbation method is employed to solve the kernel variational problem. Such a δ-perturbation method can be used in solving other variational problems such as the variational problem of Gasser, Müller and Mammitzsch (1985). We obtain the explicit expressions of the minimax kernels by an algorithm developed in the Appendix and tabulate the asymptotic relative efficiencies among the minimax kernels, optimal kernels and Gaussian-based kernels for further reference. The minimax kernels are shown to possess not only the minimax property, but also have higher asymptotic efficiency in the conventional, non-minimax sense. As a by-product of our study, the asymptotic minimax risks for the kernel density and derivative estimators are also obtained.

Confidence regions for the set of global maximizers of nonparametrically estimated curves

The identification of the location of global extremes has received much attention in the literature on nonparametric regression. However, all proposed estimates and confidence regions seem to require that the true unknown function has a unique global maximum. In cases where the estimated function has various peaks of similar altitude, this assumption is hard to justify. Even if the true function attains its global maximum at a single point, the task remains to decide to which of the peaks of the estimated function this point corresponds. For this purpose we derive (not necessarily connected) confidence sets for the set of global maximizers based on multiple comparison procedures in connection with nonparametric curve estimates.

Goodness‐of‐fit Test in Parametric Time Series Models

A goodness‐of‐fit test is proposed which uses nonparametric curve estimation methods to investigate the fit of parametric models for the spectral density. A test of the null hypothesis that the function has parametric form is considered with a test statistic which compares parametric estimates and nonparametric kernel estimates of the function and its derivatives at a preselected number of points. An important issue for the nonparametric estimator is bandwidth choice, and we propose a data‐adaptive method for local bandwidth choice. Under the null hypothesis, asymptotically the test statistic has a χ2 distribution. Some practical issues are discussed.

On kernel density estimation near endpoints

In this paper, we consider the estimation problem of f(0), the value of density f at the left endpoint 0. Nonparametric estimation of f(0) is rather formidable due to boundary effects that occur in nonparametric curve estimation. It is well known that the usual kernel density estimates require modifications when estimating the density near endpoints of the support. Here we investigate the local polynomial smoothing technique as a possible alternative method for the problem. It is observed that our density estimator also possesses desirable properties such as automatic adaptability for boundary effects near endpoints. We also obtain an ‘optimal kernel’ in order to estimate the density at endpoints as a solution of a variational problem. Two bandwidth variation schemes are discussed and investigated in a Monte Carlo study.

Miscellanea. A note on design transformation and binning in nonparametric curve estimation

SUMMARY Methods based on design transformation and equal-number binning are shown to overcome problems of design sparseness, while retaining excellent theoretical properties. In company with local linear smoothing they produce optimal estimators, although with optimality defined a little differently from in the nontransformed case. When used in conjunction with wavelet techniques they overcome problems of stochastic design, but, unlike related techniques based on convolution and interpolation, do not degrade features of the signal through prior smoothing, or inflate the variance.

Finite sample performance of deconvolving density estimators

Recent studies have shown that the asymptotic performance of nonparametric curve estimators in the presence of measurement error will often be very much inferior to that when the observations are error-free. For example, deconvolution of Gaussian measurement error worsens the usual algebraic convergence rates of kernel estimators to very slow logarithmic rates. However, the slow convergence rates mean that very large sample sizes may be required for the asymptotics to take effect, so the finite sample properties of the estimator may not be very well described by the asymptotics. In this article finite sample calculations are performed for the important cases of Gaussian and Laplacian measurement error which provide insight into the feasibility of deconvolving density estimators for practical sample sizes. Our results indicate that for lower levels of measurement error deconvolving density estimators can perform well for reasonable sample sizes.

Superefficiency in nonparametric function estimation

Fixed parameter asymptotic statements are often used in the context of nonparametric curve estimation problems (e.g., nonparametric density or regression estimation). In this context several forms of superefficiency can occur. In contrast to what can happen in regular parametric problems, here every parameter point (e.g., unknown density or regression function) can be a point of superefficiency. We begin with an example which shows how fixed parameter asymptotic statements have often appeared in the study of adaptive kernel estimators, and how superefficiency can occur in this context. We then carry out a more systematic study of such fixed parameter statements. It is shown in four general settings how the degree of superefficiency attainable depends on the structural assumptions in each case.

Covariate-Matched One-Sided Tests for the Difference Between Functional Means

Abstract When testing hypotheses about the effects of different treatments, variation among covariates can become confounded with that between treatments unless the treatments are applied using paired covariates. In the context of unpaired covariates, we propose implicit covariate-matching methods for testing the hypothesis that one treatment effect is greater than another. The methods are founded on the assumption that the mean treatment effect, conditional on the covariate, is a smooth function of the covariate. They are implemented using new interpolation techniques for nonparametric curve estimation. Bootstrap arguments are used to construct critical points. We show that even when the covariate distributions are identical for both treatments, covariate matching of the type that we propose produces tests of greater power than methods that do not attempt matching. Our techniques have application to two-sided hypothesis testing.

Covariate-Matched One-Sided Tests for the Difference between Functional Means

When testing hypotheses about the effects of different treatments, variation among covariates can become confounded with that between treatments unless the treatments are applied using paired covariates. In the context of unpaired covariates, we propose implicit covariate-matching methods for testing the hypothesis that one treatment effect is greater than another. The methods are founded on the assumption that the mean treatment effect, conditional on the covariate, is a smooth function of the covariate. They are implemented using new interpolation techniques for nonparametric curve estimation. Bootstrap arguments are used to construct critical points. We show that even when the covariate distributions are identical for both treatments, covariate matching of the type that we propose produces tests of greater power than methods that do not attempt matching. Our techniques have application to two-sided hypothesis testing.

Methodology for nonparametric regression from independent sources

Three nonparametric curve estimators with information from different sources were introduced by Gerard and Schucany (1996). Here, automatic local bandwidth selection is adapted to local linear estimation with two independent data sets. Consistency results are provided for this bandwidth selection procedure. The asymptotic results are supported by two simulation trials with the proposed data-based local bandwidth selection.

Wavelets in statistics: A review

The field of nonparametric function estimation has broadened its appeal in recent years with an array of new tools for statistical analysis. In particular, theoretical and applied research on the field of wavelets has had noticeable influence on statistical topics such as nonparametric regression, nonparametric density estimation, nonparametric discrimination and many other related topics. This is a survey article that attempts to synthetize a broad variety of work on wavelets in statistics and includes some recent developments in nonparametric curve estimation that have been omitted from review articles and books on the subject. After a short introduction to wavelet theory, wavelets are treated in the familiar context of estimation of «smooth» functions. Both «linear» and «nonlinear» wavelet estimation methods are discussed and cross-validation methods for choosing the smoothing parameters are addressed. Finally, some areas of related research are mentioned, such as hypothesis testing, model selection, hazard rate estimation for censored data, and nonparametric change-point problems. The closing section formulates some promising research directions relating to wavelets in statistics.