Local Likelihood Estimation Research Articles

Guaranteed Local Maximum Likelihood Detection of a Change Point in Nonparametric Logistic Regression

We consider nonparametric logistic regression and propose a generalized likelihood test for detecting a threshold effect that indicates a relationship between some risk factor and a defined outcome above the threshold but none below it. One important field of application is occupational medicine and in particular, epidemiological studies. In epidemiological studies, segmented fully parametric logistic regression models are often threshold models, where it is assumed that the exposure has no influence on a response up to a possible unknown threshold, and has an effect beyond that threshold. Finding efficient methods for detection and estimation of a threshold is a very important task in these studies. This article proposes such methods in a context of nonparametric logistic regression. We use a local version of unknown likelihood functions and show that under rather common assumptions the asymptotic power of our test is one. We present a guaranteed non asymptotic upper bound for the significance level of the proposed test. If applying the test yields the acceptance of the conclusion that there was a change point (and hence a threshold limit value), we suggest using the local maximum likelihood estimator of the change point and consider the asymptotic properties of this estimator.

Local partial-likelihood estimation for lifetime data

This paper considers a proportional hazards model, which allows one to examine the extent to which covariates interact nonlinearly with an exposure variable, for analysis of lifetime data. A local partial-likelihood technique is proposed to estimate nonlinear interactions. Asymptotic normality of the proposed estimator is established. The baseline hazard function, the bias and the variance of the local likelihood estimator are consistently estimated. In addition, a one-step local partial-likelihood estimator is presented to facilitate the computation of the proposed procedure and is demonstrated to be as efficient as the fully iterated local partial-likelihood estimator. Furthermore, a penalized local likelihood estimator is proposed to select important risk variables in the model. Numerical examples are used to illustrate the effectiveness of the proposed procedures.

Vitesse de convergence presque sûre de l'estimateur à noyau du maximum de vraisemblance local

We consider the local maximum likelihood estimation of θ ( x ) , unknown parameter of the conditional distribution of Y given X = x . The aim of this Note is the study of strong uniform consistency rates of the local maximum likelihood kernel estimator. Under suitable regularity conditions, we establish a uniform law of the logarithm for the maximal deviation of this estimator. The method of proof is based upon functional limit laws derived by modern empirical process theory. To cite this article: D. Blondin, C. R. Acad. Sci. Paris, Ser. I 342 (2006).

Robust multiple signal classification algorithm based on the myriad covariation matrix

A robust version of the multiple signal classification (MUSIC) bearing estimation algorithm based on robust statistics is developed for a direct sequence-code division multiple access impulsive noise channel. The proposed subspace algorithm is computed by using the antenna array covariance matrix, which is derived from the robust maximum likelihood estimator of location. Each element of the robust covariance matrix is computed as the sample myriad of a window of the received observations. The MUSIC antenna array scheme is jointly used to mitigate the effects of multipath and impulsive noise. Simulation results demonstrate that the proposed scheme significantly outperforms the other linear and nonlinear schemes.

Nonlinear regression modeling using regularized local likelihood method

We introduce a nonlinear regression modeling strategy, using a regularized local likelihood method. The local likelihood method is effective for analyzing data with complex structure. It might be, however, pointed out that the stability of the local likelihood estimator is not necessarily guaranteed in the case that the structure of system is quite complex. In order to overcome this difficulty, we propose a regularized local likelihood method with a polynomial function which unites local likelihood and regularization. A crucial issue in constructing nonlinear regression models is the choice of a smoothing parameter, the degree of polynomial and a regularization parameter. In order to evaluate models estimated by the regularized local likelihood method, we derive a model selection criterion from an information-theoretic point of view. Real data analysis and Monte Carlo experiments are conducted to examine the performance of our modeling strategy.

Propagation-Separation Approach for Local Likelihood Estimation

The paper presents a unified approach to local likelihood estimation for a broad class of nonparametric models, including e.g. the regression, density, Poisson and binary response model. The method extends the adaptive weights smoothing (AWS) procedure introduced in Polzehl and Spokoiny (2000) in context of image denoising. The main idea of the method is to describe a greatest possible local neighborhood of every design point X i in which the local parametric assumption is justified by the data. The method is especially powerful for model functions having large homogeneous regions and sharp discontinuities. The performance of the proposed procedure is illustrated by numerical examples for density estimation and classification. We also establish some remarkable theoretical nonasymptotic results on properties of the new algorithm. This includes the ``propagation'' property which particularly yields the root-n consistency of the resulting estimate in the homogeneous case. We also state an ``oracle'' result which implies rate optimality of the estimate under usual smoothness conditions and a ``separation'' result which explains the sensitivity of the method to structural changes.

On the correct convergence of the EM algorithm for Gaussian mixtures

It is well-known that the EM algorithm generally converges to a local maximum likelihood estimate. However, there have been many evidences to show that the EM algorithm can converge correctly to the true parameters as long as the overlap of Gaussians in the sample data is small enough. This paper studies this correct convergence problem asymptotically on the EM algorithm for Gaussian mixtures. It has been proved that the EM algorithm becomes a contraction mapping of the parameters within a neighborhood of the consistent solution of the maximum likelihood when the measure of average overlap among Gaussians in the original mixture is small enough and the number of samples is large enough. That is, if the initial parameters are set within the neighborhood, the EM algorithm will always converge to the consistent solution, i.e., the expected result. Moreover, the simulation results further demonstrate that this correct convergence neighborhood becomes larger as the average overlap becomes smaller.

Semiparametric three‐step estimation methods for simultaneous equation systems

AbstractThis paper proposes a new method for estimating a structural model of labour supply in which hours of work depend on (log) wages and the wage rate is considered endogenous. The main innovation with respect to other related estimation procedures is that a nonparametric additive structure in the hours of work equation is permitted. Though the focus of the paper is on this particular application, a three‐step methodology for estimating models in the presence of the above econometric problems is described. In the first step the reduced form parameters of the participation equation are estimated by a maximum likelihood procedure adapted for estimation of an additive nonparametric function. In the second step the structural parameters of the wage equation are estimated after obtaining the selection‐corrected conditional mean function. Finally, in the third step the structural parameters of the labour supply equation are estimated using local maximum likelihood estimation techniques. The paper concludes with an application to illustrate the feasibility, performance and possible gain of using this method. Copyright © 2005 John Wiley & Sons, Ltd.

Local Likelihood for non‐parametric ARCH(1) models

Abstract. We propose a non‐parametric local likelihood estimator for the log‐transformed autoregressive conditional heteroscedastic (ARCH) (1) model. Our non‐parametric estimator is constructed within the likelihood framework for non‐Gaussian observations: it is different from standard kernel regression smoothing, where the innovations are assumed to be normally distributed. We derive consistency and asymptotic normality for our estimators and show, by a simulation experiment and some real‐data examples, that the local likelihood estimator has better predictive potential than classical local regression. A possible extension of the estimation procedure to more general multiplicative ARCH(p) models with p > 1 predictor variables is also described.

Non-parametric estimation of historical volatility

Evolving volatility is a dominant feature observed in most financial time series and a key parameter used in option pricing and many other financial risk analyses. A number of methods for non-parametric scale estimation are reviewed and assessed with regard to the stylized features of financial time series. A new non-parametric procedure for estimating historical volatility is proposed based on local maximum likelihood estimation for the t-distribution. The performance of this procedure is assessed using simulated and real price data and is found to be the best among estimators we consider. We propose that it replaces the moving variance historical volatility estimator.

Efficient inference in multivariate fractionally integrated time series models

Summary A computationally simple maximum likelihood procedure for multivariate fractionally integrated time series models is introduced. This allows, e.g., efficient estimation of the memory parameters of fractional models or efficient testing of the hypothesis that two or more series are integrated of the same possibly fractional order. In particular, we show the existence of a local time domain maximum likelihood estimator and its asymptotic normality, and under Gaussianity asymptotic efficiency. The likelihood-based test statistics (Wald, likelihood ratio and Lagrange multiplier) are derived and shown to be asymptotically equivalent and chi-squared distributed under local alternatives, and under Gaussianity locally most powerful. The finite sample properties of the likelihood ratio test are evaluated by Monte Carlo experiments, which show that rejection frequencies are very close to the asymptotic local power for samples as small as n= 100.

Generalised likelihood ratio tests for spectral density

There are few techniques available for testing whether or not a family of parametric times series models fits a set of data reasonably well without serious restrictions on the forms of alternative models. In this paper, we consider generalised likelihood ratio tests of whether or not the spectral density function of a stationary time series admits certain parametric forms. We propose a bias correction method for the generalised likelihood ratio test of Fan et al. (2001). In particular, our methods can be applied to test whether or not a residual series is white noise. Sampling properties of the proposed tests are established. A bootstrap approach is proposed for estimating the null distribution of the test statistics. Simulation studies investigate the accuracy of the proposed bootstrap estimate and compare the power of the various ways of constructing the generalised likelihood ratio tests as well as some classic methods like the Cramer-von Mises and Ljung-Box tests. Our results favour the newly proposed bias reduction method using the local likelihood estimator.

EFFICIENT LIKELIHOOD INFERENCE IN NONSTATIONARY UNIVARIATE MODELS

Recent literature shows that embedding fractionally integrated time series models with spectral poles at the long-run and/or seasonal frequencies in autoregressive frameworks leads to estimators and test statistics with nonstandard limiting distributions. However, we demonstrate that when embedding such models in a general I(d) framework the resulting estimators and tests regain desirable properties from standard statistical analysis. We show the existence of a local time domain maximum likelihood estimator and its asymptotic normality—and under Gaussianity asymptotic efficiency. The Wald, likelihood ratio, and Lagrange multiplier tests are asymptotically equivalent and chi-squared distributed under local alternatives. With independent and identically distributed Gaussian errors and a scalar parameter, we show that the tests in addition achieve the asymptotic Gaussian power envelope of all invariant unbiased tests; i.e., they are asymptotically uniformly most powerful invariant unbiased against local alternatives. In a Monte Carlo study we document the finite sample superiority of the likelihood ratio test.I am grateful to Bent Jesper Christensen, Niels Haldrup, Pentti Saikkonen (the co-editor), and two anonymous referees for many useful comments and suggestions that significantly improved this paper. This work was done while the author was at the University of Aarhus, Denmark.

Logistic Discrimination Based on Regularized Local Likelihood Method

We consider the problem of constructing a nonlinear discriminant procedure, using a regularized local likelihood method. The local likelihood method is effective for analyzing data with complex structure and applicable to discriminant analysis within the framework of logistic regression. The stability of the local likelihood estimators, however, is not guaranteed in the case that the structure of the system is quite complex. Instability of the local likelihood estimators may affect the construction of the discriminant boundary region. In order to overcome this difficulty, we propose the regularized local likelihood method which unites local likelihood and regularization. A crucial issue in constructing nonlinear discriminant models is a choice of smoothing parameter and regularization parameter. In order to evaluate constructed models estimated by the regularized local likelihood method, we derive a model selection criterion from an information-theoretic point of view. We apply our discriminant procedure to real data. The results show that our technique performs well in the sense of minimizing the test error, and that clear improvements are achieved by employing the regularization.

Local likelihood method: a bridge over parametric and nonparametric regression

This paper discusses local likelihood method for estimating a regression function in a setting which includes generalized linear models. The local likelihood function is constructed by first considering a parametric model for the regression function. It is defined as a locally weighted log-likelihood with weights determined by a kernel function and a bandwidth. When a large bandwidth is chosen, the resulting estimator would be close to the fully parametric maximum likelihood estimator, so that a large bandwidth would be a relevant choice in the case where the true regression function is near the parametric family. On the other hand, when a small bandwidth is chosen, the performance of the resulting estimator would not depend much on the assumed parametric model, thus a small bandwidth would be desirable if the parametric model is largely misspecified. In this paper, we detail the way in which the risk of the local likelihood estimator is affected by bandwidth selection and model misspecification. We derive explicit formulas for the bias and variance of the local likelihood estimator for both large and small bandwidths. We look into higher order asymptotic expansions for the risk of the local likelihood estimator in the case where the bandwidth is large, which enables us to determine the optimal size of the bandwidth depending on the degree of model misspecification.

Bootstrap confidence bands for regression curves and their derivatives

Confidence bands for regression curves and their first p derivatives are obtained via local pth order polynomial estimation. The method allows for multiparameter local likelihood estimation as well as other unbiased estimating equations. As an alternative to the confidence bands obtained by asymptotic distribution theory, we also study smoothed bootstrap confidence bands. Simulations illustrate the finite sample properties of the methodology.

Local polynomial maximum likelihood estimation for Pareto-type distributions

We discuss the estimation of the tail index of a heavy-tailed distribution when covariate information is available. The approach followed here is based on the technique of local polynomial maximum likelihood estimation. The generalized Pareto distribution is fitted locally to exceedances over a high specified threshold. The method provides nonparametric estimates of the parameter functions and their derivatives up to the degree of the chosen polynomial. Consistency and asymptotic normality of the proposed estimators will be proven under suitable regularity conditions. This approach is motivated by the fact that in some applications the threshold should be allowed to change with the covariates due to significant effects on scale and location of the conditional distributions. Using the asymptotic results we are able to derive an expression for the asymptotic mean squared error, which can be used to guide the selection of the bandwidth and the threshold. The applicability of the method will be demonstrated with a few practical examples.

On Practical Efficiency of Locally Parametric Nonparametric Density Estimation Based on Local Likelihood Function

This paper offers a practical comparison of efficiency between local likelihood approach and conventional kernel approach in density estimation. The local likelihood estimation procedure maximizes a kernel smoothed log-likelihood function with respect to a polynomial approximation of the log likelihood function. We use two types of data driven bandwidths for each method and compare the mean integrated squares for several densities. Numerical results reveal that local log-linear approach with simple plug-in bandwidth shows better performance comparing to the standard kernel approach in heavy tailed distribution. For normal mixture density cases, standard kernel estimator with the bandwidth in Sheather and Jones(1991) dominates the others in moderately large sample size.

Local Likelihood Estimation in Generalized Additive Models

ABSTRACT. Generalized additive models are a popular class of multivariate non‐parametric regression models, due in large part to the ease of use of the local scoring estimation algorithm. However, the theoretical properties of the local scoring estimator are poorly understood. In this article, we propose a local likelihood estimator for generalized additive models that is closely related to the local scoring estimator fitted by local polynomial regression. We derive the statistical properties of the estimator and show that it achieves the same asymptotic convergence rate as a one‐dimensional local polynomial regression estimator. We also propose a wild bootstrap estimator for calculating point‐wise confidence intervals for the additive component functions. The practical behaviour of the proposed estimator is illustrated through a simulation experiment.

Local Linear Estimation for Time‐Dependent Coefficients in Cox's Regression Models

This article develops a local partial likelihood technique to estimate the time‐dependent coefficients in Cox's regression model. The basic idea is a simple extension of the local linear fitting technique used in the scatterplot smoothing. The coefficients are estimated locally based on the partial likelihood in a window around each time point. Multiple time‐dependent covariates are incorporated in the local partial likelihood procedure. The procedure is useful as a diagnostic tool and can be used in uncovering time‐dependencies or departure from the proportional hazards model. The programming involved in the local partial likelihood estimation is relatively simple and it can be modified with few efforts from the existing programs for the proportional hazards model. The asymptotic properties of the resulting estimator are established and compared with those from the local constant fitting. A consistent estimator of the asymptotic variance is also proposed. The approach is illustrated by a real data set from the study of gastric cancer patients and a simulation study is also presented.