Local Likelihood Density Estimation Research Articles

Semiparametric maximum likelihood probability density estimation.

A comprehensive methodology for semiparametric probability density estimation is introduced and explored. The probability density is modelled by sequences of mostly regular or steep exponential families generated by flexible sets of basis functions, possibly including boundary terms. Parameters are estimated by global maximum likelihood without any roughness penalty. A statistically orthogonal formulation of the inference problem and a numerically stable and fast convex optimization algorithm for its solution are presented. Automatic model selection over the type and number of basis functions is performed with the Bayesian information criterion. The methodology can naturally be applied to densities supported on bounded, infinite or semi-infinite domains without boundary bias. Relationships to the truncated moment problem and the moment-constrained maximum entropy principle are discussed and a new theorem on the existence of solutions is contributed. The new technique compares very favourably to kernel density estimation, the diffusion estimator, finite mixture models and local likelihood density estimation across a diverse range of simulation and observation data sets. The semiparametric estimator combines a very small mean integrated squared error with a high degree of smoothness which allows for a robust and reliable detection of the modality of the probability density in terms of the number of modes and bumps.

Likelihood-based estimation of a semiparametric time-dependent jump diffusion model of the short-term interest rate

This paper proposes a semiparametric time-dependent jump diffusion model in an effort to capture the dynamic behavior of short-term interest rates. The newly proposed model includes a wide variety of well-known interest rate models, incorporating the time-varying instantaneous return, volatility as well as jump component. The local likelihood density estimation technique together with pseudo likelihood estimation method is employed to estimate the parameters of the model. Some simulations are conducted to examine the statistical performance of our estimators. The proposed procedure is then applied to analyze daily federal funds rate.

Local-Likelihood Transformation Kernel Density Estimation for Positive Random Variables

ABSTRACTThe kernel estimator is known not to be adequate for estimating the density of a positive random variable X. The main reason is the well-known boundary bias problems that it suffers from, but also its poor behavior in the long right tail that such a density typically exhibits. A natural approach to this problem is to first estimate the density of the logarithm of X, and obtaining an estimate of the density of X using standard results on functions of random variables (“back-transformation”). Although intuitive, the basic application of this idea usually yields poor results, as was documented earlier in the literature. In this article, the main reason for this underachievement is identified, and an easy fix is suggested. It is demonstrated that combining the transformation with local-likelihood density estimation methods produces very good estimators of -supported densities, not only close to the boundary, but also in the right tail. The asymptotic properties of the proposed “local-likelihood transformation kernel density estimators” are derived for a generic transformation, not only for the logarithm, which allows one to consider other transformations as well. One of them, called the “probex” transformation, is given more focus. Finally, the excellent behavior of those estimators in practice is evidenced through a comprehensive simulation study and the analysis of several real datasets. A nice consequence of articulating the method around local-likelihood estimation is that the resulting density estimates are typically smooth and visually pleasant, without oversmoothing important features of the underlying density.

Monte Carlo local likelihood approximation.

We propose the Monte Carlo local likelihood (MCLL) method for approximating maximum likelihood estimation (MLE). MCLL initially treats model parameters as random variables, sampling them from the posterior distribution as in a Bayesian model. The likelihood function is then approximated up to a constant by fitting a density to the posterior samples and dividing the approximate posterior density by the prior. In the MCLL algorithm, the posterior density is estimated using local likelihood density estimation, in which the log-density is locally approximated by a polynomial function. We also develop a new method that allows users to efficiently compute standard errors and the Bayes factor. Two empirical and three simulation studies are provided to demonstrate the performance of the MCLL method.

Local orthogonal polynomial expansion for density estimation

ABSTRACTA local orthogonal polynomial expansion (LOrPE) of the empirical density function is proposed as a novel method to estimate the underlying density. The estimate is constructed by matching localised expectation values of orthogonal polynomials to the values observed in the sample. LOrPE is related to several existing methods, and generalises straightforwardly to multivariate settings. By manner of construction, it is similar to local likelihood density estimation (LLDE). In the limit of small bandwidths, LOrPE functions as kernel density estimation (KDE) with high-order (effective) kernels inherently free of boundary bias, a natural consequence of kernel reshaping to accommodate endpoints. Consistency and faster asymptotic convergence rates follow. In the limit of large bandwidths LOrPE is equivalent to orthogonal series density estimation (OSDE) with Legendre polynomials, thereby inheriting its consistency. We compare the performance of LOrPE to KDE, LLDE, and OSDE, in a number of simulation studies. In terms of mean integrated squared error, the results suggest that with a proper balance of the two tuning parameters, bandwidth and degree, LOrPE generally outperforms these competitors when estimating densities with sharply truncated supports.

Probit transformation for nonparametric kernel estimation of the copula density

Copula modeling has become ubiquitous in modern statistics. Here, the problem of nonparametrically estimating a copula density is addressed. Arguably the most popular nonparametric density estimator, the kernel estimator is not suitable for the unit-square-supported copula densities, mainly because it is heavily affected by boundary bias issues. In addition, most common copulas admit unbounded densities, and kernel methods are not consistent in that case. In this paper, a kernel-type copula density estimator is proposed. It is based on the idea of transforming the uniform marginals of the copula density into normal distributions via the probit function, estimating the density in the transformed domain, which can be accomplished without boundary problems, and obtaining an estimate of the copula density through back-transformation. Although natural, a raw application of this procedure was, however, seen not to perform very well in the earlier literature. Here, it is shown that, if combined with local likelihood density estimation methods, the idea yields very good and easy to implement estimators, fixing boundary issues in a natural way and able to cope with unbounded copula densities. The asymptotic properties of the suggested estimators are derived, and a practical way of selecting the crucially important smoothing parameters is devised. Finally, extensive simulation studies and a real data analysis evidence their excellent performance compared to their main competitors.

Probit Transformation for Kernel Density Estimation on the Unit Interval

Kernel estimation of a probability density function supported on the unit interval has proved difficult, because of the well-known boundary bias issues a conventional kernel density estimator would necessarily face in this situation. Transforming the variable of interest into a variable whose density has unconstrained support, estimating that density, and obtaining an estimate of the density of the original variable through back-transformation, seems a natural idea to easily get rid of the boundary problems. In practice, however, a simple and efficient implementation of this methodology is far from immediate, and the few attempts found in the literature have been reported not to perform well. In this article, the main reasons for this failure are identified and an easy way to correct them is suggested. It turns out that combining the transformation idea with local likelihood density estimation produces viable density estimators, mostly free from boundary issues. Their asymptotic properties are derived, and a practical cross-validation bandwidth selection rule is devised. Extensive simulations demonstrate the excellent performance of these estimators compared to their main competitors for a wide range of density shapes. In fact, they turn out to be the best choice overall. Finally, they are used to successfully estimate a density of nonstandard shape supported on [0, 1] from a small-size real data sample.

Bias and bandwidth for local likelihood density estimation

A local likelihood density estimator is shown to have asymptotic bias depending on the dimension of the local parameterization. Comparing with kernel estimation it is demonstrated using a variety of bandwidths that we may obtain as good and potentially even better estimates using local likelihood. Boundary effects are also examined.

Local Likelihood Density Estimation and Value‐at‐Risk

This paper presents a new nonparametric method for computing the conditional Value‐at‐Risk, based on a local approximation of the conditional density function in a neighborhood of a predetermined extreme value for univariate and multivariate series of portfolio returns. For illustration, the method is applied to intraday VaR estimation on portfolios of two stocks traded on the Toronto Stock Exchange. The performance of the new VaR computation method is compared to the historical simulation, variance‐covariance, and J. P. Morgan methods.

Estimation of Kullback–Leibler Divergence by Local Likelihood

Motivated from the bandwidth selection problem in local likelihood density estimation and from the problem of assessing a final model chosen by a certain model selection procedure, we consider estimation of the Kullback–Leibler divergence. It is known that the best bandwidth choice for the local likelihood density estimator depends on the distance between the true density and the ‘vehicle’ parametric model. Also, the Kullback–Leibler divergence may be a useful measure based on which one judges how far the true density is away from a parametric family. We propose two estimators of the Kullback-Leibler divergence. We derive their asymptotic distributions and compare finite sample properties.

Local likelihood density estimation based on smooth truncation

Two existing density estimators based on local likelihood have properties that are comparable to those of local likelihood regression but they are much less used than their counterparts in regression. We consider truncation as a natural way of localising parametric density estimation. Based on this idea, a third local likelihood density estimator is introduced. Our main result establishes that the three estimators coincide when a free multiplicative constant is used as an extra local parameter.

Local likelihood density estimation for interval censored data

en

On local likelihood density estimation when the bandwidth is large

In this paper, we provide a large bandwidth analysis for a class of local likelihood methods. This work complements the small bandwidth analysis of Park et al. (Ann. Statist. 30 (2002) 1480). Our treatment is more general than the large bandwidth analysis of Eguchi and Copas (J. Roy. Statist. Soc. B 60 (1998) 709). We provide a higher-order asymptotic analysis for the risk of the local likelihood density estimator, from which a direct comparison between various versions of local likelihood can be made. The present work, being combined with the small bandwidth results of Park et al. (2002), gives an optimal size of the bandwidth which depends on the degree of departure of the underlying density from the proposed parametric model.

Local likelihood density estimation on random fields

Local likelihood methods hold considerable promise in density estimation. They offer unmatched flexibility and adaptivity as the resulting density estimators inherit both of the best properties of nonparametric approaches and parametric inference. However, the adoption of local likelihood methods with dependent observations, in particular with random fields, is inhibited by lack of knowledge about their properties in the case. In the present paper we detail asymptotic properties of the local likelihood density estimators for stationary random fields in the usual smoothing context of the bandwidth, h, tending to zero as the sample size tends to infinity. The asymptotic analysis is substantially more complex than in ordinary kernel density estimation on random fields.

Local likelihood density estimation on random fields

Local likelihood methods hold considerable promise in density estimation. They offer unmatched flexibility and adaptivity as the resulting density estimators inherit both of the best properties of nonparametric approaches and parametric inference. However, the adoption of local likelihood methods with dependent observations, in particular with random fields, is inhibited by lack of knowledge about their properties in the case. In the present paper we detail asymptotic properties of the local likelihood density estimators for stationary random fields in the usual smoothing context of the bandwidth, h , tending to zero as the sample size tends to infinity. The asymptotic analysis is substantially more complex than in ordinary kernel density estimation on random fields.

Local likelihood density estimation in line transect sampling

A novel semiparametric estimator for the probability density function of detected distances in line transect sampling is proposed. The estimator is obtained using a local likelihood density estimation approach, a technique recently proposed which affords the advantages of both parametric and nonparametric methods, i.e. accuracy and robustness. Moreover, a procedure for the selection of the local likelihood bandwidth is obtained. The performance of the proposed estimator with respect to some existing nonparametric and semiparametric estimators is assessed by means of a Monte Carlo study. Finally, a real data set is analyzed. Copyright © 2000 John Wiley & Sons, Ltd.

On close relations of local likelihood density estimation

Recent papers of Copas (1995), Hjort and Jones (1996) and Loader (1996) have developed closely related methods for “local likelihood” density estimation. In various places, however, a more “simple-minded” and explicit analogue of local polynomial fitting in regression has been proposed for density estimation. By introducing the usual kind of binning procedure into Hjor and Jones' method, it is shown how the more and less sophisticated versions can be reconciled. Also, we attempt to understand better the role of the attractive subclass of local likelihood methodology proposed by Loader. Finally, we look at a further subset of methods and make some theoretical comparisons between members of this class.

Local likelihood density estimation

Local likelihood was introduced by Tibshirani and Hastie as a method of smoothing by local polynomials in non-Gaussian regression models. In this paper an extension of these methods to density estimation is discussed, and comparison with other methods of density estimation presented. The local likelihood method has particularly strong advantages over kernel methods when estimating tails of densities and in multivariate settings. Suppose constraints are incorporated in a simple manner. Asymptotic properties of the estimate are discussed. A method for computing the estimate is outlined. C code to implement the estimation procedure described in this paper, together with S interfaces for graphical display of results, are available at: http://cm.bell-labs.com/stat/project/locfit/index.html