Empirical Likelihood Function Research Articles

Orthogonality-based bias-corrected empirical likelihood inference for partial linear varying coefficient EV models with longitudinal data

In this paper, we study the model estimation for the partial linear varying coefficient errors-in-variables (EV) models with longitudinal data. Based on the empirical likelihood and quadratic inference functions, we propose an orthogonality-based bias-corrected empirical likelihood estimation method using the QR decomposition method of matrix. The proposed method can handle the measurement errors of covariates and unknown within-subject correlation simultaneously, estimate the parametric and nonparametric components separately. Under some regularity conditions, the resulting estimator of the parametric component is asymptotic normal and the estimators of the nonparametric varying coefficients achieve the optimal convergence rate. Furthermore, we conduct simulation studies and a real data analysis to compare the performance of the proposed method with existing methods in finite samples. The results show that the feasibility of the proposed method is obvious.

Empirical likelihood inference for monotone index model

This paper proposes an empirical likelihood inference method for monotone index models. We construct the empirical likelihood function based on a modified score function developed by Balabdaoui et al. (Scand J Stat 46:517–544, 2019), where the monotone link function is estimated by isotonic regression. It is shown that the empirical likelihood ratio statistic converges to a weighted chi-squared distribution. We suggest inference procedures based on an adjusted empirical likelihood statistic that is asymptotically pivotal, and a bootstrap calibration with recentering. A simulation study illustrates usefulness of the proposed inference methods.

Variational Bayes for Fast and Accurate Empirical Likelihood Inference

We develop a fast and accurate approach to approximate posterior distributions in the Bayesian empirical likelihood framework. Bayesian empirical likelihood allows for the use of Bayesian shrinkage without specification of a full likelihood but is notorious for leading to several computational difficulties. By coupling the stochastic variational Bayes procedure with an adjusted empirical likelihood framework, the proposed method overcomes the intractability of both the exact posterior and the arising evidence lower bound objective, and the mismatch between the exact posterior support and the variational posterior support. The optimization algorithm achieves fast algorithmic convergence by using the variational expected gradient of the log adjusted empirical likelihood function. We prove the consistency of the proposed approximate posterior distribution and an empirical likelihood analogue of the variational Bernstein-von-Mises theorem. Through several numerical examples, we confirm the accuracy and quick algorithmic convergence of our proposed method. Supplementary materials for this article are available online.

Empirical likelihood confidence interval for difference-in-differences estimator with panel data

This paper develops an empirical likelihood approach to construct the confidence interval for the average treatment effect on the treated under the difference-in-differences framework. The empirical likelihood function is constructed based on a doubly robust moment function of the parameter of interest. Under some regularity conditions, we show that the proposed method retains the nonparametric Wilks property of empirical likelihood.

A Robust Consistent Information Criterion for Model Selection Based on Empirical Likelihood

Conventional likelihood-based information criteria for model selection rely on the distribution assumption of data. However, for complex data that are increasingly available in many scientific fields, the specification of their underlying distribution turns out to be challenging, and the existing criteria may be limited and are not general enough to handle a variety of model selection problems. Here, we propose a robust and consistent model selection criterion based upon the empirical likelihood function which is data-driven. In particular, this framework adopts plug-in estimators that can be achieved by solving external estimating equations, not limited to the empirical likelihood, which avoids potential computational convergence issues and allows versatile applications, such as generalized linear models, generalized estimating equations, penalized regressions and so on. The formulation of our proposed criterion is initially derived from the asymptotic expansion of the marginal likelihood under variable selection framework, but more importantly, the consistent model selection property is established under a general context. Extensive simulation studies confirm the out-performance of the proposal compared to traditional model selection criteria. Finally, an application to the Atherosclerosis Risk in Communities Study illustrates the practical value of this proposed framework.

Bayesian Estimation and Comparison of Conditional Moment Models

Abstract We consider the Bayesian analysis of models in which the unknown distribution of the outcomes is specified up to a set of conditional moment restrictions. The non-parametric exponentially tilted empirical likelihood function is constructed to satisfy a sequence of unconditional moments based on an increasing (in sample size) vector of approximating functions (such as tensor splines based on the splines of each conditioning variable). For any given sample size, results are robust to the number of expanded moments. We derive Bernstein–von Mises theorems for the behaviour of the posterior distribution under both correct and incorrect specification of the conditional moments, subject to growth rate conditions (slower under misspecification) on the number of approximating functions. A large-sample theory for comparing different conditional moment models is also developed. The central result is that the marginal likelihood criterion selects the model that is less misspecified. We also introduce sparsity-based model search for high-dimensional conditioning variables, and provide efficient Markov chain Monte Carlo computations for high-dimensional parameters. Along with clarifying examples, the framework is illustrated with real data applications to risk-factor determination in finance, and causal inference under conditional ignorability.

Restricted Empirical Likelihood Estimation for Time Series Autoregressive Models

In this paper, we first illustrate the restricted empirical likelihood function, as an alternative to the usual empirical likelihood. Then, we use this quasi-empirical likelihood function as a basis for Bayesian analysis of AR(r) time series models. The efficiency of both the posterior computation algorithm, when the estimating equations are linear functions of the parameters, and the EM algorithm for estimating hyper-parameters is an appealing property of our proposed approach. Moreover, the competitive finite-sample performance of this proposed method is illustrated via both simulation study and analysis of a real dataset.

Robust empirical likelihood for time series

This article introduces a robust frequency domain empirical likelihood inference procedure for the parametric component in the spectral densities of stationary processes. We construct the empirical likelihood function by using a new spectral estimating function to achieve robustness against contamination in the spectral density. Simulation studies demonstrate the good performance of the proposed robust frequency domain empirical likelihood method, which produces more accurate confidence regions than the ordinary empirical likelihood counterpart.

Two-sample test based on empirical likelihood ratio under semi-competing risks data

This article considers the two-sample testing problem of the survival function of the non terminal event time under semi-competing risks data. The empirical likelihood function is constructed for the survival function estimation of the non terminal event time, then maximize it by the PSO (Particle swarm optimization) algorithm to obtain the MLE. For the testing problem, the article develops the empirical likelihood ratio test to compare the two survival curves. From simulation studies, it shows the performance of the proposed approaches is good. Finally, a real data analysis is presented for illustration.

Split sample empirical likelihood

Empirical likelihood offers a nonparametric approach to estimation and inference, which replaces the probability density-based likelihood function with a function defined by estimating equations. While this eliminates the need for a parametric specification, the restriction of numerical optimization greatly decreases the applicability of empirical likelihood for large data problems. A solution to this problem is the split sample empirical likelihood; this variant utilizes a divide and conquer approach, allowing for parallel computation of the empirical likelihood function. The results show the asymptotic distribution of the estimators and test statistics derived from the split sample empirical likelihood are the same seen in standard empirical likelihood yet have significantly decreased computational times.

IMPROVED EMPIRICAL LIKELIHOOD FUNCTION BASED ON NORMALIZATION-DEPENDENT REPLICATE MEASUREMENTS.

Based on $n$ replicate measurements that require known normalization factors and assuming an underlying normal distribution for individual measurements but with unknown standard deviation, a combined likelihood function is derived that takes the form of a Student's $t$-distribution with $\nu = n-1$ degrees of freedom and $t=(\psi -\overline{Y})/s$, where $\psi $ is the true value of the measurement quantity calculated from the forward model, and $\overline{Y}$ and $s$ are average and standard error of the mean obtained from the $n$ measurements defined with weighting proportional to the inverse of the normalization factor squared. Assuming an underlying triangle distribution rather than a normal distribution does not produce a large change for six replicates. Examples of replicate data from an animal study and sequential occupational urine and fecal monitoring are given. The use of the empirical likelihood function in data modeling is discussed.

Empirical likelihood method for longitudinal data generated from unequally-spaced Lèvy processes

By introducing the notion of “empirical likelihood function of observing sums”, unequally-spaced time series data and longitudinal data generated from Levy processes can be analyzed. Characteristic function is further incorporated to handle the situations where the density function of the increments is difficult to obtain. In the situations where both characteristic function and the density function are available, it is shown through the simulation examples that the proposed empirical maximum likelihood method does not suffer from significant information loss comparing to the maximum likelihood estimation method. The performances of the proposed method is tested for both equally-spaced and unequally-spaced observations.

Empirical Bayes method for Boltzmann machines

We consider an empirical Bayes method for Boltzmann machines and propose an algorithm for it. The empirical Bayes method allows for estimation of the values of the hyperparameters of the Boltzmann machine by maximizing a specific likelihood function referred to as the empirical Bayes likelihood function in this study. However, the maximization is computationally hard because the empirical Bayes likelihood function involves intractable integrations of the partition function. The proposed algorithm avoids this computational problem by using the replica method and the Plefka expansion. Our method is quite simple and fast because it does not require any iterative procedures and gives reasonable estimates at a certain condition. However, our method introduces a bias to the estimate, which exhibits an unnatural behavior with respect to the size of the dataset. This peculiar behavior is supposed to be due to the approximate treatment by the Plefka expansion. A possible extension to overcome this behavior is also discussed.

Bayesian empirical likelihood and variable selection for censored linear model with applications to acute myelogenous leukemia data

This paper develops the Bayesian empirical likelihood (BEL) method and the BEL variable selection for linear regression models with censored data. Empirical likelihood is a multivariate analysis tool that has been widely applied to many fields such as biomedical and social sciences. By introducing two special priors to the empirical likelihood function, we find two obvious superiorities of the BEL methods, that is (i) more precise coverage probabilities of the BEL credible region and (ii) higher accuracy and correct identification rate of the BEL model selection using an hierarchical Bayesian model, vs. some current methods such as the LASSO, ALASSO and SCAD. The numerical simulations and empirical analysis of two data examples show strong competitiveness of the proposed method.

Empirical likelihood estimation of the Markov-switching model

The Markov-switching (MS) model is one of the most popular nonlinear time series models in the literature. However, the estimation methods which are normally used to estimate the MS models rely on the assumption of a parametric distribution, which sometimes is considered as a strong assumption. This study, therefore, tries to relax the assumption and develop a more flexible estimator for the MS models that is a maximum empirical likelihood estimation. According to this approach, the parametric likelihood will be replaced by the empirical likelihood function with relatively minor modifications to existing recursive filters. A performance of the suggested estimation method is then evaluated through a Monte Carlo experiment and a real application, the U.S. business cycle. Overall results of both empirical studies indicate that the empirical likelihood could outweigh the classical likelihood estimators.

Robust confidence regions for the semi-parametric regression model with responses missing at random

ABSTRACTIn this paper, a regression semi-parametric model is considered where responses are assumed to be missing at random. From the empirical likelihood function defined based on the rank-based estimating equation, robust confidence intervals/regions of the true regression coefficient are derived. Monte Carlo simulation experiments show that the proposed approach provides more accurate confidence intervals/regions compared to its normal approximation counterpart under different model error structure. The approach is also compared with the least squares approach, and its superiority is shown whenever the error distribution in the simulation study is heavy tailed or contaminated. Finally, a real data example is given to illustrate our proposed method.

Hybrid combinations of parametric and empirical likelihoods.

This paper develops a hybrid likelihood (HL) method based on a compromise between parametric and nonparametric likelihoods. Consider the setting of a parametric model for the distribution of an observation Y with parameter θ. Suppose there is also an estimating function m(·, μ) identifying another parameter μ via Em(Y, μ) = 0, at the outset defined independently of the parametric model. To borrow strength from the parametric model while obtaining a degree of robustness from the empirical likelihood method, we formulate inference about θ in terms of the hybrid likelihood function Hn (θ) = Ln (θ)1-a Rn (μ(θ)) a . Here a ∈ [0,1) represents the extent of the compromise, Ln is the ordinary parametric likelihood for θ, Rn is the empirical likelihood function, and μ is considered through the lens of the parametric model. We establish asymptotic normality of the corresponding HL estimator and a version of the Wilks theorem. We also examine extensions of these results under misspecification of the parametric model, and propose methods for selecting the balance parameter a.

Adjusted Empirical Likelihood for Time Series Models

Empirical likelihood method has been applied to dependent observations by Monti (Biometrika, 84, 395–405 1997) through the Whittle’s estimation method. Similar asymptotic distribution of the empirical likelihood ratio statistic for stationary time series has been derived to construct the confidence regions for the parameters. However, Monti’s approach is valid only when the error terms follow a Gaussian distribution. Nordman and Lahiri (Ann. Statist., 34, 3019–50 2006) derived estimating functions and empirical likelihood ratio statistic using frequency domain empirical likelihood approach for non-Gaussian error term distributions. Nonetheless, the required numerical problem of computing profile empirical likelihood function which involves constrained maximization has no solution sometimes, which leads to the drawbacks of using the original version of the empirical likelihood ratio. In this paper, we propose an adjusted empirical likelihood ratio statistic to modify the one proposed by Nordman and Lahiri so that it guarantees the existence of the solution of the required maximization problem, while maintaining the similar asymptotic properties as Nordman and Lahiri obtained. Simulations have been conducted to illustrate the coverage probabilities obtained by the adjusted version for different time series models which are competitive to the ones based on Nordman and Lahiri’s version, especially for small sample sizes.

Adjusted empirical likelihood for long-memory time-series models

The empirical likelihood method has been applied to short-memory time-series models by Monti through the Whittle’s estimation method. Yau extended this idea to long-memory time series models. Nordman and Lahiri showed a new formulation of empirical likelihood for inference of dependent data. Asymptotic distributions of the empirical likelihood ratio statistic for short and long-memory time series have been derived to construct confidence regions for the corresponding model parameters. However, computing profile empirical likelihood function involving constrained maximization does not always have a solution, which leads to several drawbacks. In this article, we propose an adjusted empirical likelihood procedure to modify the one proposed by Yau for the autoregressive fractionally integrated moving average (ARFIMA) model. It guarantees the existence of a solution to the required maximization problem and maintains an asymptotic chi-squared distribution. Simulations have been carried out to illustrate that the adjusted empirical likelihood method for different long-time series models provides competitive confidence regions and coverage probabilities compared to the unadjusted ones, especially for small sample sizes.

Fully probabilistic seismic source inversion – Part 2: Modelling errors and station covariances

Abstract. Seismic source inversion, a central task in seismology, is concerned with the estimation of earthquake source parameters and their uncertainties. Estimating uncertainties is particularly challenging because source inversion is a non-linear problem. In a companion paper, Stähler and Sigloch (2014) developed a method of fully Bayesian inference for source parameters, based on measurements of waveform cross-correlation between broadband, teleseismic body-wave observations and their modelled counterparts. This approach yields not only depth and moment tensor estimates but also source time functions. A prerequisite for Bayesian inference is the proper characterisation of the noise afflicting the measurements, a problem we address here. We show that, for realistic broadband body-wave seismograms, the systematic error due to an incomplete physical model affects waveform misfits more strongly than random, ambient background noise. In this situation, the waveform cross-correlation coefficient CC, or rather its decorrelation D = 1 − CC, performs more robustly as a misfit criterion than ℓp norms, more commonly used as sample-by-sample measures of misfit based on distances between individual time samples. From a set of over 900 user-supervised, deterministic earthquake source solutions treated as a quality-controlled reference, we derive the noise distribution on signal decorrelation D = 1 − CC of the broadband seismogram fits between observed and modelled waveforms. The noise on D is found to approximately follow a log-normal distribution, a fortunate fact that readily accommodates the formulation of an empirical likelihood function for D for our multivariate problem. The first and second moments of this multivariate distribution are shown to depend mostly on the signal-to-noise ratio (SNR) of the CC measurements and on the back-azimuthal distances of seismic stations. By identifying and quantifying this likelihood function, we make D and thus waveform cross-correlation measurements usable for fully probabilistic sampling strategies, in source inversion and related applications such as seismic tomography.