Empirical Likelihood Research Articles

Empirical Likelihood in Functional Data Analysis

Functional data analysis (FDA) studies data that include infinite-dimensional functions or objects, generalizing traditional univariate or multivariate observations from each study unit. Among inferential approaches without parametric assumptions, empirical likelihood (EL) offers a principled method in that it extends the framework of parametric likelihood ratio–based inference via the nonparametric likelihood. There has been increasing use of EL in FDA due to its many favorable properties, including self-normalization and the data-driven shape of confidence regions. This article presents a review of EL approaches in FDA, starting with finite-dimensional features, then covering infinite-dimensional features. We contrast smooth and nonsmooth frameworks in FDA and show how EL has been incorporated into both of them. The article concludes with a discussion of some future research directions, including the possibility of applying EL to conformal inference.

Empirical Bayes Likelihood for Exponential Distribution Family under Ranked Set Sampling and Dependent Data

Empirical Bayes Likelihood for Exponential Distribution Family under Ranked Set Sampling and Dependent Data

Bayesian Empirical Likelihood Regression for Semiparametric Estimation of Optimal Dynamic Treatment Regimes.

We propose a semiparametric approach to Bayesian modeling of dynamic treatment regimes that is built on a Bayesian likelihood-based regression estimation framework. Methods based on this framework exhibit a probabilistic coherence property that leads to accurate estimation of the optimal dynamic treatment regime. Unlike most Bayesian estimation methods, our proposed method avoids strong distributional assumptions for the intermediate and final outcomes by utilizing empirical likelihoods. Our proposed method allows for either linear, or more flexible forms of mean functions for the stagewise outcomes. A variational Bayes approximation is used for computation to avoid common pitfalls associated with Markov Chain Monte Carlo approaches coupled with empirical likelihood. Through simulations and analysis of the STAR*D sequential randomized trial data, our proposed method demonstrates superior accuracy over Q-learning and parametric Bayesian likelihood-based regression estimation, particularly when the parametric assumptions of regression error distributions may be potentially violated.

Empirical Inferences Under Bayesian Framework to Identify Cellwise Outliers

Outliers are typically identified using frequentist methods. The data are classified as “outliers” or “not outliers” based on a test statistic that measures the magnitude of the difference between a value and the majority part of the data. The threshold for a data value to be an outlier is typically defined by the user. However, a subjective choice of the threshold increases the uncertainty associated with outlier status for each data value. A cellwise outlier detection algorithm named FuzzyHRT is used to automate the editing process in repeated surveys. This algorithm uses Bienaymé–Chebyshev’s inequality and fuzzy logic to detect four different types of outliers resulting from format inconsistencies, historical, tail, and relational anomalies. However, fuzzy logic is not suited for probabilistic reasoning behind the identification of anomalous cells. Bayesian methods are well suited for quantifying the uncertainty associated with the identification of outliers. Although, as suggested by the literature, there exist well-developed Bayesian methods for record-level outlier detection, Bayesian methods for identifying outliers within individual records (i.e., at the cell level) remain unexplored. This paper presents two approaches from the Bayesian perspective to study the uncertainty associated with identifying outliers. A Bayesian bootstrap approach is explored to study the uncertainty associated with the output scores from the FuzzyHRT algorithm. Empirical likelihoods in a Bayesian setting are also considered for probabilistic reasoning behind the identification of anomalous cells. NASS survey data for livestock and major crop yield (such as corn) are considered for comparing the performances of the two proposed approaches with recent cellwise outlier methods.

Block empirical likelihood inference for stochastic bounding: large deviations asymptotics under m-dependence

Block empirical likelihood inference for stochastic bounding: large deviations asymptotics under m-dependence

Empirical Likelihood for Composite Quantile Regression Models with Missing Response Data

Under the assumption of missing response data, empirical likelihood inference is studied via composite quantile regression. Firstly, three empirical likelihood ratios of composite quantile regression are given and proved to be asymptotically χ2. Secondly, without an estimation of the asymptotic covariance, confidence intervals are constructed for the regression coefficients. Thirdly, three estimators are presented for the regression parameters to obtain its asymptotic distribution. The finite sample performance is assessed through simulation studies, and the symmetry confidence intervals of the parametric are constructed. Finally, the effectiveness of the proposed methods is illustrated by analyzing a real-world data set.

On an Empirical Likelihood Based Solution to the Approximate Bayesian Computation Problem

ABSTRACTApproximate Bayesian computation (ABC) methods are applicable to statistical models specified by generative processes with analytically intractable likelihoods. These methods try to approximate the posterior density of a model parameter by comparing the observed data with additional process‐generated simulated data sets. For computational benefit, only the values of certain well‐chosen summary statistics are usually compared, instead of the whole data set. Most ABC procedures are computationally expensive, justified only heuristically, and have poor asymptotic properties. In this article, we introduce a new empirical likelihood‐based approach to the ABC paradigm called ABCel. The proposed procedure is computationally tractable and approximates the target log posterior of the parameter as a sum of two functions of the data—namely, the mean of the optimal log‐empirical likelihood weights and the estimated differential entropy of the summary functions. We rigorously justify the procedure via direct and reverse information projections onto appropriate classes of probability densities. Past applications of empirical likelihood in ABC demanded constraints based on analytically tractable estimating functions that involve both the data and the parameter; although by the nature of the ABC problem such functions may not be available in general. In contrast, we use constraints that are functions of the summary statistics only. Equally importantly, we show that our construction directly connects to the reverse information projection and estimate the relevant differential entropy by a k‐NN estimator. We show that ABCel is posterior consistent and has highly favorable asymptotic properties. Its construction justifies the use of simple summary statistics like moments, quantiles, and so forth, which in practice produce accurate approximation of the posterior density. We illustrate the performance of the proposed procedure in a range of applications.

Empirical likelihood ratio test for autocorrelation in least squares regression

Detecting autocorrelation in regression models is crucial for accurate estimation and testing procedures. Among the various methods available, the Durbin–Watson (D-W) test is commonly utilized. However, a notable limitation of the D-W test is its reliance on an indeterminate range due to the intractable distribution of the D-W statistic. Consequently, the test yields inconclusive results when the statistic falls within this range. This paper introduces a distribution-free Empirical Likelihood Ratio Test (ELRT) for detecting autocorrelation in regression models. A Monte Carlo simulation study is carried out to compare the performance of the proposed ELRT with the D-W test, ( a + b d U ) approximation and bootstrapped D-W test based on size and power. The results show that the proposed ELRT considerably outperforms the original D-W test and ( a + b d U ) approximation test irrespective of sample size, level of autocorrelation and number of regressors. The proposed test is competitor for bootstrapped D-W test in terms of maintaining the empirical size and power for large samples, n ≥ 100 . As a real-life application, the working of the proposed test is illustrated through air quality data of Delhi.

Robust estimation and test for Pearson’s correlation coefficient

In this paper, using the idea of grouping under moderate data framework, we propose the median-of-means-type nonparametric estimator for Pearson’s correlation coefficient which has been used widely in various disciplines. Under certain conditions on the growing rate of the number of subgroups, the consistency and asymptotic normality of the proposed estimator are investigated. Furthermore, we construct a new method to test Pearson’s correlation coefficient based on the empirical likelihood method for median. Extensively numerical simulations are designed to demonstrate the superiorities of our estimator. It is shown that the new proposed estimator is robust with respect to outliers. Finally, we use the proposed method to study Pearson’s correlation between open price and the rate of price spread for the Shanghai Stock Exchange composite index from 18 May 2015 to 21 June 2019.

Bounded tilting estimation

The search for one-step alternatives to the generalized method of moment (GMM) has identified broad classes of potential estimators such as generalized empirical likelihoods (GEL), empirical cressie-read (ECR), exponentially tilted empirical likelihood (ETEL), and minimum discrepancy (MD) estimators. While empirical likelihood (EL) dominates other ECR estimators in terms of higher-order asymptotics, it lacks robustness to model misspecification. ETEL was shown to combine higher-order efficiency and robustness to misspecification but demands strong moment generating function existence conditions. We show, both theoretically and via simulations, how to achieve the same goal under weaker moment existence conditions within the class of MD estimators.

The quantile-based empirical likelihood for the difference of quantiles

This paper aims to explore the inference of quantile differences using the quantile-based empirical likelihood (QEL) method. In contrast to traditional empirical likelihood-based approaches, the proposed method yields an explicit likelihood ratio, making it user-friendly in practical applications. Additionally, as an expansion, the comparison of quantile differences between two populations is initially considered as a measure of differences. The limiting distribution of the smoothed log-empirical likelihood ratio for both cases is theoretically derived. The paper also includes simulation studies and an analysis of a dataset comprising 6033 genes.

On the use of the cumulant generating function for inference on time series

Innovative inference procedures for analyzing time series data are introduced. The methodology covers density approximation and composite hypothesis testing based on Whittle's estimator, which is a widely applied M-estimator in the frequency domain. Its core feature involves the cumulant generating function of Whittle's score obtained using an approximated distribution of the periodogram ordinates. A testing algorithm not only significantly expands the applicability of the state-of-the-art saddlepoint test, but also maintains the numerical accuracy of the saddlepoint approximation. Connections are made with three other prevalent frequency domain techniques: the bootstrap, empirical likelihood, and exponential tilting. Numerical examples using both simulated and real data illustrate the advantages and accuracy of the saddlepoint methods.

A Penalized Empirical Likelihood Approach for Estimating Population Sizes under the Negative Binomial Regression Model

In capture–recapture experiments, the presence of overdispersion and heterogeneity necessitates the use of the negative binomial regression model for inferring population sizes. However, within this model, existing methods based on likelihood and ratio regression for estimating the dispersion parameter often face boundary and nonidentifiability issues. These problems can result in nonsensically large point estimates and unbounded upper limits of confidence intervals for the population size. We present a penalized empirical likelihood technique for solving these two problems by imposing a half-normal prior on the population size. Based on the proposed approach, a maximum penalized empirical likelihood estimator with asymptotic normality and a penalized empirical likelihood ratio statistic with asymptotic chi-square distribution are derived. To improve numerical performance, we present an effective expectation-maximization (EM) algorithm. In the M-step, optimization for the model parameters could be achieved by fitting a standard negative binomial regression model via the R basic function glm.nb(). This approach ensures the convergence and reliability of the numerical algorithm. Using simulations, we analyze several synthetic datasets to illustrate three advantages of our methods in finite-sample cases: complete mitigation of the boundary problem, more efficient maximum penalized empirical likelihood estimates, and more precise penalized empirical likelihood ratio interval estimates compared to the estimates obtained without penalty. These advantages are further demonstrated in a case study estimating the abundance of black bears (Ursus americanus) at the U.S. Army’s Fort Drum Military Installation in northern New York.

Bayesian jackknife empirical likelihood‐based inference for missing data and causal inference

AbstractMissing data reduce the representativeness of the sample and can lead to inference problems. In this article, we apply the Bayesian jackknife empirical likelihood (BJEL) method for inference on data that are missing at random, as well as for causal inference. The semiparametric fractional imputation estimator, propensity score‐weighted estimator, and doubly robust estimator are used for constructing the jackknife pseudo values, which are needed for conducting BJEL‐based inference with missing data. Existing methods, such as normal approximation and JEL, are compared with the BJEL approach in a simulation study. The proposed approach shows better performance in many scenarios in terms of credible intervals. Furthermore, we demonstrate the application of the proposed approach for causal inference problems in a study of risk factors for impaired kidney function.

Population-level information for improving quantile regression efficiency

Observational studies often rely on sample survey data for estimation, given the difficulty of obtaining exhaustive information for the entire population. However, the use of sample data can lead to a reduction in estimation efficiency due to sampling error. When certain population-level data are accessible, devising an effective strategy to integrate them into the underlying estimation process proves advantageous. This paper proposes a methodology based on empirical likelihood for conducting quantile regression analysis on longitudinal data while incorporating population-level information. Both theoretical analysis and numerical simulations demonstrate that the proposed approach outperforms estimation methods that do not leverage population-level data.

Smoothed empirical likelihood estimation and automatic variable selection for an expectile high-dimensional model

We consider a linear model which can have a large number of explanatory variables, the errors with an asymmetric distribution or the values of the explained variable are missing at random. In order to take in account these several situations, we consider the non parametric empirical likelihood (EL) estimation method. Because a constraint in EL contains an indicator function then a smoothed function instead of the indicator will be considered. Two smoothed expectile maximum EL methods are proposed, one of which will automatically select the explanatory variables. For each of the methods we obtain the convergence rate of the estimators and their asymptotic normality. The smoothed expectile empirical log-likelihood ratio process follow asymptotically a chi-square distribution and moreover the adaptive LASSO smoothed expectile maximum EL estimator satisfies the sparsity property which guarantees the automatic selection of zero model coefficients. In order to implement these methods, we propose four algorithms.

Empirical Likelihood Inference of the Buckley–James Estimator with Length-Biased Data

Empirical Likelihood Inference of the Buckley–James Estimator with Length-Biased Data

Empirical likelihood change point detection in quantile regression models

Empirical likelihood change point detection in quantile regression models

Joint Statistical Inference for the Area under the ROC Curve and Youden Index under a Density Ratio Model

The receiver operating characteristic (ROC) curve is a valuable statistical tool in medical research. It assesses a biomarker’s ability to distinguish between diseased and healthy individuals. The area under the ROC curve (AUC) and the Youden index (J) are common summary indices used to evaluate a biomarker’s diagnostic accuracy. Simultaneously examining AUC and J offers a more comprehensive understanding of the ROC curve’s characteristics. In this paper, we utilize a semiparametric density ratio model to link the distributions of a biomarker for healthy and diseased individuals. Under this model, we establish the joint asymptotic normality of the maximum empirical likelihood estimator of (AUC,J) and construct an asymptotically valid confidence region for (AUC,J). Furthermore, we propose a new test to determine whether a biomarker simultaneously exceeds prespecified target values of AUC0 and J0 with the null hypothesis H0:AUC≤AUC0 or J≤J0 against the alternative hypothesis Ha:AUC>AUC0 and J>J0. Simulation studies and a real data example on Duchenne Muscular Dystrophy are used to demonstrate the effectiveness of our proposed method and highlight its advantages over existing methods.

On weak convergence of quantile-based empirical likelihood process for ROC curves

On weak convergence of quantile-based empirical likelihood process for ROC curves