Approximate Maximum Likelihood Estimator Research Articles

Enhanced Laplace approximation

The Laplace approximation has been proposed as a method for approximating the marginal likelihood of statistical models with latent variables. However, the approximate maximum likelihood estimators derived from the Laplace approximation are often biased for binary or temporally and/or spatially correlated data. Additionally, the corresponding Hessian matrix tends to underestimates the standard errors of these approximate maximum likelihood estimators. While higher-order approximations have been suggested, they are not applicable to complex models, such as correlated random effects models, and fail to provide consistent variance estimators. In this paper, we propose an enhanced Laplace approximation that provides the true maximum likelihood estimator and its consistent variance estimator. We study its relationship with the variational Bayes method. We also define a new restricted maximum likelihood estimator for estimating dispersion parameters and study their asymptotic properties. Enhanced Laplace approximation generally demonstrates how to obtain the true restricted maximum likelihood estimators and their variance estimators. Our numerical studies indicate that the enhanced Laplace approximation provides a satisfactory maximum likelihood estimator and restricted maximum likelihood estimator, as well as their variance estimators in the frequentist perspective. The maximum likelihood estimator and restricted maximum likelihood estimator can be also interpreted as the posterior mode and marginal posterior mode under flat priors, respectively. Furthermore, we present some comparisons with Bayesian procedures under different priors.

Exact selective inference with randomization

Summary We introduce a pivot for exact selective inference with randomization. Not only does our pivot lead to exact inference in Gaussian regression models, but it is also available in closed form. We reduce this problem to inference for a bivariate truncated Gaussian variable. By doing so, we give up some power that is achieved with approximate maximum likelihood estimation in?. Yet our pivot always produces narrower confidence intervals than a closely related data splitting procedure. We investigate the trade-off between power and exact selective inference on simulated datasets and an HIV drug resistance dataset.

Inference for compound truncated Poisson log-normal model with application to maximum precipitation data

The main objective of this paper is to propose a new general family of distributions, namely compound truncated Poisson log-normal distribution of which log-normal distribution is a special case. The proposed model has three unknown parameters, and it can take variety of shapes. It can be used effectively in analyzing maximum precipitation data during a particular period of time obtained from different stations. It is assumed that the number of stations operate follows a zero-truncated Poisson random variables, and the daily precipitation follows a log-normal random variable. The maximum likelihood estimators can be obtained quite conveniently using Expectation-Maximization (EM) algorithm. Approximate maximum likelihood estimators are also derived. The associated confidence intervals can also be obtained from the observed Fisher information matrix. Simulation results have been performed to check the performance of the EM algorithm, and it is observed that the EM algorithm works quite well in this case. When we analyze the precipitation data set using the proposed model it is observed that the proposed model provides better fit than some of the existing models.

Maximum likelihood estimation for discrete latent variable models via evolutionary algorithms

We propose an evolutionary optimization method for maximum likelihood and approximate maximum likelihood estimation of discrete latent variable models. The proposal is based on modified versions of the expectation–maximization (EM) and variational EM (VEM) algorithms, which are based on the genetic approach and allow us to accurately explore the parameter space, reducing the chance to be trapped into one of the multiple local maxima of the log-likelihood function. Their performance is examined through an extensive Monte Carlo simulation study where they are employed to estimate latent class, hidden Markov, and stochastic block models and compared with the standard EM and VEM algorithms. We observe a significant increase in the chance to reach global maximum of the target function and a high accuracy of the estimated parameters for each model. Applications focused on the analysis of cross-sectional, longitudinal, and network data are proposed to illustrate and compare the algorithms.

Approximate Maximum Likelihood Estimation in Fractional Stochastic Transport Equation

We estimate the drift of the fractional stochastic transport equation the by maximum likelihood and the minimum contrast methods. We show consistency and asymptotic normality of the estimators. We consider both continuous and discrete time observations.

Testing Data Cloning as the Basis of an Estimator for the Stochastic Volatility in Mean Model

Developed as a refinement of stochastic volatility (SV) models, the stochastic volatility in mean (SVM) model incorporates the latent volatility as an explanatory variable in both the mean and variance equations. It, therefore, provides a way of assessing the relationship between returns and volatility, albeit at the expense of complicating the estimation process. This study introduces a Bayesian methodology that leverages data-cloning algorithms to obtain maximum likelihood estimates for SV and SVM model parameters. Adopting this Bayesian framework allows approximate maximum likelihood estimates to be attained without the need to maximize pseudo likelihood functions. The key contribution this paper makes is that it proposes an estimator for the SVM model, one that uses Bayesian algorithms to approximate the maximum likelihood estimate with great effect. Notably, the estimates it provides yield superior outcomes than those derived from the Markov chain Monte Carlo (MCMC) method in terms of standard errors, all while being unaffected by the selection of prior distributions.

Source Localization Based on Hybrid AOA, TDOA, and RSS Measurements

In this paper, a constrained weighted least squares (CWLS) problem with two quadratic constraints for hybrid AOA, TDOA, and RSS localization is formulated. The Lagrange multiplier method cannot be efficiently used in this situation, so we provide four methods to solve the CWLS problem: 1) weighted least squares (WLS); 2) iterative constrained weighted least squares (ICWLS); 3) Lagrange multiplier method combined with Newton’s method (LMM+NM); 4) semidefinite programming (SDP). Furthermore, in order to obtain a good balance between accuracy and computational burden, we develop a fixed point iteration (FPI)-based algorithm that is derived from an approximate maximum likelihood estimator (MLE). Simulation results show that the accuracy of the proposed FPI method is better than the other four methods; besides, its computational burden is one in ten of the SDP algorithm.

Regression of binary network data with exchangeable latent errors

AbstractUndirected, binary network data consist of indicators of symmetric relations between pairs of actors. Regression models of such data allow for the estimation of effects of exogenous covariates on the network and for prediction of unobserved data. Ideally, estimators of the regression parameters should account for the inherent dependencies among relations in the network that involve the same actor. To account for such dependencies, researchers have developed a host of latent variable network models; however, estimation of many latent variable network models is computationally onerous and which model is best to base inference upon may not be clear. We propose the probit exchangeable (PX) model for undirected binary network data that is based on an assumption of exchangeability, which is common to many of the latent variable network models in the literature. The PX model can represent the first two moments of any exchangeable network model. We leverage the EM algorithm to obtain an approximate maximum likelihood estimator of the PX model that is extremely computationally efficient. Using simulation studies, we demonstrate the improvement in estimation of regression coefficients of the proposed model over existing latent variable network models. In an analysis of purchases of politically aligned books, we demonstrate political polarization in purchase behavior and show that the proposed estimator significantly reduces runtime relative to estimators of latent variable network models, while maintaining predictive performance.

Subspace-based distributed target detection in compound-Gaussian clutter

This paper investigates the problem of radar detection of distributed targets embedded in unknown compound-Gaussian clutter. To solve the problem, we first derive two detectors for fixed covariance matrix according to Rao and Wald tests with the use of order statistics theory. Then, the unknown covariance matrix is estimated by the approximate maximum likelihood (AML) estimation. Numerical examples through simulated and real data show that the Wald test-based detector has a higher probability of detection (PD) than existing detectors.

Semiparametric approach to estimation of marginal mean effects and marginal quantile effects

We consider a semiparametric generalized linear model and study estimation of both marginal mean effects and marginal quantile effects in this model. We propose an approximate maximum likelihood estimator, and rigorously establish the consistency, the asymptotic normality, and the semiparametric efficiency of our method in both the marginal mean effect and the marginal quantile effect estimation. Simulation studies are conducted to illustrate the finite sample performance, and we apply the new tool to analyze a Swiss non-labor income data and discover a new interesting predictor.

Unsupervised mixture estimation via approximate maximum likelihood based on the Cramér - von Mises distance

Mixture distributions with dynamic weights are an efficient way of modeling loss data characterized by heavy tails. However, maximum likelihood estimation of this family of models is difficult, mostly because of the need to evaluate numerically an intractable normalizing constant. In such a setup, simulation-based estimation methods are an appealing alternative. The approximate maximum likelihood estimation (AMLE) approach is employed. It is a general method that can be applied to mixtures with any component densities, as long as simulation is feasible. The focus is on the dynamic lognormal-generalized Pareto distribution, and the Cramér - von Mises distance is used to measure the discrepancy between observed and simulated samples. After deriving the theoretical properties of the estimators, a hybrid procedure is developed, where standard maximum likelihood is first employed to determine the bounds of the uniform priors required as input for AMLE. Simulation experiments and two real-data applications suggest that this approach yields a major improvement with respect to standard maximum likelihood estimation.

Approximate MLEs for the location and scale parameters of the Poisson-half-logistic distribution

Recently, the application of compound distributions has increased due to the flexibility in fitting to actual data in various fields such as economics, insurance, etc. Poisson-half-logistic distribution is one of these distributions with an increasing-constant hazard rate that can be used in parallel systems and complementary risk models. Because of the complexity of the form of this distribution, it is not possible to obtain classical parameter estimates (such as MLE) by the analytical method for the location and scale parameters. We present a simple way of deriving explicit estimators by approximating the likelihood equations appropriately. This paper presents AMLE (Approximate MLE) method to obtain the location and scale parameters estimation. Using simulation, we show that this method is as efficient as the maximum likelihood estimators (MLEs), we obtain the variance of estimators from the inverse of the observed Fisher information matrix, and we see that when sample size increases bias and variance of these estimators, MSEs of parameters decrease. Finally, we present a numerical example to illustrate the methods of inference developed here.

Le Cam–Stratonovich–Boole theory for Itô diffusions

Abstract We connect the theory of local asymptotic normality (LAN) of Le Cam to Boole’s approximation of the Stratonovich stochastic integral by estimating the parameter in the nonlinear drift coefficient of an ergodic diffusion process satisfying a homogeneous Itô stochastic differential equation based on discretely spaced dense observations of the process. The asymptotic normality and local asymptotic minimaxity (in the Hajek–Le Cam sense) of approximate maximum likelihood estimators, approximate maximum probability estimators and approximate Bayes estimators based on Itô and Boole’s approximations of the continuous likelihood are obtained under an almost slowly increasing experimental design (ASIED) condition ( T n 6 / 7 → 0 {\frac{T}{n^{6/7}}\to 0} as T → ∞ {T\to\infty} and n → ∞ {n\to\infty} , where T is the length of the observation time and n is the number of observations) through the weak convergence of the approximate likelihood ratio random fields. Among other things, the Bernstein–von Mises type theorems concerning the convergence of suitably normalized and centered approximate posterior distributions to normal distribution under the same design condition are proved. Asymptotic normality and asymptotic efficiency of the conditional least squares estimator under the same design condition are obtained as a by-product. The log-likelihood derivatives based on Itô approximations are martingales, but the log-likelihood derivatives based on Boole’s approximations are not martingales but weighted averages of forward and backward martingales. These new approximations have faster rate of convergence than the martingale approximations. The methods would have advantages over Euler and Milstein approximations for Monte Carlo simulations.

Proportional stochastic generalized Lotka–Volterra model with an application to learning microbial community structures

Inferring microbial community structure based on temporal metagenomics data is an important goal in microbiome studies. The deterministic generalized Lotka–Volterra (GLV) differential equations have been commonly used to model the dynamics of microbial taxa. However, these approaches fail to take random environmental fluctuations into account and usually ignore the compositional nature of relative abundance data, which may deteriorate the estimates. In this article, we consider the microbial dynamics in terms of relative abundances by introducing a reference taxon, and propose a new proportional stochastic GLV (pSGLV) differential equation model, where the random perturbations of Brownian motion in this model can naturally account for the external environmental effects on the microbial community. We establish conditions and show some mathematical properties of the solutions including general existence and uniqueness, stochastic ultimate boundedness, stochastic permanence, the existence of stationary distribution, and ergodicity property. We further develop approximate maximum likelihood estimators (AMLEs) based on discrete observations and systematically investigate the consistency and asymptotic normality of the proposed estimators. At last, numerical simulations support our theoretical findings and our method is demonstrated through an application to the well-known “moving picture” temporal microbial dataset.

Fast, Approximate Maximum Likelihood Estimation of Log-Gaussian Cox Processes

Log-Gaussian Cox processes (LGCPs) offer a framework for regression-style modeling of point patterns that can accommodate spatial latent effects. These latent effects can be used to account for missing predictors or other sources of clustering that could not be explained by a Poisson process. Fitting LGCP models can be challenging because the marginal likelihood does not have a closed form and it involves a high dimensional integral to account for the latent Gaussian field. We propose a novel methodology for fitting LGCP models that addresses these challenges using a combination of variational approximation and reduced rank interpolation. Additionally, we implement automatic differentiation to obtain exact gradient information, for computationally efficient optimization and to consider integral approximation using the Laplace method. We demonstrate the method’s performance through both simulations and a real data application, with promising results in terms of computational speed and accuracy compared to that of existing approaches. Supplementary material for this article is available online.

Uncertainty quantification in operational modal analysis of time-varying structures based on time-dependent autoregressive moving average model

The identified modal parameters in Operational Modal Analysis (OMA) are always subject to statistical uncertainties from many sources and the uncertainty is also important to assess the quality of the identified results. This study proposes an uncertainty quantification method in OMA for time-varying structures based on the Functional Series Time-dependent AutoRegressive Moving Average (FS-TARMA) model. A multi-stage pseudo-linear optimization scheme is first adopted to obtain the approximate Maximum Likelihood Estimates (MLEs) of FS-TARMA model parameters and the covariance is computed as the inverse Hessian matrix of the Negative Log-Likelihood Function (NLLF) at the MLEs. Finally, the approximate closed-form covariance of modal parameters is derived by the first-order sensitivity method. The proposed method is verified by two numerical examples and an experimental beam. Furthermore, the ability of the uncertainty to extinguish the spurious modes caused by colored noise excitation and the noise modes caused by measurement noise or inappropriate model structure is also demonstrated in these examples.

Reliability Estimation for the Inverse Weibull Distribution Under Adaptive Type-II Progressive Hybrid Censoring: Comparative Study

The aim of this study is to investigate different methods of estimating the stress-strength reliability parameter, $\theta =P(Y<X)$, when the strength (X) and the stress (Y) are independent random variables taken from the inverse Weibull distribution (IWD), with the same shape parameter and different scale parameters. Based on Adaptive Type-II Hybrid progressive censored samples, we employ classical and Bayesian approaches. In the classical approach, we use the maximum likelihood estimator (MLE), the approximate maximum likelihood estimator (AMLE), and the least squares estimator (LSE). In contrast, the Bayesian approach utilizes symmetric and asymmetric loss functions. Due to the absence of explicit forms for Bayes estimators, we propose using Lindley's approximation method for computing the Bayes estimators. We compare these estimators using extensive simulations and two criteria: the bias and the mean square error (MSE). Finally, two real-life data examples are provided for illustrations.

Statistical inference of the stress-strength reliability for inverse Weibull distribution under an adaptive progressive type-Ⅱ censored sample

<abstract><p>In this paper, we investigate classical and Bayesian estimation of stress-strength reliability $\delta = P(X > Y)$ under an adaptive progressive type-Ⅱ censored sample. Assume that $X$ and $Y$ are independent random variables that follow inverse Weibull distribution with the same shape but different scale parameters. In classical estimation, the maximum likelihood estimator and asymptotic confidence interval are deduced. An approximate maximum likelihood estimator approach is used to obtain the explicit form. In Bayesian estimation, the Bayesian estimators are derived based on symmetric entropy loss function and LINEX loss function. Due to the complexity of integrals, we proposed Lindley's approximation to get the approximate Bayesian estimates. To compare the different estimators, we performed Monte Carlo simulations. Under gamma prior, the approximate maximum likelihood estimator performs better than Bayesian estimators. Under non-informative prior, the approximate maximum likelihood estimator has the same behavior as Bayesian estimators. In the end, two data sets are used to prove the effectiveness of the proposed methods.</p></abstract>

Computing Pseudolikelihood Estimators for Exponential-Family Random Graph Models

The reputation of the maximum pseudolikelihood estimator (MPLE) for Exponential Random Graph Models (ERGM) has undergone a drastic change over the past 30 years. While first receiving broad support, mainly due to its computational feasibility and the lack of alternatives, general opinions started to change with the introduction of approximate maximum likelihood estimator (MLE) methods that became practicable due to increasing computing power and the introduction of MCMC methods. Previous comparison studies appear to yield contradicting results regarding the preference of these two point estimators; however, there is consensus that the prevailing method to obtain an MPLE’s standard error by the inverse Hessian matrix generally underestimates standard errors. We propose replacing the inverse Hessian matrix by an approximation of the Godambe matrix that results in confidence intervals with appropriate coverage rates and that, in addition, enables examining for model degeneracy. Our results also provide empirical evidence for the asymptotic normality of the MPLE under certain conditions.

Venc Design and Velocity Estimation for Phase Contrast MRI.

In phase-contrast magnetic resonance imaging (PC-MRI), spin velocity contributes to the phase measured at each voxel. Therefore, estimating velocity from potentially wrapped phase measurements is the task of solving a system of noisy congruence equations. We propose Phase Recovery from Multiple Wrapped Measurements (PRoM) as a fast, approximate maximum likelihood estimator of velocity from multi-coil data with possible amplitude attenuation due to dephasing. The estimator can recover the fullest possible extent of unambiguous velocities, which can greatly exceed twice the highest venc. The estimator uses all pairwise phase differences and the inherent correlations among them to minimize the estimation error. Correlations are directly estimated from multi-coil data without requiring knowledge of coil sensitivity maps, dephasing factors, or the actual per-voxel signal-to-noise ratio. Derivation of the estimator yields explicit probabilities of unwrapping errors and the probability distribution for the velocity estimate; this, in turn, allows for optimized design of the phase-encoded acquisition. These probabilities are also incorporated into spatial post-processing to further mitigate wrapping errors. Simulation, phantom, and in vivo results for three-point PC-MRI acquisitions validate the benefits of reduced estimation error, increased recovered velocity range, optimized acquisition, and fast computation. A phantom study at 1.5T demonstrates 48.5% decrease in root mean squared error using PRoM with post-processing versus a conventional "dual-venc" technique. Simulation and 3T in vivo results likewise demonstrate the proposed benefits.