Approximate Maximum Likelihood Estimator Research Articles

Reliability Estimation of Scale Parameter in Type II Generalized Half-Logistic Distribution from Doubly Censored Samples Based on Monte Carlo Simulation

Objectives: This article deals with estimation of the reliability function for one-parameter Type II Generalized Half- Logistic distribution (Type II GHLD). Methods: The Maximum likelihood (ML) and Approximate Maximum likelihood (AML) methods provided by Jilani et.al. 1 . Findings: Comparisons between the estimators were made using Monte Carlo Simulation based on statistical indicter bias and mean squared error (MSE). We have generated 3000 random samples of size n=10(5)20 from a standard one-parameter Type II GHLD with = 0.5, 1.5 and 2.0. Each sample is then ordered and from this sample a doubly (including left and right) censored sample is considered, by censoring r smallest observations and s largest observations with all possible choices of r and s corresponding to this resultant sample we compute MLEs (using Newton-Raphson method), the LAMLE and unbiased LAMLE . Finally, the results are explained with tables. Novelty: This study has empirically proves that all the three reliability estimators have the same asymptotic variance. Hence, in this case, any one of the three estimators can be used but preferable estimator based unbiased estimator. Keywords: Maximum likelihood estimation, Approximate Maximum likelihood estimation, Bias, Mean squared error, Variances

Applying the Expectation-Maximization Algorithm to Multivariate Signal Extraction

Understanding time series signal extraction is vital for federal agencies, especially as data collection accelerates. One natural extension is moving from univariate to multivariate signal extraction, which offers the promise of reducing extraction error by exploiting cross-sectional relationships. However, such an extension ushers in new computational challenges, viz. larger parameter spaces and more complicated objective functions. The Expectation-Maximization (EM) algorithm provides a methodology to implicitly compute (or approximate) maximum likelihood estimates (MLEs). This paper provides methodology for applying the EM algorithm to a class of latent component multivariate time series models that allow for a nuanced specification of the unobserved signal. We derive an explicit formula for the maximization step, which facilitates computation speed while also improving the stability of the algorithm. Numerical studies demonstrate EM’s ability to efficiently compute MLEs in low-dimensional systems, while also providing feasible estimates in moderate-dimensional systems where MLEs are infeasible to compute. Applications to monthly housing starts and daily immigration are provided.

Enhanced Laplace approximation

The Laplace approximation has been proposed as a method for approximating the marginal likelihood of statistical models with latent variables. However, the approximate maximum likelihood estimators derived from the Laplace approximation are often biased for binary or temporally and/or spatially correlated data. Additionally, the corresponding Hessian matrix tends to underestimates the standard errors of these approximate maximum likelihood estimators. While higher-order approximations have been suggested, they are not applicable to complex models, such as correlated random effects models, and fail to provide consistent variance estimators. In this paper, we propose an enhanced Laplace approximation that provides the true maximum likelihood estimator and its consistent variance estimator. We study its relationship with the variational Bayes method. We also define a new restricted maximum likelihood estimator for estimating dispersion parameters and study their asymptotic properties. Enhanced Laplace approximation generally demonstrates how to obtain the true restricted maximum likelihood estimators and their variance estimators. Our numerical studies indicate that the enhanced Laplace approximation provides a satisfactory maximum likelihood estimator and restricted maximum likelihood estimator, as well as their variance estimators in the frequentist perspective. The maximum likelihood estimator and restricted maximum likelihood estimator can be also interpreted as the posterior mode and marginal posterior mode under flat priors, respectively. Furthermore, we present some comparisons with Bayesian procedures under different priors.

Exact selective inference with randomization

Summary We introduce a pivot for exact selective inference with randomization. Not only does our pivot lead to exact inference in Gaussian regression models, but it is also available in closed form. We reduce this problem to inference for a bivariate truncated Gaussian variable. By doing so, we give up some power that is achieved with approximate maximum likelihood estimation in Panigrahi & Taylor (2023). Yet our pivot always produces narrower confidence intervals than a closely related data-splitting procedure. We investigate the trade-off between power and exact selective inference on simulated datasets and an HIV drug resistance dataset.

Inference for compound truncated Poisson log-normal model with application to maximum precipitation data

The main objective of this paper is to propose a new general family of distributions, namely compound truncated Poisson log-normal distribution of which log-normal distribution is a special case. The proposed model has three unknown parameters, and it can take variety of shapes. It can be used effectively in analyzing maximum precipitation data during a particular period of time obtained from different stations. It is assumed that the number of stations operate follows a zero-truncated Poisson random variables, and the daily precipitation follows a log-normal random variable. The maximum likelihood estimators can be obtained quite conveniently using Expectation-Maximization (EM) algorithm. Approximate maximum likelihood estimators are also derived. The associated confidence intervals can also be obtained from the observed Fisher information matrix. Simulation results have been performed to check the performance of the EM algorithm, and it is observed that the EM algorithm works quite well in this case. When we analyze the precipitation data set using the proposed model it is observed that the proposed model provides better fit than some of the existing models.

Maximum likelihood estimation for discrete latent variable models via evolutionary algorithms

We propose an evolutionary optimization method for maximum likelihood and approximate maximum likelihood estimation of discrete latent variable models. The proposal is based on modified versions of the expectation–maximization (EM) and variational EM (VEM) algorithms, which are based on the genetic approach and allow us to accurately explore the parameter space, reducing the chance to be trapped into one of the multiple local maxima of the log-likelihood function. Their performance is examined through an extensive Monte Carlo simulation study where they are employed to estimate latent class, hidden Markov, and stochastic block models and compared with the standard EM and VEM algorithms. We observe a significant increase in the chance to reach global maximum of the target function and a high accuracy of the estimated parameters for each model. Applications focused on the analysis of cross-sectional, longitudinal, and network data are proposed to illustrate and compare the algorithms.

Self-localisation of IoT devices with imperfect measurements via approximate maximum-likelihood estimation

Self-localisation of IoT devices with imperfect measurements via approximate maximum-likelihood estimation

Approximate Maximum Likelihood Estimation in Fractional Stochastic Transport Equation

We estimate the drift of the fractional stochastic transport equation the by maximum likelihood and the minimum contrast methods. We show consistency and asymptotic normality of the estimators. We consider both continuous and discrete time observations.

Testing Data Cloning as the Basis of an Estimator for the Stochastic Volatility in Mean Model

Developed as a refinement of stochastic volatility (SV) models, the stochastic volatility in mean (SVM) model incorporates the latent volatility as an explanatory variable in both the mean and variance equations. It, therefore, provides a way of assessing the relationship between returns and volatility, albeit at the expense of complicating the estimation process. This study introduces a Bayesian methodology that leverages data-cloning algorithms to obtain maximum likelihood estimates for SV and SVM model parameters. Adopting this Bayesian framework allows approximate maximum likelihood estimates to be attained without the need to maximize pseudo likelihood functions. The key contribution this paper makes is that it proposes an estimator for the SVM model, one that uses Bayesian algorithms to approximate the maximum likelihood estimate with great effect. Notably, the estimates it provides yield superior outcomes than those derived from the Markov chain Monte Carlo (MCMC) method in terms of standard errors, all while being unaffected by the selection of prior distributions.

Source Localization Based on Hybrid AOA, TDOA, and RSS Measurements

In this paper, a constrained weighted least squares (CWLS) problem with two quadratic constraints for hybrid AOA, TDOA, and RSS localization is formulated. The Lagrange multiplier method cannot be efficiently used in this situation, so we provide four methods to solve the CWLS problem: 1) weighted least squares (WLS); 2) iterative constrained weighted least squares (ICWLS); 3) Lagrange multiplier method combined with Newton’s method (LMM+NM); 4) semidefinite programming (SDP). Furthermore, in order to obtain a good balance between accuracy and computational burden, we develop a fixed point iteration (FPI)-based algorithm that is derived from an approximate maximum likelihood estimator (MLE). Simulation results show that the accuracy of the proposed FPI method is better than the other four methods; besides, its computational burden is one in ten of the SDP algorithm.

Regression of binary network data with exchangeable latent errors

AbstractUndirected, binary network data consist of indicators of symmetric relations between pairs of actors. Regression models of such data allow for the estimation of effects of exogenous covariates on the network and for prediction of unobserved data. Ideally, estimators of the regression parameters should account for the inherent dependencies among relations in the network that involve the same actor. To account for such dependencies, researchers have developed a host of latent variable network models; however, estimation of many latent variable network models is computationally onerous and which model is best to base inference upon may not be clear. We propose the probit exchangeable (PX) model for undirected binary network data that is based on an assumption of exchangeability, which is common to many of the latent variable network models in the literature. The PX model can represent the first two moments of any exchangeable network model. We leverage the EM algorithm to obtain an approximate maximum likelihood estimator of the PX model that is extremely computationally efficient. Using simulation studies, we demonstrate the improvement in estimation of regression coefficients of the proposed model over existing latent variable network models. In an analysis of purchases of politically aligned books, we demonstrate political polarization in purchase behavior and show that the proposed estimator significantly reduces runtime relative to estimators of latent variable network models, while maintaining predictive performance.

Subspace-based distributed target detection in compound-Gaussian clutter

This paper investigates the problem of radar detection of distributed targets embedded in unknown compound-Gaussian clutter. To solve the problem, we first derive two detectors for fixed covariance matrix according to Rao and Wald tests with the use of order statistics theory. Then, the unknown covariance matrix is estimated by the approximate maximum likelihood (AML) estimation. Numerical examples through simulated and real data show that the Wald test-based detector has a higher probability of detection (PD) than existing detectors.

Semiparametric approach to estimation of marginal mean effects and marginal quantile effects

We consider a semiparametric generalized linear model and study estimation of both marginal mean effects and marginal quantile effects in this model. We propose an approximate maximum likelihood estimator, and rigorously establish the consistency, the asymptotic normality, and the semiparametric efficiency of our method in both the marginal mean effect and the marginal quantile effect estimation. Simulation studies are conducted to illustrate the finite sample performance, and we apply the new tool to analyze a Swiss non-labor income data and discover a new interesting predictor.

Unsupervised mixture estimation via approximate maximum likelihood based on the Cramér - von Mises distance

Mixture distributions with dynamic weights are an efficient way of modeling loss data characterized by heavy tails. However, maximum likelihood estimation of this family of models is difficult, mostly because of the need to evaluate numerically an intractable normalizing constant. In such a setup, simulation-based estimation methods are an appealing alternative. The approximate maximum likelihood estimation (AMLE) approach is employed. It is a general method that can be applied to mixtures with any component densities, as long as simulation is feasible. The focus is on the dynamic lognormal-generalized Pareto distribution, and the Cramér - von Mises distance is used to measure the discrepancy between observed and simulated samples. After deriving the theoretical properties of the estimators, a hybrid procedure is developed, where standard maximum likelihood is first employed to determine the bounds of the uniform priors required as input for AMLE. Simulation experiments and two real-data applications suggest that this approach yields a major improvement with respect to standard maximum likelihood estimation.

Approximate MLEs for the location and scale parameters of the Poisson-half-logistic distribution

Recently, the application of compound distributions has increased due to the flexibility in fitting to actual data in various fields such as economics, insurance, etc. Poisson-half-logistic distribution is one of these distributions with an increasing-constant hazard rate that can be used in parallel systems and complementary risk models. Because of the complexity of the form of this distribution, it is not possible to obtain classical parameter estimates (such as MLE) by the analytical method for the location and scale parameters. We present a simple way of deriving explicit estimators by approximating the likelihood equations appropriately. This paper presents AMLE (Approximate MLE) method to obtain the location and scale parameters estimation. Using simulation, we show that this method is as efficient as the maximum likelihood estimators (MLEs), we obtain the variance of estimators from the inverse of the observed Fisher information matrix, and we see that when sample size increases bias and variance of these estimators, MSEs of parameters decrease. Finally, we present a numerical example to illustrate the methods of inference developed here.

Le Cam–Stratonovich–Boole theory for Itô diffusions

Abstract We connect the theory of local asymptotic normality (LAN) of Le Cam to Boole’s approximation of the Stratonovich stochastic integral by estimating the parameter in the nonlinear drift coefficient of an ergodic diffusion process satisfying a homogeneous Itô stochastic differential equation based on discretely spaced dense observations of the process. The asymptotic normality and local asymptotic minimaxity (in the Hajek–Le Cam sense) of approximate maximum likelihood estimators, approximate maximum probability estimators and approximate Bayes estimators based on Itô and Boole’s approximations of the continuous likelihood are obtained under an almost slowly increasing experimental design (ASIED) condition ( T n 6 / 7 → 0 {\frac{T}{n^{6/7}}\to 0} as T → ∞ {T\to\infty} and n → ∞ {n\to\infty} , where T is the length of the observation time and n is the number of observations) through the weak convergence of the approximate likelihood ratio random fields. Among other things, the Bernstein–von Mises type theorems concerning the convergence of suitably normalized and centered approximate posterior distributions to normal distribution under the same design condition are proved. Asymptotic normality and asymptotic efficiency of the conditional least squares estimator under the same design condition are obtained as a by-product. The log-likelihood derivatives based on Itô approximations are martingales, but the log-likelihood derivatives based on Boole’s approximations are not martingales but weighted averages of forward and backward martingales. These new approximations have faster rate of convergence than the martingale approximations. The methods would have advantages over Euler and Milstein approximations for Monte Carlo simulations.

Proportional stochastic generalized Lotka–Volterra model with an application to learning microbial community structures

Inferring microbial community structure based on temporal metagenomics data is an important goal in microbiome studies. The deterministic generalized Lotka–Volterra (GLV) differential equations have been commonly used to model the dynamics of microbial taxa. However, these approaches fail to take random environmental fluctuations into account and usually ignore the compositional nature of relative abundance data, which may deteriorate the estimates. In this article, we consider the microbial dynamics in terms of relative abundances by introducing a reference taxon, and propose a new proportional stochastic GLV (pSGLV) differential equation model, where the random perturbations of Brownian motion in this model can naturally account for the external environmental effects on the microbial community. We establish conditions and show some mathematical properties of the solutions including general existence and uniqueness, stochastic ultimate boundedness, stochastic permanence, the existence of stationary distribution, and ergodicity property. We further develop approximate maximum likelihood estimators (AMLEs) based on discrete observations and systematically investigate the consistency and asymptotic normality of the proposed estimators. At last, numerical simulations support our theoretical findings and our method is demonstrated through an application to the well-known “moving picture” temporal microbial dataset.

Fast, Approximate Maximum Likelihood Estimation of Log-Gaussian Cox Processes

Log-Gaussian Cox processes (LGCPs) offer a framework for regression-style modeling of point patterns that can accommodate spatial latent effects. These latent effects can be used to account for missing predictors or other sources of clustering that could not be explained by a Poisson process. Fitting LGCP models can be challenging because the marginal likelihood does not have a closed form and it involves a high dimensional integral to account for the latent Gaussian field. We propose a novel methodology for fitting LGCP models that addresses these challenges using a combination of variational approximation and reduced rank interpolation. Additionally, we implement automatic differentiation to obtain exact gradient information, for computationally efficient optimization and to consider integral approximation using the Laplace method. We demonstrate the method’s performance through both simulations and a real data application, with promising results in terms of computational speed and accuracy compared to that of existing approaches. Supplementary material for this article is available online.

Uncertainty quantification in operational modal analysis of time-varying structures based on time-dependent autoregressive moving average model

The identified modal parameters in Operational Modal Analysis (OMA) are always subject to statistical uncertainties from many sources and the uncertainty is also important to assess the quality of the identified results. This study proposes an uncertainty quantification method in OMA for time-varying structures based on the Functional Series Time-dependent AutoRegressive Moving Average (FS-TARMA) model. A multi-stage pseudo-linear optimization scheme is first adopted to obtain the approximate Maximum Likelihood Estimates (MLEs) of FS-TARMA model parameters and the covariance is computed as the inverse Hessian matrix of the Negative Log-Likelihood Function (NLLF) at the MLEs. Finally, the approximate closed-form covariance of modal parameters is derived by the first-order sensitivity method. The proposed method is verified by two numerical examples and an experimental beam. Furthermore, the ability of the uncertainty to extinguish the spurious modes caused by colored noise excitation and the noise modes caused by measurement noise or inappropriate model structure is also demonstrated in these examples.

Reliability Estimation for the Inverse Weibull Distribution Under Adaptive Type-II Progressive Hybrid Censoring: Comparative Study

The aim of this study is to investigate different methods of estimating the stress-strength reliability parameter, $\theta =P(Y<X)$, when the strength (X) and the stress (Y) are independent random variables taken from the inverse Weibull distribution (IWD), with the same shape parameter and different scale parameters. Based on Adaptive Type-II Hybrid progressive censored samples, we employ classical and Bayesian approaches. In the classical approach, we use the maximum likelihood estimator (MLE), the approximate maximum likelihood estimator (AMLE), and the least squares estimator (LSE). In contrast, the Bayesian approach utilizes symmetric and asymmetric loss functions. Due to the absence of explicit forms for Bayes estimators, we propose using Lindley's approximation method for computing the Bayes estimators. We compare these estimators using extensive simulations and two criteria: the bias and the mean square error (MSE). Finally, two real-life data examples are provided for illustrations.