Approximate Maximum Likelihood Estimator Research Articles

Detecting Co-Movements in Noncausal Time Series

This paper introduces the notion of common noncausal features and proposes tools to detect them in multivariate time series models. We argue that the existence of co-movements might not be detected using the conventional stationary vector autoregressive (VAR) model as the common dynamics are present in the noncausal (i.e. forward-looking) component of the series. In particular, we show that the presence of a reduced rank structure allows to identify purely causal and noncausal VAR processes of order two and higher even in the Gaussian likelihood framework. Hence, usual test statistics and canonical correlation analysis can still be applied, where both lags and leads are used as instruments to determine whether the common features are present in either the backward- or forward-looking dynamics of the series. The proposed de finitions of co-movements also valid for the mixed causal-noncausal VAR, with the exception that an approximate non-Gaussian maximum likelihood estimator is necessary for these cases. This means however that one loses the bene fits of the simple tools proposed in this paper. An empirical analysis on European Brent and U.S. West Texas Intermediate oil prices illustrates the main fi ndings. Whereas we fail to fi nd any short run co-movements in a conventional causal VAR, they are detected in the growth rates of the series when considering a purely noncausal VAR.

Detection of Sources in Non-Negative Blind Source Separation by Minimum Description Length Criterion.

While non-negative blind source separation (nBSS) has found many successful applications in science and engineering, model order selection, determining the number of sources, remains a critical yet unresolved problem. Various model order selection methods have been proposed and applied to real-world data sets but with limited success, with both order over- and under-estimation reported. By studying existing schemes, we have found that the unsatisfactory results are mainly due to invalid assumptions, model oversimplification, subjective thresholding, and/or to assumptions made solely for mathematical convenience. Building on our earlier work that reformulated model order selection for nBSS with more realistic assumptions and models, we report a newly and formally revised model order selection criterion rooted in the minimum description length (MDL) principle. Adopting widely invoked assumptions for achieving a unique nBSS solution, we consider the mixing matrix as consisting of deterministic unknowns, with the source signals following a multivariate Dirichlet distribution. We derive a computationally efficient, stochastic algorithm to obtain approximate maximum-likelihood estimates of model parameters and apply Monte Carlo integration to determine the description length. Our modeling and estimation strategy exploits the characteristic geometry of the data simplex in nBSS. We validate our nBSS-MDL criterion through extensive simulation studies and on four real-world data sets, demonstrating its strong performance and general applicability to nBSS. The proposed nBSS-MDL criterion consistently detects the true number of sources, in all of our case studies.

A characteristic function-based approach to approximate maximum likelihood estimation

ABSTRACTThe choice of the summary statistics in approximate maximum likelihood is often a crucial issue. We develop a criterion for choosing the most effective summary statistic and then focus on the empirical characteristic function. In the iid setting, the approximating posterior distribution converges to the approximate distribution of the parameters conditional upon the empirical characteristic function. Simulation experiments suggest that the method is often preferable to numerical maximum likelihood. In a time-series framework, no optimality result can be proved, but the simulations indicate that the method is effective in small samples.

Estimation of Synaptic Conductances in Presence of Nonlinear Effects Caused by Subthreshold Ionic Currents.

Subthreshold fluctuations in neuronal membrane potential traces contain nonlinear components, and employing nonlinear models might improve the statistical inference. We propose a new strategy to estimate synaptic conductances, which has been tested using in silico data and applied to in vivo recordings. The model is constructed to capture the nonlinearities caused by subthreshold activated currents, and the estimation procedure can discern between excitatory and inhibitory conductances using only one membrane potential trace. More precisely, we perform second order approximations of biophysical models to capture the subthreshold nonlinearities, resulting in quadratic integrate-and-fire models, and apply approximate maximum likelihood estimation where we only suppose that conductances are stationary in a 50–100 ms time window. The results show an improvement compared to existent procedures for the models tested here.

Bayesian and Classical Estimation of Stress-Strength Reliability for Inverse Weibull Lifetime Models

In this paper, we consider the problem of estimating stress-strength reliability for inverse Weibull lifetime models having the same shape parameters but different scale parameters. We obtain the maximum likelihood estimator and its asymptotic distribution. Since the classical estimator doesn’t hold explicit forms, we propose an approximate maximum likelihood estimator. The asymptotic confidence interval and two bootstrap intervals are obtained. Using the Gibbs sampling technique, Bayesian estimator and the corresponding credible interval are obtained. The Metropolis-Hastings algorithm is used to generate random variates. Monte Carlo simulations are conducted to compare the proposed methods. Analysis of a real dataset is performed.

Maximum Likelihood TDOA Estimation From Compressed Sensing Samples Without Reconstruction

One application for time-difference-of-arrival (TDOA) estimation is in emitter localization. A signal from an emitter reaching a group of sensors, each in a separate location, will have different arrival times. Finding the TDOAs between the output of pairs of sensors will provide the necessary measurements for the hyperbolic localization of the emitter. When the sensors acquire the signal by compressed sensing (CS), their outputs are reduced dimension linear transformation of the time samples of the signal. This shuffling of the time samples breaks up their time relation. Thus, a cross correlation of the CS output of two sensors cannot determine the TDOA. To apply cross correlation, it is necessary to reconstruct the time samples. This letter proposes an alternative that uses only the coefficients of the discrete Fourier transform (DFT) of the CS samples. It begins with the derivation of the maximum likelihood (ML) equation and the ML estimator. This estimator requires known values of signal and noise powers. Substituting these values by their estimates lead to the approximate ML estimator. The phase of the product of two DFT coefficients from each sensor is proportional to the unknown TDOA. Hence, these coefficients can provide an estimation of the TDOA. Simulation results show that although ML is the best, as expected, all these estimators have very close performance.

Parameterization of nonideal quantizers for simultaneous estimation of quantizer and excitation signal parameters

This paper deals with the estimation of signal and quantizer parameters based on a nonideally sampled and quantized measurement record. It proposes solutions to reduce the inherently high-dimensional parameter space via approximation of the quantizer nonideality to make the computation possible in PC/MATLAB environment. The performance of the approximation techniques is examined as well. The computational demand of the proposed approximate maximum likelihood (AML) estimators is investigated and compared to the computational demand of the maximum likelihood (ML) estimators without approximation.

Gini index based goodness-of-fit test for the logistic distribution

ABSTRACTTo model growth curves in survival analysis and biological studies the logistic distribution has been widely used. In this article, we propose a goodness-of-fit test for the logistic distribution based on an estimate of the Gini index. The exact distribution of the proposed test statistic and also its asymptotic distribution are presented. In order to compute the proposed test statistic, parameters of the logistic distribution are estimated by approximate maximum likelihood estimators (AMLEs), which are simple explicit estimators. Through Monte Carlo simulations, power comparisons of the proposed test with some known competing tests are carried. Finally, an illustrative example is presented and analyzed.

Estimation for the scaled half-logistic distribution under Type-I progressively hybrid censoring scheme

ABSTRACTThis paper addresses the estimation for the unknown scale parameter of the half-logistic distribution based on a Type-I progressively hybrid censoring scheme. We evaluate the maximum likelihood estimate (MLE) via numerical method, and EM algorithm, and also the approximate maximum likelihood estimate (AMLE). We use a modified acceptance rejection method to obtain the Bayes estimate and corresponding highest posterior confidence intervals. We perform Monte Carlo simulations to compare the performances of the different methods, and we analyze one dataset for illustrative purposes.

Testing the validity of the logistic model based on the empirical distribution function

ABSTRACTThe logistic distribution is one of the fundamental distribution and is widely used for describing model growth curves in survival analysis and biological studies. Applications of this distribution are presented in statistical literature. In this article, goodness of fit tests for the logistic distribution based on the empirical distribution function (EDF) are considered. In order to compute the test statistics, because the MLEs cannot be obtained explicitly, we use the approximate maximum likelihood estimates (AMLEs) suggested by Balakrishnan and Cohen (1990), which are simple explicit estimators. Power comparisons of the considered tests are carried out via simulations. Finally, two illustrative examples are presented and analyzed.

A Variational Maximization-Maximization Algorithm for Generalized Linear Mixed Models with Crossed Random Effects.

We present a variational maximization-maximization algorithm for approximate maximum likelihood estimation of generalized linear mixed models with crossed random effects (e.g., item response models with random items, random raters, or random occasion-specific effects). The method is based on a factorized variational approximation of the latent variable distribution given observed variables, which creates a lower bound of the log marginal likelihood. The lower bound is maximized with respect to the factorized distributions as well as model parameters. With the proposed algorithm, a high-dimensional intractable integration is translated into a two-dimensional integration problem. We incorporate an adaptive Gauss-Hermite quadrature method in conjunction with the variational method in order to increase computational efficiency. Numerical studies show that under the small sample size conditions that are considered the proposed algorithm outperforms the Laplace approximation.

Systematic inference of the long-range dependence and heavy-tail distribution parameters of ARFIMA models

Long-Range Dependence (LRD) and heavy-tailed distributions are ubiquitous in natural and socio-economic data. Such data can be self-similar whereby both LRD and heavy-tailed distributions contribute to the self-similarity as measured by the Hurst exponent. Some methods widely used in the physical sciences separately estimate these two parameters, which can lead to estimation bias. Those which do simultaneous estimation are based on frequentist methods such as Whittle’s approximate maximum likelihood estimator. Here we present a new and systematic Bayesian framework for the simultaneous inference of the LRD and heavy-tailed distribution parameters of a parametric ARFIMA model with non-Gaussian innovations. As innovations we use the α-stable and t-distributions which have power law tails. Our algorithm also provides parameter uncertainty estimates. We test our algorithm using synthetic data, and also data from the Geostationary Operational Environmental Satellite system (GOES) solar X-ray time series. These tests show that our algorithm is able to accurately and robustly estimate the LRD and heavy-tailed distribution parameters.

Estimating Spatial Autocorrelation With Sampled Network Data

Spatial autocorrelation is a parameter of importance for network data analysis. To estimate spatial autocorrelation, maximum likelihood has been popularly used. However, its rigorous implementation requires the whole network to be observed. This is practically infeasible if network size is huge (e.g., Facebook, Twitter, Weibo, WeChat, etc.). In that case, one has to rely on sampled network data to infer about spatial autocorrelation. By doing so, network relationships (i.e., edges) involving unsampled nodes are overlooked. This leads to distorted network structure and underestimated spatial autocorrelation. To solve the problem, we propose here a novel solution. By temporarily assuming that the spatial autocorrelation is small, we are able to approximate the likelihood function by its first-order Taylor’s expansion. This leads to the method of approximate maximum likelihood estimator (AMLE), which further inspires the development of paired maximum likelihood estimator (PMLE). Compared with AMLE, PMLE is computationally superior and thus is particularly useful for large-scale network data analysis. Under appropriate regularity conditions (without assuming a small spatial autocorrelation), we show theoretically that PMLE is consistent and asymptotically normal. Numerical studies based on both simulated and real datasets are presented for illustration purpose.

A Low-Cost and Robust Maximum Likelihood Joint Estimator for the Doppler Spread and CFO Parameters Over Flat-Fading Rayleigh Channels

This paper addresses the problem of Doppler spread and carrier frequency offset (CFO) estimation under flat-fading Rayleigh channels. We develop a new low-cost and robust approximate maximum likelihood (ML) estimator for these two key parameters that builds upon an elegant two-ray approximation model of the channel’s covariance matrix. The latter is then inverted analytically thereby yielding a closed-form expression for the underlying log-likelihood function that is prone to easy evaluation by the fast Fourier transform. Computer simulations show that the new estimator is accurate over wide ranges of the Doppler spread and CFO parameters. Moreover, it outperforms many state-of-the-art techniques under the adverse conditions of short data records and/or low SNR thresholds. Most prominently, it exhibits an unprecedented robustness to the Doppler spectrum shape of the channel since it does not require its a priori knowledge.

An improved direct position determination method based on correlation accumulation of short-time signals with variable velocity receivers

Wireless target location technology has been widely used in civil and military fields. In the two-step localization algorithms, the signal measurements, such as thetime of arrival, theangle of arrival, the frequency difference of arrival, etc., should be extracted first from the receivedsource signal. Then the target position is identified by calculating the location equation. Compared with the two-step localization algorithms, the direct position determination (DPD) method, which need not estimate the signal parameters and calculate the position step by step, but obtainsthe source position from the received signals directly based on the maximum likelihood criterion, has been shown to have a goodestimation accuracy and robustness, especially under low signal-to-noise ratio (SNR) conditions. So it has been widely studied in recent years and has made remarkable achievements in academic research. However, the DPD algorithm of wideband signals emitters is not performing well with moving receivers in the joint positioning based on time delay and Doppler shift under the low SNRs. To obtaina better positioning performance, in this paper we present a DPD algorithm with variable velocity receivers based on coherent summation of short-time signal segments, and derive the source position Cramer-Rao lower bound (CRLB). The algorithm designs a positioning model in which the multiple variable velocity receivers are usedtoobtain the source signal, then the signal received at the same receiver is patitioned into multiple non-overlapping short-time signal segments, based on which, an approximate maximum likelihood estimator for the new DPD algorithm is developed. The algorithm makes full use of the location information contained in the coherency among the signals segments, while extra target position information is acquired through the speed variability in the positioning model, and thus the problem of location ambiguity is solved. The simulation results show thatthe algorithm proposed in this paper further improves the positioning performance, and outperforms the traditional DPD algorithms with more accurate results. Especially in the low SNR, it is closer to the CRLB.

Approximate maximum likelihood estimation of the Bingham distribution

Maximum likelihood estimation of the Bingham distribution is difficult because the density function contains a normalization constant that cannot be computed in closed form. Given the availability of sufficient statistics, Approximate Maximum Likelihood Estimation (AMLE) is an appealing method that allows one to bypass the evaluation of the likelihood function. The impact of the input parameters of the AMLE algorithm is investigated and some methods for choosing their numerical values are suggested. Moreover, AMLE is compared to the standard approach which numerically maximizes the (approximate) likelihood obtained with the normalization constant estimated via the Holonomic Gradient Method (HGM). For the Bingham distribution on the sphere, simulation experiments and real-data applications produce similar outcomes for both methods. On the other hand, AMLE outperforms HGM when the dimension increases.

Statistical analysis for competing risks model from a Weibull distribution under progressively hybrid censoring

ABSTRACTThis paper considers the statistical analysis for competing risks model under the Type-I progressively hybrid censoring from a Weibull distribution. We derive the maximum likelihood estimates and the approximate maximum likelihood estimates of the unknown parameters. We then use the bootstrap method to construct the confidence intervals. Based on the non informative prior, a sampling algorithm using the acceptance–rejection sampling method is presented to obtain the Bayes estimates, and Monte Carlo method is employed to construct the highest posterior density credible intervals. The simulation results are provided to show the effectiveness of all the methods discussed here and one data set is analyzed.

Generalized Fractional Processes with Long Memory and Time Dependent Volatility Revisited

In recent years, fractionally-differenced processes have received a great deal of attention due to their flexibility in financial applications with long-memory. This paper revisits the class of generalized fractionally-differenced processes generated by Gegenbauer polynomials and the ARMA structure (GARMA) with both the long-memory and time-dependent innovation variance. We establish the existence and uniqueness of second-order solutions. We also extend this family with innovations to follow GARCH and stochastic volatility (SV). Under certain regularity conditions, we give asymptotic results for the approximate maximum likelihood estimator for the GARMA-GARCH model. We discuss a Monte Carlo likelihood method for the GARMA-SV model and investigate finite sample properties via Monte Carlo experiments. Finally, we illustrate the usefulness of this approach using monthly inflation rates for France, Japan and the United States.

Analysis of Masked Data in a Series System Subjected to Sources of Shocks Under Type I Progressive Hybrid Censoring

Abstract In this paper, we consider the statistical analysis of masked data based on the fatal shock model under type I progressive hybrid censoring. It is assumed that the time of shock follows Weibull distribution. Maximum-likelihood estimators for the unknown parameters are obtained by solving a one-dimensional optimization problem. Approximate maximum-likelihood estimators have been proposed based on a Taylor series expansion, and they have explicit expressions. In addition, the Bayesian approach, combined with Gibbs sampling, are developed based on the assumption that the shape parameter has a log-concave function, and for the given shape parameter, the scale parameters have Gamma-Dirichlet priors. Finally, Monte Carlo simulations are performed to compare the performances of the proposed methods under different progressive censoring schemes and masking levels.

Approximate maximum likelihood estimation using data-cloning ABC

A maximum likelihood methodology for a general class of models is presented, using an approximate Bayesian computation (ABC) approach. The typical target of ABC methods is models with intractable likelihoods, and we combine an ABC-MCMC sampler with so-called “data cloning” for maximum likelihood estimation. Accuracy of ABC methods relies on the use of a small threshold value for comparing simulations from the model and observed data. The proposed methodology shows how to use large threshold values, while the number of data-clones is increased to ease convergence towards an approximate maximum likelihood estimate. We show how to exploit the methodology to reduce the number of iterations of a standard ABC-MCMC algorithm and therefore reduce the computational effort, while obtaining reasonable point estimates. Simulation studies show the good performance of our approach on models with intractable likelihoods such as g-and-k distributions, stochastic differential equations and state-space models.