ML Estimation Research Articles

Online Appendix: Estimation in a Generalization of Bivariate Probit Models with Dummy Endogenous Regressors

Full paper is available at: https://ssrn.com/abstract=3233422 This online appendix contains all the proofs of the main results in Han and Lee (2018), technical assumptions, and additional simulation results. Section A contains the proofs of the identification/non-identification results. Section B collects the proofs in the asymptotic theory of the sieve ML estimators, related technical assumptions, and other expressions. Finally, Section C shows more simulation results.

Small sample properties of ML estimator in Vasicek and CIR models: a simulation experiment

In this paper we analyze small sample properties of the ML estimation procedure in Vasicek and CIR models. In particular, we consider short time series, with a length between 20 and 200, typically values observed in the field of survival data. We perform a simulation study in order to investigate which properties of the parameter estimators still remain valid and to evaluate the effect of a bootstrap bias correction method. The results show that the bias of the estimators can be really strong for small samples and the relative bias seems to be worse when the true parameters of the models are near to the nonstationarity case. The bootstrap bias correction is enough efficient in correcting the bias also for very small sample sizes, but the increase in RMSE of the estimator is greater as much as smaller is the bias in the ML estimator. Moreover, the bootstrap correction does not improve the performance of the tests on the parameters.

Bayesian and classical estimation of reliability in a multicomponent stress-strength model under adaptive hybrid progressive censored data

The statistical inference of multicomponent stress-strength reliability under the adaptive Type-II hybrid progressive censored samples for the Weibull distribution is considered. It is assumed that both stress and strength are two Weibull independent random variables. We study the problem in three cases. First assuming that the stress and strength have the same shape parameter and different scale parameters, the maximum likelihood estimation (MLE), approximate maximum likelihood estimation (AMLE) and two Bayes approximations, due to the lack of explicit forms, are derived. Also, the asymptotic confidence intervals, two bootstrap confidence intervals and highest posterior density (HPD) credible intervals are obtained. In the second case, when the shape parameter is known, MLE, exact Bayes estimation, uniformly minimum variance unbiased estimator (UMVUE) and different confidence intervals (asymptotic and HPD) are studied. Finally, assuming that the stress and strength have the different shape and scale parameters, ML, AML and Bayesian estimations on multicomponent reliability have been considered. The performances of different methods are compared using the Monte Carlo simulations and for illustrative aims, one data set is investigated.

Estimating Parameters of a Directed Weighted Graph Model with Beta-Distributed Edge-Weights

We introduce a directed, weighted random graph model, where the edge-weights are independent and beta distributed with parameters depending on their endpoints. We will show that the row- and column-sums of the transformed edge-weight matrix are sufficient statistics for the parameters, and use the theory of exponential families to prove that the ML estimate of the parameters exists and is unique. Then an algorithm to find this estimate is introduced together with convergence proof that uses properties of the digamma function. Simulation results and applications are also presented.

Asymmetry, Fat-tail and Autoregressive Conditional Density in Daily Stocks Return Data

This paper focuses on the study of unbiasedness and efficiency of the maximum likelihood estimates of the GARCH (1,1) model volatility parameters when the error distribution assumed is Johnson s SU under varying skewness and kurtosis levels. The study is based on a simulation experiment and a real application to daily returns of five stock indices. In general the ML estimates of volatility parameters are found to be unbiased with high efficiency when the true distribution is asymmetric and fat-tailed for all levels of skewness and kurtosis and all parameter levels. Models with time-varying shape parameters are found to give a better fit than models with constant shape parameters.

Estimation of time-varying coefficient dynamic panel data models

In this paper, we consider dynamic panel data models where the autoregressive parameter changes over time. We propose the GMM and ML estimators for this model. We conduct Monte Carlo simulation to compare the performance of these two estimators. The simulation results show that the ML estimator outperforms the GMM estimator.

Subdistribution hazard models for competing risks in discrete time.

A popular modeling approach for competing risks analysis in longitudinal studies is the proportional subdistribution hazards model by Fine and Gray (1999. A proportional hazards model for the subdistribution of a competing risk. Journal of the American Statistical Association94, 496-509). This model is widely used for the analysis of continuous event times in clinical and epidemiological studies. However, it does not apply when event times are measured on a discrete time scale, which is a likely scenario when events occur between pairs of consecutive points in time (e.g., between two follow-up visits of an epidemiological study) and when the exact lengths of the continuous time spans are not known. To adapt the Fine and Gray approach to this situation, we propose a technique for modeling subdistribution hazards in discrete time. Our method, which results in consistent and asymptotically normal estimators of the model parameters, is based on a weighted ML estimation scheme for binary regression. We illustrate the modeling approach by an analysis of nosocomial pneumonia in patients treated in hospitals.

A Maximum Likelihood-Based Minimum Mean Square Error Separation and Estimation of Stationary Gaussian Sources From Noisy Mixtures

In the context of Independent Component Analysis (ICA), noisy mixtures pose a dilemma regarding the desired objective. On one hand, a "maximally separating" solution, providing the minimal attainable Interference-to-Source-Ratio (ISR), would often suffer from significant residual noise. On the other hand, optimal Minimum Mean Square Error (MMSE) estimation would yield estimates which are the "closest possible" to the true sources, often at the cost of compromised ISR. In this work, we consider noisy mixtures of temporally-diverse stationary Gaussian sources in a semi-blind scenario, which conveniently lends itself to either one of these objectives. We begin by deriving the ML Estimates (MLEs) of the unknown (deterministic) parameters of the model: the mixing matrix and the (possibly different) noise variances in each sensor. We derive the likelihood equations for these parameters, as well as the corresponding Cram\'er-Rao lower bound, and propose an iterative solution for obtaining the MLEs. Based on these MLEs, the asymptotically-optimal "maximally separating" solution can be readily obtained. However, we also present the ML-based MMSE estimate of the sources, alongside a frequency-domain-based computationally efficient scheme, exploiting their stationarity. We show that this estimate is asymptotically optimal and attains the (oracle) MMSE lower bound. Furthermore, for non-Gaussian signals, we show that this estimate serves as a Quasi ML (QML)-based Linear MMSE (LMMSE) estimate, and attains the (oracle) LMMSE lower bound asymptotically. Empirical results of three simulation experiments are presented, corroborating our analytical derivations.

Risks of Classification of the Gaussian Markov Random Field Observations

Given the spatial lattice endowed with particular neighborhood structure, the problem of classifying a scalar Gaussian Markov random field (GMRF) observation into one of two populations specified by different regression coefficients and special parametric covariance (precision) matrix is considered. Classification rule based on the plug-in Bayes discriminant function with inserted ML estimators of regression coefficients, spatial dependence and scale parameters is studied. The novel closed-form expression for the actual risk and the approximation of the expected risk (AER) associated with the aforementioned classifier are derived. This is the extension of the previous study of GMRF classification to the case of complete parametric uncertainty. Derived AER is used as the main performance measure for the considered classifier. GMRF sampled on a regular 2-dimensional unit spacing lattice endowed with neighborhood structure based on the Euclidean distance between sites is used for a simulation experiment. The sampling properties of ML estimators and the accuracy of the derived AER for various values of spatial dependence parameters and Mahalanobis distance are studied. The influence of the neighborhood size on the accuracy of the proposed AER is examined as well.

Approaches for Structural Investigations of Binary Data Using Confirmatory Factor Models

An investigation of the suitability of threshold-based and threshold-free approaches for structural investigations of binary data is reported. Both approaches implicitly establish a relationship between binary data following the binomial distribution on one hand and continuous random variables assuming a normal distribution on the other hand. In two simulation studies we investigated: whether the fit results confirm the establishment of such a relationship, whether the differences between correct and incorrect models are retained and to what degree the sample size influences the results. Both approaches proved to establish the relationship. Using the threshold-free approach it was achieved by customary ML estimation whereas robust ML estimation was necessary in the threshold-based approach. Discrimination between correct and incorrect models was observed for both approaches. Larger CFI differences were found for the threshold-free approach than for the threshold-based approach. Dependency on sample size characterized the threshold-based approach but not the threshold-free approach. The threshold-based approach tended to perform better in large sample sizes, while the threshold-free approach performed better in smaller sample sizes.

The Stochastic Stationary Root Model

We propose and study the stochastic stationary root model. The model resembles the cointegrated VAR model but is novel in that: (i) the stationary relations follow a random coefficient autoregressive process, i.e., exhibhits heavy-tailed dynamics, and (ii) the system is observed with measurement error. Unlike the cointegrated VAR model, estimation and inference for the SSR model is complicated by a lack of closed-form expressions for the likelihood function and its derivatives. To overcome this, we introduce particle filter-based approximations of the log-likelihood function, sample score, and observed Information matrix. These enable us to approximate the ML estimator via stochastic approximation and to conduct inference via the approximated observed Information matrix. We conjecture the asymptotic properties of the ML estimator and conduct a simulation study to investigate the validity of the conjecture. Model diagnostics to assess model fit are considered. Finally, we present an empirical application to the 10-year government bond rates in Germany and Greece during the period from January 1999 to February 2018.

Return Levels Approach and Periods of Currency Crises

Dating and observing currency crisis periods lie at the heart of much international researchers. This is due to the lack of agreement in one research methodology. Until today, there does not exist a single theory or specific international policy regulation that can explain this phenomenon in global. To identify the periods of currency crisis, many methods have been brought out. Literature first employed a combination of sample mean and standard deviation. Some recent studies have attempted to use extreme value theory (EVT). Although these procedures have been more criticized in most of the literature. These drawbacks of existing approaches give rise to a new approach which is the main goal of this research. The main purpose of this study is to employ return levels technique to date currency crisis periods. The study will discuss only one method the block maxima approach. The stress losses i.e the generalized extreme value (GEV) distribution will be fitted to the annual block maxima to estimate the T-year return levels of extreme exchange market pressure index (EMPI). The parameters of the GEV distribution are estimated using the ML estimator method. Beside, a detailed procedure of the new approach is implemented. A comparison study between our identification approach and the existing conventional approach in the most literature is also conducted. We further illustrate the method by an empirical study on identifying periods of currency crisis of Kenya as case study. For practical implement the study focuses only on one single currency crisis model known as the alternative EMP index model for the intent of arbitrating the performance among various techniques. Results suggest that our new approach (RLDT) is performing better than the conventional method when the return period is considered big. Nonetheless, our technique appears to dominate the existing conventional approaches. This paper covers only a small area of this growing field of research. Hopefully, our investigations to contribute to these efforts by showing that return level dating technique derived from stress-losses model may have a place in the toolbox of economists looking for more accurate techniques in predicting currency crises.

Statistical classification of multivariate conditionally autoregressive Gaussian random field observations

The problem of supervised classifying of a multivariate Gaussian Markov random field (GMRF) observation into one of two populations specified by different regression mean models is considered. We focus on a multivariate conditionally autoregressive model, a subclass of GMRF with parametrical structure proposed by Pettitt et al. (2002) and generalized by Jin et al. (2005). For complete parametric certainty and with fixed training sample locations, the formula of the Bayes error rate is derived. Plug-in Bayes discriminant function obtained by replacing the regression and scale parameters with their ML estimators in the Bayes discriminant function is investigated. The novel formulas for the actual error rate and the approximation of the expected error rate (AER) associated with plug-in Bayes discriminant function are derived. These results are multivariate generalizations of the univariate ones. A numerical analysis of the derived formulas is implemented for the bivariate conditionally autoregressive model sampling on a regular 2-dimensional unit spacing lattice with the neighborhood structure based on the Euclidean distance. A comparison of the proposed classifier performance for different parametric structures of populations is done by studying the accuracy of the derived AER. Finally, the influence of the neighborhood order on the accuracy of the derived AER is examined.

TOA-based source localization using ML estimation

This paper proposes the Time of arrival (TOA) measurement model for finding the position of a stationary emitting source for Line-of-Sight (LOS) scenario. Here Maximum Likelihood Estimation (MLE) is used as the positioning algorithm. For approximation of the roots of the solution, which directly corresponds to the source location, the optimization techniques used are Gauss-Newton, Gradient descent and Newton-Raphson methods. Two different cases are considered for investigation in this paper. The first case compares the three different optimization techniques in terms of convergence rate. In the second case the error values obtained from two different scenarios are compared, one involving a single trial only, while the second scenario uses Monte Carlo method of simulations. Firstly, the error values, for both the coordinates (two-dimensional), obtained by getting the difference between the measured source positions and the initially guessed source position are obtained for a single trial. Later using Monte Carlo simulation method, the Root-Mean-Square (RMS) error values, for both the coordinates (two-dimensional), for the optimization techniques are obtained. To improve the performance of the algorithm, Monte Carlo simulation has been used for multiple trials.

Quantile regression for nonlinear mixed effects models: a likelihood based perspective

Longitudinal data are frequently analyzed using normal mixed effects models. Moreover, the traditional estimation methods are based on mean regression, which leads to non-robust parameter estimation under non-normal error distribution. However, at least in principle, quantile regression (QR) is more robust in the presence of outliers/influential observations and misspecification of the error distributions when compared to the conventional mean regression approach. In this context, this paper develops a likelihood-based approach for estimating QR models with correlated continuous longitudinal data using the asymmetric Laplace distribution. Our approach relies on the stochastic approximation of the EM algorithm (SAEM algorithm), obtaining maximum likelihood estimates of the fixed effects and variance components in the case of nonlinear mixed effects (NLME) models. We evaluate the finite sample performance of the SAEM algorithm and asymptotic properties of the ML estimates through simulation experiments. Moreover, two real life datasets are used to illustrate our proposed method using the $$\texttt {qrNLMM}$$ package from $$\texttt {R}$$.

An improved nonlocal maximum likelihood estimation method for denoising magnetic resonance images with spatially varying noise levels

Abstract Magnetic resonance images (MRI) reconstructed with parallel MRI (pMRI) techniques generally have spatially varying (non-stationary) noise levels. However, most of the existing MRI denoising methods rely on a stationary noise model and end with suboptimal results when applied to pMRI images. To address this problem, this paper proposes an improved nonlocal maximum likelihood (NLML) estimation method. In the proposed method, a noise map is computed with a robust noise estimator before the ML estimation of the underlying signal. Also, a similarity measure based on local frequency descriptors (LFD) is introduced to find the nonlocal samples for ML estimation. The experiments on simulated and real magnetic resonance (MR) data demonstrate that the proposed technique has superior filtering capabilities in terms of subjective and quantitative assessments when compared with other state-of-the-art methods.

Efficient estimation of Pareto model using modified maximum likelihood estimators

In this article, we have proposed some modifications in the maximum likelihood estimation for estimating the parameters of the Pareto distribution and evaluated performance of these modified estimators in comparison to the existing maximum likelihood estimators. Total Relative Deviation (TRD) and Mean Square Error (MSE) have been used as performance indicators for goodness of fit analysis. The modified and traditional estimators have been compared for different sample sizes and different parameter combinations using a Monto Carlo simulation in R-language. We have concluded that the modified maximum likelihood estimator based on expectation of empirical Cumulative Distribution Function (CDF) of first-order statistic performs much better than the traditional ML estimator and other modified estimators based on median and coefficient of variation. The superiority of the said estimator is independent of sample size and choice of true parameter values. The simulation results were further corroborated by employing the proposed estimation strategies on two real-life data sets.

A Further Look at Modified ML Estimation of the Panel AR(1) Model with Fixed Effects and Arbitrary Initial Conditions. (Newer Version)

In this paper we consider two kinds of generalizations of Lancaster's (Review of Economic Studies, 2002) Modified ML estimator (MMLE) for the panel AR(1) model with fixed effects and arbitrary initial conditions and possibly covariates when the time dimension, T, is fixed. When the autoregressive parameter ρ=1, the limiting modified profile log-likelihood function for this model has a stationary point of inflection and ρ is first-order underidentified but second-order identified. We show that the generalized MMLEs exist w.p.a.1 and are uniquely defined w.p.1. and consistent for any value of ρ≥-1. When ρ=1, the rate of convergence of the MMLEs is N^{1/4}, where N is the cross-sectional dimension of the panel. We then develop an asymptotic theory for GMM estimators when one of the parameters is only second-order identified and use this to derive the limiting distributions of the MMLEs. They are generally asymmetric when ρ=1. One kind of generalized MMLE depends on a weight matrix W_{N} and we show that a suitable choice of W_{N} yields an asymptotically unbiased MMLE. We also show that Quasi LM tests that are based on the modified profile log-likelihood and use its expected rather than observed Hessian, with an additional modification for ρ=1, and confidence regions that are based on inverting these tests have correct asymptotic size in a uniform sense when |ρ|≤1. Finally, we investigate the finite sample properties of the MMLEs and the QLM test in a Monte Carlo study.

Sieve maximum likelihood estimation of the spatial autoregressive Tobit model

This paper extends the ML estimation of a spatial autoregressive Tobit model under normal disturbances in Xu and Lee (2015b, Journal of Econometrics) to distribution-free estimation. We examine the sieve MLE of the model, where the disturbances are i.i.d.with an unknown distribution. We show that the spatial autoregressive process with Tobit censoring and related variables are spatial near-epoch dependent (NED). A related contribution is that we develop some exponential inequalities for spatial NED random fields. With these inequalities, we establish the consistency of the estimator. Asymptotic distributions of structural parameters of the model are derived from a functional central limit theorem and projection.Simulations show that the sieve MLE can improve the finite sample performance upon misspecified normal MLEs. As an empirical application, we examine the school district income surtax rates in Iowa. Our results show that the spatial spillover effects are significant, but they may be overestimated if disturbances are restricted to be normally distributed.

Earthquake Magnitude Relationships for the Saint Peter and Saint Paul Archipelago, Equatorial Atlantic

We have investigated several relationships between ML, M(NEIC) and Mw for the earthquakes locally recorded in the Saint Peter and Saint Paul Archipelago (SPSPA), Equatorial Atlantic. Because we only have one station in the area, we could not derive attenuation relations for events recorded at different distances at different stations. Our approach was then to compare our ML estimates with magnitudes reported by NEIC. This approach produced acceptable results particularly for epicentral distance smaller than 100 km. For distances greater that 100 km, there is a systematic increase in the residuals probable due to the lack of station correction and our inability to accurately estimate Q. We also investigate the Mw—M(NEIC) relationship. We find that Mw estimates using S-wave produce smaller residuals when compared with both M(NEIC). Finally, we also investigate the ML—Mw relationship and observe that given the data set we have, the 1:1 holds. We believe that the use of the present methodologies provide consistent magnitude estimates between all the magnitudes investigated that could be used to better assess seismic hazard in the region.