ML Estimation Research Articles

Heavy or semi-heavy tail, that is the question

While there has been considerable research on the analysis of extreme values and outliers by using heavy-tailed distributions, little is known about the semi-heavy-tailed behaviors of data when there are a few suspicious outliers. To address the situation where data are skewed possessing semi-heavy tails, we introduce two new skewed distribution families of the hyperbolic secant with exciting properties. We extend the semi-heavy-tailedness property of data to a linear regression model. In particular, we investigate the asymptotic properties of the ML estimators of the regression parameters when the error term has a semi-heavy-tailed distribution. We conduct simulation studies comparing the ML estimators of the regression parameters under various assumptions for the distribution of the error term. We also provide three real examples to show the priority of the semi-heavy-tailedness of the error term comparing to heavy-tailedness. Online supplementary materials for this article are available. All the new proposed models in this work are implemented by the shs R package, which can be found on the GitHub webpage.

A simulation study comparing model fit measures of structural equation modeling with multivariate contaminated normal distribution

SEM is a very popular analytical method for quantitative studies in many disciplines. The goal of SEM is to determine the model that best fits the data. The data sets were generated from the multivariate contaminated normal distribution in this study. The aim of the study was to determine the best goodness of fit measures not affected by data structures and sample sizes. ADF, ML, and GLS estimation methods were applied under the multivariate nonnormality. According to the ADF method, AGFI, GFI, and RMSEA were not affected by sample size and correlation changes. CFI, NFI, and NNFI values were affected by both sample size and correlation changes. RMR always received values outside the acceptable fit limits in all sample sizes and correlation values. Consequently; if the data are multivariate nonnormality, the sample size should be greater than 250 units and AGFI, GFI and RMSEA fit measures should be used.

Exponentiated Exponential Weibull Distribution: Mathematical Properties and Application

An Exponentiated Exponential Weibull Distribution (EEWD) is proposed. Some statistical properties of the proposed distribution are studied. The Maximum Likelihood method (ML) is applied to estimate the model parameters. The efficiency of the ML estimates of the proposed model is demonstrated using a Monte Carlo simulation study with different samples sizes. Moreover, real data set application is provided to show the flexibility of the proposed model in comparison to some selected distributions.

Single-Pulse Simultaneous Target Detection and Angle Estimation in a Multichannel Phased Array Radar

This paper is focused on simultaneous target detection and angle estimation with a multichannel phased array radar. Resorting to a linearized expression for the array steering vector around the beam pointing direction, the problem is formulated as a composite binary hypothesis test where the unknowns, under the alternative hypothesis, include the target directional cosines displacements with respect to the array nominal coarse pointing direction. The problem is handled via the Generalized Likelihood Ratio (GLR) criterion (both one-step and two-step) where decision statistics leveraging the Maximum Likelihood Estimates (MLEs) of the parameters are compared with a detection threshold. If crossed, target presence is declared and the MLEs of the aforementioned displacements directly provide target angular position with respect to the pointing direction. From the analytic point of view, ML estimation involves a constrained fractional quadratic optimization problem whose optimal solution can be found via the Dinkelbach's algorithm or approximated through a fast-converging procedure based on a Coordinate Descent (CD) optimization. The performance analysis of the proposed architectures as well as the corresponding discussion is developed in terms of computational complexity, Constant False Alarm Rate (CFAR) behavior, detection performance, and angular estimation accuracy, also in comparison with some counterparts available in the open literature and theoretical benchmark limits.

Spatial inequality in human development in India- A case study of Karnataka

The paper contributes to the literature by investigating the spatial unevenness in human development and enquiries, whether the patterns of human development are conditioned by spatially dependent factors, by viewing space as one of the factor influencing human development. It compares human development indices at district levels and investigates the intensity of spatial clusters. Over the last two decades, there is a considerable improvement in the level of human development in the state, also faces staggering inequalities. As evidenced by the Moran's I value the overall intra-regional inequality increased for the period 1990–2012, however post 2000 the Moran's I increased significantly, indicating strong spatial correlation. The result of the ML estimation of spatial lag model elucidate that the spatial lag coefficient (ρ) of HDI is positive and highly significant reflecting strong spatial influence, the HDI value of a region is robustly influenced by the neighbouring observations. The results of the study helps to reduce spatial inequalities at micro level [sub-state level], curbing inequalities in all its form and dimensions, which is essential to meet the 2030 sustainable development goals.

ML Estimation and CRBs for Reverberation, Speech, and Noise PSDs in Rank-Deficient Noise Field

Speech communication systems are prone to performance degradation in reverberant and noisy acoustic environments. Dereverberation and noise reduction algorithms typically require several model parameters, e.g. the speech, reverberation and noise power spectral densities (PSDs). A commonly used assumption is that the noise PSD matrix is known. However, in practical acoustic scenarios, the noise PSD matrix is unknown and should be estimated along with the speech and reverberation PSDs. In this article, we consider the case of rank-deficient noise PSD matrix, which arises when the noise signal consists of multiple directional noise sources, whose number is less than the number of microphones. We derive two closed-form maximum likelihood estimators (MLEs). The first is a non-blocking-based estimator which jointly estimates the speech, reverberation and noise PSDs, and the second is a blocking-based estimator, which first blocks the speech signal and then jointly estimates the reverberation and noise PSDs. Both estimators are analytically compared and analyzed, and mean square errors (MSEs) expressions are derived. Furthermore, Cramer-Rao Bounds (CRBs) on the estimated PSDs are derived. The proposed estimators are examined using both simulation and real reverberant and noisy signals, demonstrating the advantage of the proposed method compared to competing estimators.

Estimating the Term Structure with Linear Regressions: Getting to the Roots of the Problem

Abstract Linear estimators of the affine term structure model are inconsistent since they cannot reproduce the factors used in estimation. This is a serious handicap empirically, giving a worse fit than the conventional ML estimator that ensures consistency. We show that a simple self-consistent estimator can be constructed using the eigenvalue decomposition of a regression estimator. The remaining parameters of the model follow analytically. Estimates from this model are virtually indistinguishable from that of the ML estimator. We apply the method to estimate various models of U.S. Treasury yields. These exercises greatly extend the range of models that can be estimated.

Fast effect size shrinkage software for beta-binomial models of allelic imbalance

Allelic imbalance occurs when the two alleles of a gene are differentially expressed within a diploid organism, and can indicate important differences in cis-regulation and epigenetic state across the two chromosomes. Because of this, the ability to accurately quantify the proportion at which each allele of a gene is expressed is of great interest to researchers. This becomes challenging in the presence of small read counts and/or sample sizes, which can cause estimates for allelic expression proportions to have high variance. Investigators have traditionally dealt with this problem by filtering out genes with small counts and samples. However, this may inadvertently remove important genes that have truly large allelic imbalances. Another option is to use Bayesian estimators to reduce the variance. To this end, we evaluated the accuracy of three different estimators, the latter two of which are Bayesian shrinkage estimators: maximum likelihood, approximate posterior estimation of GLM coefficients (apeglm) and adaptive shrinkage (ash). We also wrote C++ code to quickly calculate ML and apeglm estimates, and integrated it into theapeglmpackage. The three methods were evaluated on both simulated and real data. Apeglm consistently performed better than ML according to a variety of criteria, including mean absolute error and concordance at the top. While ash had lower error and greater concordance than ML on the simulations, it also had a tendency to over-shrink large effects, and performed worse on the real data according to error and concordance. Furthermore, when compared to five other packages that also fit beta-binomial models, theapeglmpackage was substantially faster, making our package useful for quick and reliable analyses of allelic imbalance.Apeglmis available as an R/Bioconductor package at http://bioconductor.org/packages/apeglm.

A comparison of two approaches to correction of restriction of range in correlation analysis

A common problem in predictive validity studies in the educational and psychological fields, e.g. in educational and employment selection, is restriction in range of the predictor variables. There are several methods for correcting correlations for restriction of range. The aim of this paper was to examine the usefulness of two approaches to correcting for range restriction; Thorndike’s case 2 correction and ML estimates obtained from the EM algorithm, by comparing the corrected correlations with the correlation from an unrestricted sample. The unrestricted sample consisted of examinees who took the practical Swedish driving-license test regardless of their result on the theory test. Examinees that passed the theory test and took the practical test were regarded as a restricted sample. The result provided empirical support for the appropriateness of Thorndike’s case 2 correction method. Although using the EM algorithm yielded a good estimate of the correlation in the unrestricted sample, further studies are needed on this topic.

Scoring-Based ML Estimation and CRBs for Reverberation, Speech, and Noise PSDs in a Spatially Homogeneous Noise Field

Hands-free speech systems are subject to performance degradation due to reverberation and noise. Common methods for enhancing reverberant and noisy speech require the knowledge of the speech, reverberation and noise power spectral densities (PSDs). Most literature on this topic assumes that the noise power spectral density (PSD) matrix is known. However, in many practical acoustic scenarios, the noise PSD is unknown and should be estimated along with the speech and the reverberation PSDs. In this article, the noise is modeled as a spatially homogeneous sound field, with an unknown time-varying PSD multiplied by a known time-invariant spatial coherence matrix. We derive two maximum likelihood estimators (MLEs) for the various PSDs, including the noise: The first is a non-blocking-based estimator, that jointly estimates the PSDs of the speech, reverberation and noise components. The second MLE is a blocking-based estimator, that blocks the speech signal and estimates the reverberation and noise PSDs. Since a closed-form solution does not exist, both estimators iteratively maximize the likelihood using the Fisher scoring method. In order to compare both methods, the corresponding Cramer-Rao Bounds (CRBs) are derived. For both the reverberation and the noise PSDs, it is shown that the non-blocking-based CRB is lower than the blocking-based CRB. Performance evaluation using both simulated and real reverberant and noisy signals, shows that the proposed estimators outperform competing estimators, and greatly reduce the effect of reverberation and noise.

Psychometric evaluation of the Working Alliance Inventory—Therapist version: Current and new short forms

Background: The Working Alliance Inventory (WAI) and its short forms are widely used, although the properties of the therapists’ versions have been little studied. Method: We examined the psychometric properties of two short forms (WAI-S-T, WAI-SR-T), and explored the creation of a psychometrically stronger short form using contemporary measure development techniques. Well-fitting items from the full 36-item WAI were identified in a development sample (131 therapists, 688 patients) using multi-level Bayesian Structural Equation Modeling, accounting for therapist rated effects. Multi-level Item Response Theory (IRT) methods aided creation of a revised short form (WAI-S-T-IRT). Factor structures of the three forms were assessed using multi-level ML estimation with robust standard errors. Results: Collinearity problems for the Goal and Task dimensions led to testing a two-factor model (Goal–Task, Bond). All three measures showed satisfactory fit; the WAI-S-T-IRT fit slightly better but differences were minor. Testing the structures in an independent sample (N = 1117) yielded essentially the same results. No version showed strong measurement invariance. Discussion: Continued use of current therapist forms is supported; differentiation of theoretical dimensions is difficult with current measures, and may not be possible with self-report forms.

The restricted maximum likelihood method for variance estimation in a mixed model with additive penalized-spline

Estimation of variance components in the model often needed in the problem of model-based prediction. We present the concept of restricted maximum likelihood estimation for variance components in the mixed model with additive p-spline. We provide simulations as a result of applying the REML estimator concept to variance components in diverse models with p-spline additives. The model is then used to predict the means of sub-area of the population. The limited simulation shows the performance of the model using the REML estimator for the variance component to provide better predictive results, indicated by the average RMSPE value and smaller absolute average values and the denser distribution of the MSPE with a lower median, than that produced by the model the same as the ML estimator for component variance.

Study on Effect of Left-Truncated Data Rate on ML Estimation for Power Law Process

Study on Effect of Left-Truncated Data Rate on ML Estimation for Power Law Process

Identification of outlying and influential data with principal components regression estimation in binary logistic regression

In this study, we settle on the issue that when multicollinearity and unusual observations arise simultaneously and we straightforwardly extend leverages, Pearson residuals, delta beta and delta chi-square statistics using the principal components logistic regression (PCLR) estimator where the extensions typically take the advantage of the computation of PCLR estimator by one-step approximation. We then applied two simulation studies and a numerical example to illustrate the behavior of statistics for the PCLR estimator versus the traditional ML estimator.

A Khmaladze-transformed test of fit with ML estimation in the presence of recurrent events

This article provides a goodness-of-fit test for the distribution function or the survival function in a recurrent event setting, when the inter-event time parametric structure is estimated from the observed data. Of concern is the null hypothesis that the inter-event time distribution is absolutely continuous and belongs to a parametric family , where the q-dimensional parameter space is neither known nor specified. We proposed a Khmaladze martingale-transformed type of test (Khmaladze, 1981), adapted to recurrent events. The test statistic combines two likelihood sources of estimation to form a parametric empirical process: (1) a product-limit nonparametric maximum likelihood estimator (NPMLE; Peña et al., 2001a) that is a consistent estimator of F, say, and (2) a point process likelihood estimator (Jacod, 1974/1975). These estimators are combined to construct a Kolmogorov-Smirnov (KS) type of test (Kolmogorov 1933; Smirnov, 1933). Empirical process and martingale weak convergence frameworks are utilized for theoretical derivations and motivational justification of the proposed transformation. A simulation study is conducted for performance assessment, and the test is applied to a problem investigated by Proschan (1963) on air-conditioning failure in a fleet of Boeing 720 jets.

Comparison of Direct L-moments, L-moments and ML Estimation Methods for Weibull Distribution with Type-I Censoring

Comparison of Direct L-moments, L-moments and ML Estimation Methods for Weibull Distribution with Type-I Censoring

Generalized inverted Kumaraswamy generated family of distributions: theory and applications

This paper proposes a new generator function based on the inverted Kumaraswamy distribution and introduces ‘generalized inverted Kumaraswamy-G’ family of distributions. We provide a comprehensive account of some of its mathematical properties that include the ordinary and incomplete moments, quantile and generating functions and order statistics. The infinite mixture representations for probability density and cumulative distribution and entropy functions of the new family are also established. The density function of the ith-order statistics is expressed as an infinite linear combination of baseline densities and model parameters are estimated by maximum likelihood method. Four special models of this family are also derived along with their respective hazard rate functions. The maximum likelihood estimation (MLE) method is used to obtain the model parameters. Monte Carlo simulation experiments are executed to assess the performance of the ML estimators under the corresponding generated models while some data applications are also illustrated. The results of the study show that the proposed distribution is more flexible as compared to the baseline model. This distribution especially can be used to model symmetric, left-skewed, right-skewed and reversed-J data sets.

Simultaneous estimation of parameter uncertainties and disturbance trajectory for robotic manipulator

In this work, a systematic approach is proposed to estimate the disturbance trajectory using a new generalized Lyapunov matrix valued function of the joint angle variables and the robot’s physical parameters using the maximum likelihood estimate (MLE). It is also proved that the estimated disturbance error remains bounded over the infinite time interval. Here, the manipulator is excited with a periodic torque and by the position and velocity data collected at discrete time points construct an ML estimator of the parameters at time \( t + dt \). This process is carried over hand in hand in a recursive manner, thus resulting in a novel unified disturbance rejection and parameter estimation in a general frame work. These parameter estimates are then analyzed for mean and covariance and compared with the Cramer Rao Lower Bound (CRLB) for the parametric statistical model. Using the Lyapunov method, convergence of the “disturbance estimation error” to zero is established. We assume that a Lyapunov matrix dependent on the link angle and form the energy corresponding to this matrix as a quadratic function of the disturbance estimate error. Using the dynamics of the disturbance observer, the rate of change of the Lyapunov energy is evaluated as a quadratic form in the disturbance error. This quadratic form is negative definite for the angular velocity in a certain range and for a certain structured form of the Lyapunov energy matrix. The most general form of the Lyapunov matrix is obtained that guarantees negative rate of increase of the energy and a better bound on the disturbance estimation error convergence rate to zero. This is possible only because we have used the most general form of the Lyapunov energy matrix.

Pseudo-maximum likelihood estimators in linear regression models with fractional time series

Fractal time series and linear regression models are known to play an important role in many scientific disciplines and applied fields. Although there have been enormous development after their appearance, nobody investigates them together. The paper studies a linear regression model (or trending fractional time series model) $$\begin{aligned} y_t=x_t^T\beta +\varepsilon _t,t=1,2,\ldots ,n, \end{aligned}$$ where $$\begin{aligned} \varepsilon _t=\Delta ^{-\delta }g(L;\varphi )\eta _t \end{aligned}$$ with parameters \(0\le \delta \le 1,\varphi ,\beta ,\sigma ^2\) and \(\eta _t\) i.i.d. with zero mean and variance \(\sigma ^2\). Firstly, the pseudo-maximum likelihood (ML) estimators of \(\varphi ,\beta ,\sigma ^2\) are given. Secondly, under general conditions, the asymptotic properties of the ML estimators are investigated. Lastly, the validity of method is illuminated by a real example.

Measuring public inflation perceptions and expectations in the UK

The Bank of England Inflation Attitudes Survey asks individuals about their inflation perceptions and expectations in eight intervals including an indifference limen. This paper studies fitting a mixture normal distribution to such interval data, allowing for multiple modes. Bayesian analysis is useful since ML estimation may fail. A hierarchical prior helps to obtain a weakly informative prior. The No-U-Turn Sampler speeds up posterior simulation. Permutation invariant parameters are free from the label switching problem. The paper estimates the distributions of public inflation perceptions and expectations in the UK during 2001Q1–2017Q4. The estimated means are useful for measuring information rigidity.