Traditional Maximum Likelihood Estimation Research Articles

Are Cramér-Rao lower bounds an accurate estimate for standard deviations in in vivo magnetic resonance spectroscopy?

Due to inherent time constraints for in vivo experiments, it is infeasible to repeat multiple MRS scans to estimate standard deviations on the desired measured parameters. As such, the Cramér-Rao lower bounds (CRLBs) have become the routine method to approximate standard deviations for in vivo experiments. Cramér-Rao lower bounds, however, as the name suggests, are theoretically a lower bound on the standard deviation and it is not clear if and under what circumstances this approximation is valid. Realistic synthetic 3 T spectra were used to investigate the relationship between estimated CRLBs, true CRLBs and standard deviations. Here we demonstrate that, although the CRLBs are theoretically truly a lower bound on the standard deviation (not an equality) for the problem typically encountered in quantification, they are still an adequate approximation to standard deviation as long as the model perfectly characterizes the data. In the case when the macromolecule basis deviates from the measured macromolecules it was shown that the CRLBs can deviate from standard deviations by approximately 50% for N-acetylaspartic acid, creatine and glutamate and of the order of 100% or more for myo-inositol and γ-aminobutyric acid. In the case when the model perfectly reflects the data the CRLBs are within approximately 10% of standard deviations for all metabolites. The result of the CRLB being within 10% of standard deviations means that, for an accurate model, novel quantification methods such as machine learning or deep learning will not be able to obtain substantially more precise estimates for the desired parameters than traditional maximum-likelihood estimation.

Parameter estimation of the Pareto distribution using least squares approaches blended with different rank methods and its applications in modeling natural catastrophes

The current article evaluates least-squares-based approaches for estimating parameters of the two-parameter Pareto distribution. The algebraic expressions for least squares (LS), relative least squares (RLS) and weighted least squares (WLS) estimators are derived by generating empirical cumulative distribution function (CDF) using mean rank, median rank and symmetrical CDF methods. The performance of the estimation approaches is evaluated through Monte Carlo simulations for different combinations of parameter values and sample sizes. The performance of the regression-based methods is then compared with one another and with the traditional maximum likelihood (ML) estimation method. Our simulation results unveil that among the regression-based methods, RLS has an improved or better performance compared to the other two regression-based approaches for samples of all sizes. Moreover, RLS performs better than the ML method for small samples. Among the rank methods used for generating empirical CDF, it is observed that the mean rank method outperformed other two rank methods. The simulation results are further corroborated by the application of all the methods on two real-life datasets representing damages caused by natural catastrophes.

Estimating the failure rate of the log-logistic distribution by smooth adaptive and bias-correction methods

The Log-logistic distribution has successfully earned attention in practical applications due to its good statistical properties. Because the traditional maximum likelihood estimators of the Log-logistic distribution parameters do not have an explicit form and are biased when the sample size is small. Therefore, the estimation and prediction of the failure rate is not well. In this study, we study the quality of the maximum likelihood, asymptotic maximum likelihood and bias-corrected maximum likelihood methods, and propose a smooth adaptive estimation method for estimating the Log-logistic distribution parameters. To reduce the bias of the asymptotic maximum likelihood and smooth adaptive estimators of the Log-logistic distribution parameters, the bias-corrected method is used to improve the asymptotic maximum likelihood and smooth adaptive estimation methods. Two new bias-corrected estimation methods are also proposed to obtain reliable estimates of the Log-logistic distribution parameters. An intensive Monte Carlo simulation study is conducted to evaluate the performance of these estimation methods. Simulation results show that the smooth adaptive and two new bias-corrected estimation methods are more competitive than other competitors. Finally, two real example is used for illustrating the applications of the smooth adaptive, CAML and CSA estimation methods.

Influence Diagnostic Methods in the Poisson Regression Model with the Liu Estimator.

There is a long history of interest in modeling Poisson regression in different fields of study. The focus of this work is on handling the issues that occur after modeling the count data. For the prediction and analysis of count data, it is valuable to study the factors that influence the performance of the model and the decision based on the analysis of that model. In regression analysis, multicollinearity and influential observations separately and jointly affect the model estimation and inferences. In this article, we focused on multicollinearity and influential observations simultaneously. To evaluate the reliability and quality of regression estimates and to overcome the problems in model fitting, we proposed new diagnostic methods based on Sherman–Morrison Woodbury (SMW) theorem to detect the influential observations using approximate deletion formulas for the Poisson regression model with the Liu estimator. A Monte Carlo method is done for the assessment of the proposed diagnostic methods. Real data are also considered for the evaluation of the proposed methods. Results show the superiority of the proposed diagnostic methods in detecting unusual observations in the presence of multicollinearity compared to the traditional maximum likelihood estimation method.

On the estimation of Bell regression model using ridge estimator

The bell regression is used, when the response variable is in the form of counts with over dispersion. The bell regression coefficients are generally estimated using the maximum likelihood estimator (MLE). It is known that the performance of the traditional MLE is very sensitive to multicollinearity. Therefore, we propose a Bell ridge regression (BRR) as a solution to the multicollinearity problems. For the assessment of BRR, we conduct a Monte Carlo simulation study to monitor the performance of the proposed estimator where the mean squared error (MSE) is considered as an evaluation criterion. Also, two real examples are included to show the superiority of the BRR estimator.

Multi-Frame Star Image Denoising Algorithm Based on Deep Reinforcement Learning and Mixed Poisson-Gaussian Likelihood.

Mixed Poisson–Gaussian noise exists in the star images and is difficult to be effectively suppressed via maximum likelihood estimation (MLE) method due to its complicated likelihood function. In this article, the MLE method is incorporated with a state-of-the-art machine learning algorithm in order to achieve accurate restoration results. By applying the mixed Poisson–Gaussian likelihood function as the reward function of a reinforcement learning algorithm, an agent is able to form the restored image that achieves the maximum value of the complex likelihood function through the Markov Decision Process (MDP). In order to provide the appropriate parameter settings of the denoising model, the key hyperparameters of the model and their influences on denoising results are tested through simulated experiments. The model is then compared with two existing star image denoising methods so as to verify its performance. The experiment results indicate that this algorithm based on reinforcement learning is able to suppress the mixed Poisson–Gaussian noise in the star image more accurately than the traditional MLE method, as well as the method based on the deep convolutional neural network (DCNN).

Assessment of Hazard Ratios in Oncology Clinical Trials Terminated Early for Superiority

Group sequential designs allow potential early trial termination at the interim analysis, before study completion. Traditional maximum likelihood estimate is commonly used to quantify the treatment effect in group sequential design trials; however, in published clinical trials, a bias-adjusted estimator has rarely been reported. To emphasize the need for considering overestimation of treatment effect by applying 2 bias-adjusted estimators to previously published, early-terminated oncology clinical trials. Trials published from 2013 to 2017 were identified by searching MEDLINE and Embase on February 23, 2018. This review was restricted to oncology clinical trials using group sequential designs with a single preplanned interim analysis as well as 2-arm randomized clinical trials that were subsequently stopped for efficacy reasons. Each article was independently reviewed by 3 biostatisticians during text screening, and differences in opinion were resolved by discussion. This report presents the unadjusted hazard ratio (HR) of an experimental arm to a reference arm and 2 bias-adjusted HRs calculated by using the conditional mean-adjusted estimator (CMAE) and weighted CMAE (WCMAE). In total, 198 abstracts were screened for eligibility, of which, 19 eligible clinical trials were identified as applicable to the bias-adjusted estimators. Unadjusted HRs ranged from 0.203 (95% CI, 0.150-0.276) to 0.71 (95% CI, 0.60-0.84), number of events at the interim analysis from 58 to 540, and information time from 48% to 82%. In each study, the HRs adjusted by CMAE and WCMAE were higher than the unadjusted HR. Bias-adjusted estimates in large trials (243 and 414 events at the interim analysis) were similar to the unadjusted HR. However, in small trials (eg, with 58 events at the interim analysis), bias-adjusted estimates were highly disparate from the unadjusted HR. In trials with large treatment effects (eg, HRs of 0.20 and 0.22), the difference between unadjusted and bias-adjusted HRs was small even though the number of events at the interim analysis was small; larger differences were observed when the unadjusted HR was greater than 0.5. In this systematic review of oncology clinical trials that were stopped for efficacy at the interim analysis, relatively large differences were noted between the unadjusted and adjusted HRs when the number of events at the interim analysis was small or when the unadjusted HR was close to the boundaries. These findings suggest presenting the 2 bias-adjusted HRs along with the unadjusted HR in the data monitoring committee meeting.

Pivotal Inference for the Inverted Exponentiated Rayleigh Distribution Based on Progressive Type-II Censored Data

In this article, the pivotal inference is proposed to estimate the two unknown parameters of the inverse exponentiated Rayleigh distribution based on progressive censored data. We derive the point estimator and construct an interval estimator using the pivotal quantity method. To compare the performance of this proposed method and the traditional maximum likelihood estimation method, a simulation study is conducted. The simulation results show that the proposed method performs better in terms of bias and mean squared error. Finally, a real dataset is used to illustrate the proposed approaches.

A Novel Dynamic Baysian Canonical Correlation Analysis Method for Fault Detection

In the field of Multivariate Statistical Process Monitoring (MSPM), process dynamics has always been the focus. Besides, considering the uncertainty in chemical processes, latent variable models are extended to the probabilistic framework, in which maximum likelihood estimation with expectation maximization (EM) algorithm is adopted for parameter learning. However, the modelling performance is restricted owing to the reason that these models either neglect the static characteristics reflecting process structure or suffer from over fitting and local optimum. To tackle these issues, a dynamic Baysian canonical correlation analysis (DBCCA) model is developed through combining the consideration of process dynamics with the variational CCA and utilized for fault detection. More specifically, both static structural characteristics and process dynamics can be simultaneously captured in DBCCA model. In essence, the variational Bayesian approach renders effects of regularization, alleviating the dilemma in traditional maximum likelihood estimation methods by nature. The effectiveness of proposed method is testified on the well-known Tennessee Eastman (TE) benchmark, where improvements are attained.

New Parsimonious Multivariate Spatial Model: Spatial Envelope

Dimension reduction provides a useful tool for analyzing high dimensional data. The recently developed \textit{Envelope} method is a parsimonious version of the classical multivariate regression model through identifying a minimal reducing subspace of the responses. However, existing envelope methods assume an independent error structure in the model. While the assumption of independence is convenient, it does not address the additional complications associated with spatial or temporal correlations in the data. In this article, we introduce a \textit{Spatial Envelope} method for dimension reduction in the presence of dependencies across space. We study the asymptotic properties of the proposed estimators and show that the asymptotic variance of the estimated regression coefficients under the spatial envelope model is smaller than that from the traditional maximum likelihood estimation. Furthermore, we present a computationally efficient approach for inference. The efficacy of the new approach is investigated through simulation studies and an analysis of an Air Quality Standard (AQS) dataset from the Environmental Protection Agency (EPA).

Using degradation-with-jump measures to estimate life characteristics of lithium-ion battery

Degradation-with-jump measures are time series data sets containing the information of both continuous and randomly jumping degradation evolution of a system. Traditional maximum likelihood estimation and Bayesian estimation are not convenient for such general jump processes without closed-form distributions. Based on general degradation models derived using Lévy driven non-Gaussian Ornstein-Uhlenbeck (OU) processes, we propose a systematic statistical method using linear programing estimators and empirical characteristic functions. The point estimates of reliability function and lifetime moments are obtained by deriving their explicit expressions. We also construct bootstrap procedures for the confidence intervals. Simulation studies for a stable process and a stable driven OU process are performed. In the case study, we use a general Lévy process to fit the Li-ion battery life data, and then estimate the reliability and lifetime moments of the battery. By integrally analyzing degradation data series embedded with jump measures, our work provides the efficient and precise estimation for life characteristics.

Statistical inference for the exponentiated half logistic distribution based on censored data

In this paper, we consider the estimates for the scale parameter of an exponentiated half logistic distribution under progressively Type-II censoring and general progressively Type-II censoring, respectively. Because of the complex form of the traditional maximum likelihood estimator, it is hard to get an explicit solution. Therefore, we propose three new estimation methods based on censored data to obtain approximate estimates of the scale parameter. The first two are approximate maximum likelihood estimation methods based on Taylor expansion and the third is the approximate estimation method based on pivotal quantities we establish. Furthermore, we consider asymptotic confidence intervals based on pivotal quantities. Finally, simulation studies are conducted to compare the performance of the proposed estimators. A real dataset analysis is carried out to illustrate the proposed methods.

Dynamic panel of count data with initial event and correlated heterogeneity

ABSTRACTExisting literature on the specification of a dynamic panel model for counts raises several potential challenges. These include (a) the issue of a potentially explosive model when the lagged-dependent variable appears in the conventional exponential conditional mean function and (b) appropriate handling of the problem of the initial conditions that drive a dynamic process. This study addresses both issues within the context of a panel count model with Mundlak–Chamberlain type conditionally correlated heterogeneity. This correlated random-effects model is a useful compromise between the standard fixed- and random-effects models; it is then combined with two alternative specifications of the conditional mean function; one allows exponential feedback (EFB), whereas the other allows linear feedback (LFB). Monte Carlo experiments are conducted to check the robustness of these specifications by using the traditional maximum likelihood estimator for the EFB model and a nonlinear least squares estimator for the LFB model.

Modelling of left-truncated heavy-tailed data with application to catastrophe bond pricing

In this article, we concentrate on modelling heavy-tailed data which can be subjected to left-truncation. We modify an existing procedure for modelling left-truncated data via a compound non-homogeneous Poisson process to make it systematically applicable in the context heavy-tailed data. The introduced procedure can be applied when the underlying severities of the process follow Burr type XII, Generalised Pareto and Generalised Extreme Value distributions by using the Maximum Product of Spacings (MPS) parameter estimation technique. As a natural consequence of the MPS technique, we consider how Moran’s log spacings statistic for testing goodness-of-fit of the severity distributions can be adapted to suit left-truncated data. Thereafter, we compare the performance of this new fitting procedure against traditional maximum likelihood estimation in the context of natural catastrophe loss data, and evidence in favour of MPS is found. Within the context of these data, we also compare our procedure to a one that does not account for left-truncation. We end our contribution by proposing, for our modelling procedure, a Monte Carlo importance sampling algorithm which ensures that large losses are satisfactorily simulated. In closing, we illustrate the potential usage of both the new fitting and simulation procedures by presenting catastrophe bond prices with a trigger based on the analysed heavy-tailed data.

A Comparison of Estimation Techniques for IRT Models With Small Samples

ABSTRACT The usefulness of item response theory (IRT) models depends, in large part, on the accuracy of item and person parameter estimates. For the standard 3 parameter logistic model, for example, these parameters include the item parameters of difficulty, discrimination, and pseudo-chance, as well as the person ability parameter. Several factors impact traditional marginal maximum likelihood (ML) estimation of IRT model parameters, including sample size, with smaller samples generally being associated with lower parameter estimation accuracy, and inflated standard errors for the estimates. Given this deleterious impact of small samples on IRT model performance, use of these techniques with low-incidence populations, where it might prove to be particularly useful, estimation becomes difficult, especially with more complex models. Recently, a Pairwise estimation method for Rasch model parameters has been suggested for use with missing data, and may also hold promise for parameter estimation with small samples. This simulation study compared item difficulty parameter estimation accuracy of ML with the Pairwise approach to ascertain the benefits of this latter method. The results support the use of the Pairwise method with small samples, particularly for obtaining item location estimates.

A consistent estimator for logistic mixed effect models.

We propose a consistent and locally efficient estimator to estimate the model parameters for a logistic mixed effect model with random slopes. Our approach relaxes two typical assumptions: the random effects being normally distributed, and the covariates and random effects being independent of each other. Adhering to these assumptions is particularly difficult in health studies where in many cases we have limited resources to design experiments and gather data in long-term studies, while new findings from other fields might emerge, suggesting the violation of such assumptions. So it is crucial if we could have an estimator robust to such violations and then we could make better use of current data harvested using various valuable resources. Our method generalizes the framework presented in Garcia & Ma (2016) which also deals with a logistic mixed effect model but only considers a random intercept. A simulation study reveals that our proposed estimator remains consistent even when the independence and normality assumptions are violated. This contrasts from the traditional maximum likelihood estimator which is likely to be inconsistent when there is dependence between the covariates and random effects. Application of this work to a Huntington disease study reveals that disease diagnosis can be further improved using assessments of cognitive performance.

Regularized Gaussian Mixture Model for High-Dimensional Clustering.

Finding low-dimensional representation of high-dimensional data sets is an important task in various applications. The fact that data sets often contain clusters embedded in different subspaces poses barrier to this task. Driven by the need in methods that enable clustering and finding each cluster's intrinsic subspace simultaneously, in this paper, we propose a regularized Gaussian mixture model (GMM) for clustering. Despite the advantages of GMM, such as its probabilistic interpretation and robustness against observation noise, traditional maximum-likelihood estimation for GMMs shows disappointing performance in high-dimensional setting. The proposed regularization method finds low-dimensional representations of the component covariance matrices, resulting in better estimation of local feature correlations. The regularization problem can be incorporated in the expectation maximization algorithm for maximizing the likelihood function of a GMM, with the M -step modified to incorporate the regularization. The M -step involves a determinant maximization problem, which can be solved efficiently. The performance of the proposed method is demonstrated using several simulated data sets. We also illustrate the potential value of the proposed method in applications using four real data sets.

Statistical Inference for Type-I Generalized Birnbaum–Saunders Distribution

A new generalized Birnbaum–Saunders distribution (Type-I GBS) was presented in Owen (IEEE Trans Reliab 55:475–479, 2006) to model a lifetime of a product under cyclic stresses by using a long memory process on the crack extensions. This highly flexible model includes the original BS distribution as a special case and can be widely applied in fatigue studies. In this article, we present the relevant properties, parameter estimation, and hypothesis testing for the distribution. We explore the traditional maximum likelihood estimation approach, and propose a new inference method for the GBS-I distribution. An extensive simulation study is carried out to assess performance of the methods, and a real data is analyzed where it is shown that the GBS-I model with the proposed method provides an efficient estimation and achieves a better fit than the classic likelihood-based procedure.

Regularized continuous-time Markov Model via elastic net.

Continuous-time Markov models are commonly used to analyze longitudinal transitions between multiple disease states in panel data, where participants' disease states are only observed at multiple time points, and the exact state paths between observations are unknown. However, when covariate effects are incorporated and allowed to vary for different transitions, the number of potential parameters to estimate can become large even when the number of covariates is moderate, and traditional maximum likelihood estimation and subset model selection procedures can easily become unstable due to overfitting. We propose a novel regularized continuous-time Markov model with the elastic net penalty, which is capable of simultaneous variable selection and estimation for large number of parameters. We derive an efficient coordinate descent algorithm to solve the penalized optimization problem, which is fully automatic and data driven. We further consider an extension where one of the states is death, and time of death is exactly known but the state path leading to death is unknown. The proposed method is extensively evaluated in a simulation study, and demonstrated in an application to real-world data on airflow limitation state transitions.

A Novel Multi-Phase Stochastic Model for Lithium-Ion Batteries’ Degradation with Regeneration Phenomena

A lithium-Ion battery is a typical degradation product, and its performance will deteriorate over time. In its degradation process, regeneration phenomena have been frequently encountered, which affect both the degradation state and rate. In this paper, we focus on how to build the degradation model and estimate the lifetime. Toward this end, we first propose a multi-phase stochastic degradation model with random jumps based on the Wiener process, where the multi-phase model and random jumps at the changing point are used to describe the variation of degradation rate and state caused by regeneration phenomena accordingly. Owing to the complex structure and random variables, the traditional Maximum Likelihood Estimation (MLE) is not suitable for the proposed model. In this case, we treat these random variables as latent parameters, and then develop an approach for model identification based on expectation conditional maximum (ECM) algorithm. Moreover, depending on the proposed model, how to estimate the lifetime with fixed changing point is presented via the time-space transformation technique, and the approximate analytical solution is derived. Finally, a numerical simulation and a practical case are provided for illustration.