Variance Of Maximum Likelihood Estimator Research Articles

Second term improvement to generalised linear mixed model asymptotics

Summary A recent article on generalised linear mixed model asymptotics, Jiang et al. (2022), derived the rates of convergence for the asymptotic variances of maximum likelihood estimators. If m denotes the number of groups and n is the average within-group sample size then the asymptotic variances have orders m − 1 and (mn)−1, depending on the parameter. We extend this theory to provide explicit forms of the (mn)−1 second terms of the asymptotically harder-to-estimate parameters. Improved accuracy of statistical inference and planning are consequences of our theory.

Modified jackknife ridge estimator for the Conway-Maxwell-Poisson model

Recently, research papers have shown a strong interest in modeling count data. The over-dispersion or under-dispersion are frequently seen in the count data. The count data modeling with a broad range of dispersion has been successfully accomplished using the Conway-Maxwell-Poisson regression (COMP) model. The multicollinearity issue is known to have a detrimental impact on the maximum likelihood estimator's variance. To solve this issue, biased estimators as a ridge estimator have repeatedly shown to be an appealing way to minimize the effects of multicollinearity. In this paper, we suggested the jackknife ridge estimator (JCOMPRE) and the modified version of the jackknife ridge estimator (MJCOMPRE) for the COMP model. The proposed estimators (JCOMPRE and MJCOMPRE) are reducing the effects of the multicollinearity and the biasedness of the ridge estimator at the same time. According to the findings of the Monte Carlo simulation study and real-life applications, the proposed estimators have a minimum bias and a minimum mean squared error. This means that the proposed estimators are more efficient than the maximal likelihood estimator and the ridge estimator.

A new Jackknifing ridge estimator for logistic regression model

The ridge regression model has been consistently demonstrated to be an attractive shrinkage method to reduce the effects of multicollinearity. The logistic regression model is a well-known model in application when the response variable is binary data. However, it is known that multicollinearity negatively affects the variance of maximum likelihood estimator of the logistic regression coefficients. To address this problem, a logistic ridge estimator has been proposed by numerous researchers. In this paper, a new Jackknifing logistic ridge estimator (NLRE) is proposed and derived. The idea behind the NLRE is to get diagonal matrix with small values of diagonal elements that leading to decrease the shrinkage parameter and, therefore, the resultant estimator can be better with small amount of bias. Our Monte Carlo simulation results suggest that the NLRE estimator can bring significant improvement relative to other existing estimators. In addition, the real application results demonstrate that the NLRE estimator outperforms both logistic ridge estimator and maximum likelihood estimator in terms of predictive performance.

Entropy-Based Non-Local Means Filter for Single-Look SAR Speckle Reduction

Speckle is an interference phenomenon that contaminates images captured by coherent illumination systems. Due to its multiplicative and non-Gaussian nature, it is challenging to eliminate. The non-local means approach to noise reduction has proven flexible and provided good results. We propose in this work a new non-local means filter for single-look speckled data using the Shannon and Rényi entropies under the G0 model. We obtain the necessary mathematical apparatus (the Fisher information matrix and asymptotic variance of maximum likelihood estimators). The similarity between samples of the patches relies on a parametric statistical test that verifies the evidence whether two samples have the same entropy or not. Then, we build the convolution mask by transforming the p-value into weights with a smooth activation function. The results are encouraging, as the filtered images have a better signal-to-noise ratio, they preserve the mean, and the edges are not severely blurred. The proposed algorithm is compared with three successful filters: SRAD (Speckle Reducing Anisotropic Diffusion), Lee, and FANS (Fast Adaptive Nonlocal SAR Despeckling), showing the new method’s competitiveness.

Developing a Liu‐type estimator in beta regression model

AbstractThe beta regression model is a commonly used when the response variable has the form of fractions or percentages. The maximum likelihood (ML) estimator is used to estimate the regression coefficients of this model. However, it is known that multicollinearity problem affects badly the variance of ML estimator. Therefore, this paper introduces the Liu‐type estimator for the beta regression model to handle the multicollinearity problem. The performance of the proposed (Liu‐type) estimator is compared to the ML estimator and other biased (ridge and Liu) estimators depending on the mean squared error (MSE) criterion by conducting a simulation study and through an empirical application. The results indicated that the proposed estimator outperformed the ML, ridge, and Liu estimators.

Almost unbiased ridge estimator in the count data regression models

The ridge estimator has been consistently demonstrated to be an attractive shrinkage method to reduce the effects of multicollinearity. The Poisson regression negative binomial regression models are well-known model in application when the response variable is count data. However, it is known that multicollinearity negatively affects the variance of maximum likelihood estimator of the count regression coefficients. To address this problem, a count data ridge estimator has been proposed by numerous researchers. In this paper, an almost unbiased regression estimator is proposed and derived. Our Monte Carlo simulation results suggest that the proposed estimator can bring significant improvement relative to other existing estimators. In addition, the real application results demonstrate that the proposed estimator outperforms both negative binomial ridge regression and maximum likelihood estimators in terms of predictive performance.

Generalized ridge estimator in negative binomial regression model

The ridge estimator has been consistently demonstrated to be an attractive shrinkage method to reduce the effects of multicollinearity. The negative binomial regression model (NBRM) is a well-known model in application when the response variable is a count data with overdispersion. However, it is known that the variance of maximum likelihood estimator (MLE) of the NBRM coefficients can negatively affected in the presence of multicollinearity. In this paper, the generalized ridge estimator is proposed to overcome the limitation of ridge estimator. Several methods for estimating the shrinkage matrix have been adapted. Our Monte Carlo simulation results suggest that the proposed estimator, regardless the type of estimating method of shrinkage matrix is better than the MLE estimator and ridge estimator, in terms of MSE. In addition, some estimating method of shrinkage matrix can bring significant improvement relative to others.

Bootstrap Liu estimators for Poisson regression model

The Liu estimator is used to get precise estimatesby introducing bootstrap technique to reduce the problem of multicollinearity in Poisson regression model. In the presence of multicollinearity, the variance of maximum likelihood estimator (MLE) becomes overstated and theinference based on MLEdoes not remain trustworthy. In this article, we proposed some new Poisson bootstrap Liu and ridge estimators. The proposed estimators are then compared with the non-bootstrap Liu and ridge estimators. Based on mean-squared error criterion, the simulation study revealed showed that the proposed estimators showed efficient results as compared to other existing estimators. Finally, a real example is used to illustrate the application ofproposed estimators.

Bayesian optimal design for non-linear model under non-regularity condition

The common classical approach for finding optimal design is minimizing the variance of an unbiased maximum likelihood estimator (MLE) of parameters. However, under regularity condition, the variance of MLE is approximated by the Cramer–Rao lower bound. In this article, optimal designs are obtained under non-regularity condition in non-linear models. In the Bayesian approach, conditional mutual information is used to propose a new optimality criterion; Bayesian optimal design maximizes the mutual information between the observation and the model parameters. A Bayesian compound criterion is also provided to facilitate the performance comparison of the optimal designs. Finally, the equivalence theorem is given for criterion to allow for checking the optimality of the obtained Bayesian design points.

A New Ridge Estimator for the Poisson Regression Model

The ridge regression model has been consistently demonstrated to be an attractive shrinkage method to reduce the effects of multicollinearity. The Poisson regression model is a well-known model in application when the response variable is count data. However, it is known that multicollinearity negatively affects the variance of maximum likelihood estimator of the Poisson regression coefficients. To address this problem, a Poisson ridge regression model has been proposed by numerous researchers. In this paper, a new Poisson ridge estimator (NPRRM) is proposed and derived. The idea behind the NPRRM is to get diagonal matrix with small values of diagonal elements that leading to decrease the shrinkage parameter, and therefore, the resultant estimator can be better with small amount of bias. Our Monte Carlo simulation results suggest that the NPRRM estimator can bring significant improvement relative to other existing estimators. In addition, the real application results demonstrate that the NPRRM estimator outperforms both Poisson ridge regression and maximum likelihood estimators in terms of predictive performance.

Optimal Plans of Constant-Stress Accelerated Life Tests for Extension of the Exponential Distribution

Abstract In this article, the optimal plans for a k-level constant-stress accelerated life test are presented for extension of the exponential failure data under complete sampling. According to the log-linear life-stress relationship, the optimal proportion of test units allocated to each stress level is determined under D- and C-optimality criteria. Moreover, a real data set is analyzed to illustrate the proposed procedures. Furthermore, the real data set is used to show that extension of the exponential (EE) distribution can be a better model than both Weibull distribution and generalized exponential distribution. In addition, numerical examples are used to illustrate the proposed procedures and to compare between the D-optimal plan and C-optimal plan through asymptotic variance of maximum likelihood estimators. Finally, some interesting conclusions are obtained.

Shrinkage estimators for gamma regression model

The ridge regression model has been consistently demonstrated to be an attractive shrinkage method to reduce the effects of multicollinearity. The gamma regression model is a well-known model in application when the response variable positively skewed. However, it is known that multicollinearity negatively affects the variance of maximum likelihood estimator of the gamma regression coefficients. To address this problem, a gamma ridge regression model (GRRM) has been proposed. The performance of GRRM is fully depending on the shrinkage parameter. In this paper, numerous selection methods of the shrinkage parameter are explored and investigated. In addition, their predictive performances are considered. Our Monte Carlo simulation results suggest that some estimators can bring significant improvement relative to others, in terms of mean squared error and prediction mean squared error.

Restricted ridge estimator in generalized linear models: Monte Carlo simulation studies on Poisson and binomial distributed responses

ABSTRACTIt is known that collinearity among the explanatory variables in generalized linear models (GLMs) inflates the variance of maximum likelihood estimators. To overcome multicollinearity in GLMs, ordinary ridge estimator and restricted estimator were proposed. In this study, a restricted ridge estimator is introduced by unifying the ordinary ridge estimator and the restricted estimator in GLMs and its mean squared error (MSE) properties are discussed. The MSE comparisons are done in the context of first-order approximated estimators. The results are illustrated by a numerical example and two simulation studies are conducted with Poisson and binomial responses.

On the restricted almost unbiased Liu estimator in the logistic regression model

ABSTRACTIt is known that when the multicollinearity exists in the logistic regression model, variance of maximum likelihood estimator is unstable. As a remedy, in the context of biased shrinkage Liu estimation, Chang introduced an almost unbiased Liu estimator in the logistic regression model. Making use of his approach, when some prior knowledge in the form of linear restrictions are also available, we introduce a restricted almost unbiased Liu estimator in the logistic regression model. Statistical properties of this newly defined estimator are derived and some comparison results are also provided in the form of theorems. A Monte Carlo simulation study along with a real data example are given to investigate the performance of this estimator.

Optimal designs for accelerated life tests with multiple stress factors and heteroscedasticity

ABSTRACTAccelerated life testing (ALT) provides a means of obtaining data on product lifetime and reliability relatively quickly by subjecting products to higher-than-usual levels of stress factors. We present methods for obtaining optimal designs for multiple-factor ALTs with time censoring and heteroscedasticity in order to estimate percentiles of product life at usage conditions. We assume a Weibull life distribution and log-linear life–stress relationships with non constant shape parameter for the ALT stress factors. The primary optimality criterion is the minimization of the asymptotic variance of maximum likelihood estimator of the percentile estimator at usage stress. We also consider a secondary criterion for our design optimization. The design construction methods are illustrated by two practical examples.

Partial accelerated degradation test plans for Wiener degradation processes

Securing reliability information on newly developed products is difficult, as only a few samples are available in the development stage. In practice, life tests for new products are conducted under both the normal use condition and accelerated stress conditions. In this study, a two-phase partially accelerated degradation test (PADT) is proposed, which is designed to efficiently obtain product lifetime information in the development stage for new products, when reliability-related information is generally limited and there is limited sample availability. In the first phase, a two-stress-level PADT plan is developed to estimate the parameters of an accelerated degradation model based on the D-optimality criterion. In the second phase, a three-stress-level compromise plan is developed such that the asymptotic variance of maximum likelihood estimate of q-th quantile of the lifetime distribution under the use condition is minimized. Finally, the two-phase systematic test plan is illustrated with a numerical example.

Optimal Plans of Constant-Stress Accelerated Life Tests for the Lindley Distribution

Abstract In this paper, the optimal plans for k-level constant-stress accelerated life test are presented for Lindley failure data under complete sampling. According to the log-linear life-stress relationship, the optimal proportion of test units allocated to each stress level is determined under D- and C-optimality criteria. Moreover, two real data sets are analyzed to illustrate the proposed procedures. Furthermore, the real data sets are used to show that the Lindley distribution can be a better model than one based on the exponential distribution. In addition, numerical examples are used to compare between the D-optimal plan, C-optimal plan, and traditional plan through asymptotic variance of maximum likelihood estimators (MLEs). Finally, some interesting conclusions are obtained.

Some new methods to solve multicollinearity in logistic regression

ABSTRACTThe binary logistic regression is a widely used statistical method when the dependent variable is binary or dichotomous. In some of the situations of logistic regression, independent variables are collinear which leads to the problem of multicollinearity. It is known that multicollinearity affects the variance of maximum likelihood estimator (MLE) negatively. Thus, this article introduces new methods to estimate the shrinkage parameters of Liu-type logistic estimator proposed by Inan and Erdogan (2013) which is a generalization of the Liu-type estimator defined by Liu (2003) for the linear model. A Monte Carlo study is used to show the effectiveness of the proposed methods over MLE using the mean squared error (MSE) and mean absolute error (MAE) criteria. A real data application is illustrated to show the benefits of new methods. According to the results of the simulation and application proposed methods have better performance than MLE.

Restricted ridge estimator in the logistic regression model

ABSTRACTIt is known that when the multicollinearity exists in the logistic regression model, variance of maximum likelihood estimator is unstable. As a remedy, Schaefer et al. presented a ridge estimator in the logistic regression model. Making use of the ridge estimator, when some linear restrictions are also present, we introduce a restricted ridge estimator in the logistic regression model. Statistical properties of this newly defined estimator will be studied and comparisons are done in the simulation study in the sense of mean squared error criterion. A real-data example and a simulation study are introduced to discuss the performance of this estimator.

Optimum quadratic step-stress accelerated life test plan for Weibull distribution under type-I censoring

This paper presents the optimum stress changing times for 3-step, step stress accelerated life testing under the cumulative exposure model with type-I censoring. The lifetimes of test units are assumed to follow Weibull distribution. The scale parameter of the Weibull failure time at constant stress level is assumed to be a log-quadratic function of the stress level. We derive an optimum test plans to minimize the asymptotic variance of maximum likelihood estimator of given pth percentile of the distribution at a design stress. The optimum test plan based on simulated observations is illustrated through a numerical example. The maximum likelihood estimates and asymptotic interval estimates are obtained using R software.