Bias Of Maximum Likelihood Estimator Research Articles

Bayesian solution to the monotone likelihood in the standard mixture cure model

An advantage of the standard mixture cure model over an usual survival model is how it accounts for the population heterogeneity. It allows a joint estimation for the distribution related to the susceptible and non‐susceptible subjects. The estimation algorithm may provide coefficients when the likelihood cannot be maximized. This phenomenon is known as Monotone Likelihood (ML), common in survival and logistic regressions. The ML tends to appear in situations with small sample size, many censored times, many binary or unbalanced covariates. Particularly, it occurs when all uncensored cases correspond to one level of a binary covariate. The existing frequentist solution is an adaptation of the Firth correction, originally proposed to reduce bias of maximum likelihood estimates. It prevents estimates by penalizing the likelihood, with the penalty interpreted as the Bayesian Jeffreys prior. In this paper, the penalized likelihood of the standard mixture cure model is considered with different penalties (Bayesian priors). A Monte Carlo simulation study indicates good inference results, especially for balanced data sets. Finally, a real application involving a melanoma data illustrates the approach.

Cross-section asymptotic for random-effects panel data models with autoregressive errors

A useful strategy to examine serial correlation in panel regression models is to adopt serially correlated error terms. The random effect is also present to address the individual-specific heterogeneity. Familiar transformations can restructure the AR(1) process into a serially uncorrelated dynamic model. It may result in unfitting estimation results once the statistical inference is carried on the conditional likelihood. We discuss in the article the consequence of this action. In particular, in the case of large cross-section units and short time-sequences, we provide analytical expressions for the asymptotic bias of maximum likelihood estimates when the analyst naïvely neglects the initial conditions problem. Under several conditions, we compute the size of biases and measure the mean squared errors to illustrate this violation. We also construct a joint regression model of the initial and subsequent responses to handle the initial conditions. Then, we conduct simulation studies to determine the asymptotic behavior.

Item Count Technique with a Continuous or Count Control Variable for Analyzing Sensitive Questions in Surveys

Abstract The paper introduces a new item count technique (ICT) for dealing with sensitive features. One of the aims of the proposed method is to eliminate both the “ceiling” and “floor” effects that can occur in the original ICT. In the new model, in contrast to the classical one and recently proposed models, the two groups into which the sample is split are treated similarly. Therefore, there is no traditional distinction between the control and treatment groups. The control variable can be either a qualitative count variable or a continuous variable. To our knowledge, ICT methods with continuous control have not been proposed before. Although our new model differs substantially in the approach, it requires only a slight modification of the questionnaire design, which in no way leaks respondents’ privacy. Substantial gain in statistical efficiency is achieved as compared to the earlier proposed Poisson and negative binomial ICTs. The new method also reduces the bias of maximum likelihood estimators and allows to obtain bootstrap confidence intervals with a substantially smaller width. The statistical properties of the method are investigated theoretically and by simulations.

Bias correction for estimates from linear excess relative risk models in small case-control studies.

Epidemiologic studies conducted to quantify the impact of radiation dose d on an outcome typically model the hazard ratio (HR) for association using a linear term, , via a linear excess relative risk (ERR) model, based on biological considerations. To study associations of risk of a second cancer with radiation treatment for a first cancer, several nested case-control designs to estimate have been proposed that use refined doses received by different locations in the organ of interest. Here we first evaluated the small sample bias in maximum likelihood estimates of for the linear ERR model using location-specific radiation doses in simulations. As we found substantial upward bias for studies of realistic sample sizes (more than 50% relative bias for studies with 75 cases), we also proposed and investigated several approaches to correct this bias. We studied first and second order jackknife bias corrections and we derived a modified set of score functions under retrospective case-control sampling, from which we directly obtained bias-corrected estimates. In simulations based on doses from a study of stomach cancer among testicular cancer survivors and synthetically generated data, neither the first nor second order jackknife bias correction performed well. Estimates based on the modified score equations corrected the bias much better, albeit not completely, and were numerically much more stable.

A comparison of estimators for the network autocorrelation model based on observed social networks

Network autocorrelation models (NAMs) are widely used to study a response variable of interest among subjects embedded within a network. Although the NAM is highly useful for studying such networked observational units, several simulation studies have raised concerns about point estimation. Specifically, these studies have consistently demonstrated a negative bias of maximum likelihood estimators (MLEs) of the network effect parameter. However, in order to gain a practical understanding of point estimation in the NAM, these findings need to be expanded in three important ways. First, these simulation studies are based on relatively simple network generative models rather than observed networks, thereby leaving as an open question how realistic network topologies may affect point estimation in practice. Second, although there has been strong work done in developing two-stage least squares estimators as well as Bayesian estimators, only the MLE has received extensive attention in the literature, thus leaving practitioners in question as to best practices. Third, the performance of these estimators need to be compared using both bias and variance, as well as the coverage rate of each estimator's corresponding confidence or credible interval. In this paper we describe a simulation study which aims to overcome these shortcomings in the following way. We first fit real social networks using the exponential random graph model and used the Bayesian predictive posterior distribution to generate networks with realistic topologies. We then compared the performance of the three different estimators mentioned above.

On the Generalized Odd Transmuted Two-Sided Class of Distributions

In this paper, a general class of two-sided lifetime distributions is introduced via odd ratio function, the well-known concept in survival analysis and reliability engineering. Some statistical and reliability properties including survival function, quantiles, moments function, asymptotic and maximum likelihood estimation are provided in a general setting. A special case of this new family is taken up by considering the exponential model as the parent distribution. Some characteristics of this specialized model and also a discussion associated with survival regression are provided.A simulation study is presented to investigate the bias and mean square error of the maximum likelihood estimators. Moreover, two examples of real data sets are studied; point and interval estimations of all parameters are obtained by maximum likelihood and bootstrap (parametric and non-parametric) procedures. Finally, the superiority of the proposed model over some common statistical distributions is shown through the different criteria for model selection including loglikelihood values, Akaike information criterion and Kolmogorov-Smirnov test statistic values.

Corrected Maximum Likelihood Estimations of the Lognormal Distribution Parameters

As a result of asymmetry in practical problems, the Lognormal distribution is more suitable for data modeling in biological and economic fields than the normal distribution, while biases of maximum likelihood estimators are regular of the order O ( n − 1 ) , especially in small samples. It is of necessity to derive logical expressions for the biases of the first-order and nearly consistent estimators by bias correction techniques. Two methods are adopted in this article. One is the Cox-Snell method. The other is the resampling method known as parametric Bootstrap. They can improve maximum likelihood estimators performance and correct biases of the Lognormal distribution parameters. Through Monte Carlo simulations, we obtain average root mean squared error and bias, which are two important indexes to compare the effect of different methods. The numerical results reveal that for small and medium-sized samples, the performance of analytical bias correction estimation is superior than bootstrap estimation and classical maximum likelihood estimation. Finally, an example is given based on the actual data.

Bias Reduction for the Marshall-Olkin Extended Family of Distributions with Application to an Airplane’s Air Conditioning System and Precipitation Data

The Marshall-Olkin extended family of distributions is an alternative for modeling lifetimes, and considers more or less asymmetry than its parent model, achieved by incorporating just one extra parameter. We investigate the bias of maximum likelihood estimators and use it to develop an estimator with less bias than traditional estimators, by a modification of the score function. Unlike other proposals, in this paper, we consider a bias reduction methodology that can be applied to any member of the family and not necessarily to any particular distribution. We conduct a Monte Carlo simulation in order to study the performance of the corrected estimators in finite samples. This simulation shows that the maximum likelihood estimator is quite biased and the proposed estimator is much less biased; in small sample sizes, the bias is reduced by around 50 percent. Two applications, related to the air conditioning system of an airplane and precipitations, are presented to illustrate the results. In those applications, the bias reduction for the shape parameters is close to 25% and the bias reduction also reduces, among others things, the width of the 95% confidence intervals for quantiles lower than 0.594.

Logistic Regression Procedure Using Penalized Maximum Likelihood Estimation for Differential Item Functioning

AbstractIn the logistic regression (LR) procedure for differential item functioning (DIF), the parameters of LR have often been estimated using maximum likelihood (ML) estimation. However, ML estimation suffers from the finite‐sample bias. Furthermore, ML estimation for LR can be substantially biased in the presence of rare event data. The bias of ML estimation due to small samples and rare event data can degrade the performance of the LR procedure, especially when testing the DIF of difficult items in small samples. Penalized ML (PML) estimation was originally developed to reduce the finite‐sample bias of conventional ML estimation and also was known to reduce the bias in the estimation of LR for the rare events data. The goal of this study is to compare the performances of the LR procedures based on the ML and PML estimation in terms of the statistical power and Type I error. In a simulation study, Swaminathan and Rogers's Wald test based on PML estimation (PSR) showed the highest statistical power in most of the simulation conditions, and LRT based on conventional PML estimation (PLRT) showed the most robust and stable Type I error. The discussion about the trade‐off between bias and variance is presented in the discussion section.

Bias induced by adaptive dose-finding designs

There is a long literature on bias in maximum likelihood estimators. Here we demonstrate that adaptive dose-finding procedures (such as Continual Reassessment Methods, Up-and-Down and Interval Designs) themselves induce bias. In particular, with Bernoulli responses and dose assignments that depend on prior responses, we provide an explicit formula for the bias of observed response rates. We illustrate the patterns of bias for designs that aim to concentrate dose allocations around a target dose, which represents a specific quantile of a cumulative response-threshold distribution. For such designs, bias tends to be positive above the target dose and negative below it. To our knowledge, this property of dose-finding designs has not previously been recognized by design developers. We discuss the implications of this bias and suggest a simple shrinkage mitigation formula that improves estimation at doses away from the target.

The empirical likelihood prior applied to bias reduction of general estimating equations

The practice of employing empirical likelihood (EL) components in place of parametric likelihood functions in the construction of Bayesian-type procedures has been well-addressed in the modern statistical literature. The EL prior, a Jeffreys-type prior, which asymptotically maximizes the Shannon mutual information between data and the parameters of interest, is rigorously derived. The focus of the proposed approach is on an integrated Kullback–Leibler distance between the EL-based posterior and prior density functions. The EL prior density is the density function for which the corresponding posterior form is asymptotically negligibly different from the EL. The proposed result can be used to develop a methodology for reducing the asymptotic bias of solutions of general estimating equations and M-estimation schemes by removing the first-order term. This technique is developed in a similar manner to methods employed to reduce the asymptotic bias of maximum likelihood estimates via penalizing the underlying parametric likelihoods by their Jeffreys invariant priors. A real data example related to a study of myocardial infarction illustrates the attractiveness of the proposed technique in practical aspects.

Small sample bias correction or bias reduction?

Many problems in biomedical and other sciences are subject to biased estimates (maximum likelihood or of similar types). In two seminal papers Cox and Snell (1968) and Firth (1993) deal with first order bias of maximum likelihood estimates. Cox and Snell obtain a correction term that corrects, approximately, first order bias and Firth uses an adjustment to the score function; the solution of the estimating equation obtained by solving the adjusted score function to zero, removes the first order bias of the maximum likelihood estimates approximately. In many applications authors use one of these two procedures for bias correction without being aware that the other exists or whether these two procedures are equivalent. In this paper we investigate the equivalence issue of the two methods through theoretical analysis, simulation study and data analysis. We show that the two methods yield either exactly the same estimates or that the preventive method has some edge over the other.

A Bias Compensation Method for Distributed Moving Source Localization Using TDOA and FDOA with Sensor Location Errors.

Current bias compensation methods for distributed localization consider the time difference of arrival (TDOA) and frequency difference of arrival (FDOA) measurements noise, but ignore the negative influence by the sensor location uncertainties on source localization accuracy. Therefore, a new bias compensation method for distributed localization is proposed to improve the localization accuracy in this paper. This paper derives the theoretical bias of maximum likelihood estimation when the sensor location errors and positioning measurements noise both exist. Using the rough estimate result by MLE to subtract the theoretical bias can obtain a more accurate source location estimation. Theoretical analysis and simulation results indicate that the theoretical bias derived in this paper matches well with the actual bias in moderate noise level so that it can prove the correctness of the theoretical derivation. Furthermore, after bias compensation, the estimate accuracy of the proposed method achieves a certain improvement compared with existing methods.

Improved estimation in a general multivariate elliptical model

The problem of reducing the bias of maximum likelihood estimator in a general multivariate elliptical regression model is considered. The model is very flexible and allows the mean vector and the dispersion matrix to have parameters in common. Many frequently used models are special cases of this general formulation, namely: errors-in-variables models, nonlinear mixed-effects models, heteroscedastic nonlinear models, among others. In any of these models, the vector of the errors may have any multivariate elliptical distribution. We obtain the second-order bias of the maximum likelihood estimator, a bias-corrected estimator, and a bias-reduced estimator. Simulation results indicate the effectiveness of the bias correction and bias reduction schemes.

Brief Research Report: Growth Models With Small Samples and Missing Data

ABSTRACTSmall samples are common in growth models due to financial and logistical difficulties of following people longitudinally. For similar reasons, longitudinal studies often contain missing data. Though full information maximum likelihood (FIML) is popular to accommodate missing data, the limited number of studies in this area have found that FIML tends to perform poorly with small-sample growth models. This report demonstrates that the fault lies not with how FIML accommodates missingness but rather with maximum likelihood estimation itself. We discuss how the less popular restricted likelihood form of FIML, along with small-sample-appropriate methods, yields trustworthy estimates for growth models with small samples and missing data. That is, previously reported small sample issues with FIML are attributable to finite sample bias of maximum likelihood estimation not direct likelihood. Estimation issues pertinent to joint multiple imputation and predictive mean matching are also included and discussed.

On the statistical model of source localization based on Range Difference measurements

In this work we study the statistical model of source localization based on Range Difference measurements, under the assumption of Gaussian noise on the data. Our analysis is based on a previous work of the same authors concerning the localization in a noiseless scenario. We investigate the case of planar localization of a source using a minimal configuration of three non aligned receivers. We have four curved exponential families corresponding to four different, non disjoint, regions of the feasible set. For each family we solve Maximum Likelihood Estimation (MLE). This requires to find the projection of a point on a set of segments and arcs of ellipse. Then, we perform the analytic study of the localization accuracy. In particular, we give formulas for mean square error and bias of MLE, depending on the displacement vectors. We validate the results through Monte Carlo simulations, in a given setup of the receivers. As the set of feasible measurements is a semialgebraic variety, this investigation makes use of techniques from Algebraic Statistics and Information Geometry.

The Gamma log-logistic Weibull distribution: model, properties and application

In this paper, a new generalized distribution called the gamma log-logistic Weibull (GLLoGW) distribution is proposed and studied. The GLLoGW distribution include the gamma log-logistic, gamma log-logistic Rayleigh, gamma log logistic exponential, log-logistic Weibull, log-logistic Rayleigh, log-logistic exponential, log-logistic as well as other special cases as sub-models. Some mathematical properties of the new distribution including moments, conditional moments, mean and median deviations, Bonferroni and Lorenz curves, distribution of the order statistics and R\'enyi entropy are derived. Maximum likelihood estimation technique is used to estimate the model parameters. A Monte Carlo simulation study to examine the bias and mean square error of the maximum likelihood estimators is presented and an application to real dataset to illustrate the usefulness of the model is given.

Firth's logistic regression with rare events: accurate effect estimates and predictions?

Firth's logistic regression has become a standard approach for the analysis of binary outcomes with small samples. Whereas it reduces the bias in maximum likelihood estimates of coefficients, bias towards one-half is introduced in the predicted probabilities. The stronger the imbalance of the outcome, the more severe is the bias in the predicted probabilities. We propose two simple modifications of Firth's logistic regression resulting in unbiased predicted probabilities. The first corrects the predicted probabilities by a post hoc adjustment of the intercept. The other is based on an alternative formulation of Firth's penalization as an iterative data augmentation procedure. Our suggested modification consists in introducing an indicator variable that distinguishes between original and pseudo-observations in the augmented data. In a comprehensive simulation study, these approaches are compared with other attempts to improve predictions based on Firth's penalization and to other published penalization strategies intended for routine use. For instance, we consider a recently suggested compromise between maximum likelihood and Firth's logistic regression. Simulation results are scrutinized with regard to prediction and effect estimation. We find that both our suggested methods do not only give unbiased predicted probabilities but also improve the accuracy conditional on explanatory variables compared with Firth's penalization. While one method results in effect estimates identical to those of Firth's penalization, the other introduces some bias, but this is compensated by a decrease in the mean squared error. Finally, all methods considered are illustrated and compared for a study on arterial closure devices in minimally invasive cardiac surgery. Copyright © 2017 John Wiley & Sons, Ltd.

Estimators for longitudinal latent exposure models: examining measurement model assumptions.

Latent variable (LV) models are increasingly being used in environmental epidemiology as a way to summarize multiple environmental exposures and thus minimize statistical concerns that arise in multiple regression. LV models may be especially useful when multivariate exposures are collected repeatedly over time. LV models can accommodate a variety of assumptions but, at the same time, present the user with many choices for model specification particularly in the case of exposure data collected repeatedly over time. For instance, the user could assume conditional independence of observed exposure biomarkers given the latent exposure and, in the case of longitudinal latent exposure variables, time invariance of the measurement model. Choosing which assumptions to relax is not always straightforward. We were motivated by a study of prenatal lead exposure and mental development, where assumptions of the measurement model for the time-changing longitudinal exposure have appreciable impact on (maximum-likelihood) inferences about the health effects of lead exposure. Although we were not particularly interested in characterizing the change of the LV itself, imposing a longitudinal LV structure on the repeated multivariate exposure measures could result in high efficiency gains for the exposure-disease association. We examine the biases of maximum likelihood estimators when assumptions about the measurement model for the longitudinal latent exposure variable are violated. We adapt existing instrumental variable estimators to the case of longitudinal exposures and propose them as an alternative to estimate the health effects of a time-changing latent predictor. We show that instrumental variable estimators remain unbiased for a wide range of data generating models and have advantages in terms of mean squared error. Copyright © 2017 John Wiley & Sons, Ltd.

Bias corrected MLEs under progressive type-II censoring scheme

ABSTRACTCensoring frequently occurs in survival analysis but naturally observed lifetimes are not of a large size. Thus, inferences based on the popular maximum likelihood (ML) estimation which often give biased estimates should be corrected in the sense of bias. Here, we investigate the biases of ML estimates under the progressive type-II censoring scheme (pIIcs). We use a method proposed in Efron and Johnstone [Fisher's information in terms of the hazard rate. Technical Report No. 264, January 1987, Stanford University, Stanford, California; 1987] to derive general expressions for bias corrected ML estimates under the pIIcs. This requires derivation of the Fisher information matrix under the pIIcs. As an application, exact expressions are given for bias corrected ML estimates of the Weibull distribution under the pIIcs. The performance of the bias corrected ML estimates and ML estimates are compared by simulations and a real data application.