Usual Maximum Likelihood Estimator Research Articles

Two-sample empirical likelihood method for difference between coefficients in linear regression model

The empirical likelihood method is proposed to construct the confidence regions for the difference in value between coefficients of two-sample linear regression model. Unlike existing empirical likelihood procedures for one-sample linear regression models, as the empirical likelihood ratio function is not concave, the usual maximum empirical likelihood estimation cannot be obtained directly. To overcome this problem, we propose to incorporate a natural and well-explained restriction into likelihood function and obtain a restricted empirical likelihood ratio statistic (RELR). It is shown that RELR has an asymptotic chi-squared distribution. Furthermore, to improve the coverage accuracy of the confidence regions, a Bartlett correction is applied. The effectiveness of the proposed approach is demonstrated by a simulation study.

A Fresh Look at the Discriminant Function Approach for Estimating Crude or Adjusted Odds Ratios

Assuming a binary outcome, logistic regression is the most common approach to estimating a crude or adjusted odds ratio corresponding to a continuous predictor. We revisit a method termed the discriminant function approach, which leads to closed-form estimators and corresponding standard errors. In its most appealing application, we show that the approach suggests a multiple linear regression of the continuous predictor of interest on the outcome and other covariates, in place of the traditional logistic regression model. If standard diagnostics support the assumptions (including normality of errors) accompanying this linear regression model, the resulting estimator has demonstrable advantages over the usual maximum likelihood estimator via logistic regression. These include improvements in terms of bias and efficiency based on a minimum variance unbiased estimator of the log odds ratio, as well as the availability of an estimate when logistic regression fails to converge due to a separation of data points. Use of the discriminant function approach as described here for multivariable analysis requires less stringent assumptions than those for which it was historically criticized, and is worth considering when the adjusted odds ratio associated with a particular continuous predictor is of primary interest. Simulation and case studies illustrate these points.

New classes of estimators for dispersion parameter of a chi-distributed radial error with application to target analysis

In this paper we have investigated some classes of shrinkage estimators for estimating dispersion parameter σ in p-dimensional analogue of the probable error of a single variate. The need to study this parameter arises due to its importance in target analysis problems to estimate Circular Probable Error (CPE) and Spherical Probable Error (SPE). It is assumed that the prior information or guessed value of the parameter σ say σ0 is available from the past experiences. The properties of the developed shrinkage estimators have been studied when center of impact is known and when it is unknown by using appropriate numerical integration method (10-point Gauss-Laguerre integration method). Simulation studies confirm the high efficiency of the developed classes of shrunken estimators when compared with the usual maximum likelihood estimators (MLE), unbiased estimators and minimum mean squared error (MMSE) estimators.

A Few Remarks on “A Capture–Recapture Approach for Screening Using Two Diagnostic Tests With Availability of Disease Status for the Test Positives Only” by Böhning and Patilea

Using a capture-recapture approach, Bohning and Patilea (2008) proposed two useful estimators for unobserved cell counts, assuming homogeneous association of the screening tests over disease status. However, they are mistaken in claiming that the maximum likelihood estimators (MLEs) are difficult to obtain. The point of this note is to present closed-form MLEs for, in their notation: 1) the α model where α=p11(i)p00(i)/(p01(i)p10(i)) is assumed to be identical for all i = 1, 2, …, d; and 2) the θ model where θ=p1|1(i)/p1+(i) is assumed to be identical for all i. One way to write the likelihood function (ignoring constant terms) in this setting is in terms of qi and pjk(i)(i=1,2,…,d;j=0,1;k=0,1), as the authors did: x00(+)log(∑ip00(i)qi)+∑ix11(i)log(p11(i)qi)+∑ix10(i)log(p10(i)qi)+∑ix01(i)log(p01(i)qi). (1) This parameterization involves a mixture likelihood, preventing closed-form solution for the MLEs. To obtain closed-form MLEs, we consider an alternative parameterization in terms of πjk and πjk(i)(j,k=0,1andi=1,2,…,d) where πjk = P(T1 = j, T2 = k), πjk(i)=P(D=i|T1=j,T2=k). The log-likelihood function is (ignoring constant terms), logL=∑j∑kxjk(+)log(πjk)+∑ix11(i)log(π11(i))+∑ix10(i)log(π10(i))+∑ix01(i)log(π01(i))=x00(+)log(π00)+∑dx11(i){log(π11(i))+log(π11)}+∑ix10(i){log(π10(i))+log(π10)}+∑ix01(i){log(π01(i))+log(π01)}=x00(+)log(π00)+∑ix11(i)log(π11(i)π11)+∑ix10(i)log(π10(i)π10)+∑ix01(i)log(π01(i)π01) (2) This representation relates to previous work in some other settings (Satten and Kupper 1993; Lyles 2002; Pepe and Janes 2007). Note that, πjk(i)πjk=P(D=i|T1=j,T2=k)P(T1=j,T2=k)=P(T1=j,T2=k|D=i)P(D=i)=pjk(i)qi, and π00=P(T1=0,T2=0)=∑iP(T1=0,T2=0,D=i)=∑iP(T1=0,T2=0|D=i)P(D=i)=∑ip00(i)qi Therefore equation (2) is equivalent to equation (1). These equations are tractable and yield closed-form MLEs of πjk (j, k =0, 1) and πjk(i) if j + k > 0. Omitting the algebra, we obtain the MLEs as π`jk=xjk/n(j,k=0,1) and π`jk(i)=xjk(i)/xjk if j + k > 0. Therefore, the MLEs of qi s, which can be written as functions of πjk (j, k = 0, 1) and πjk(i)(j+k>0) under the α or θ model assumptions, have closed-form solutions. The details are given below. Under the α model, α=p11(i)p00(i)p01(i)p10(i) is assumed to be identical for all i = 1, 2, …, d; by Bayes’ rule, α=p11(i)p00(i)p01(i)p10(i)=P(T1=1,T2=1|D=i)P(T1=0,T2=0|D=i)P(T1=0,T2=1|D=i)P(T1=1,T2=0|D=i)=P(D=i|T1=1,T2=1)P(T1=1,T2=1)P(D=i|T1=0,T2=0)P(T1=0,T2=0)P(D=i|T1=1,T2=0)P(T1=1,T2=0)P(D=i|T1=0,T2=1)P(T1=0,T2=1)=π11π00π01π10×π11(i)π00(i)π01(i)π10(i), Thus α=π11π00π01π10×[∑iπ01(i)π10(i)π11(i)]−1,π00α(i)=π01(i)π10(i)π11(i)×[∑iπ01(i)π10(i)π11(i)]−1,qiα=π11π11(i)+π10π10(i)+π01π01(i)+π00π01(i)π10(i)π11(i)[∑iπ01(i)π10(i)π11(i)]−1, where the subscript α indicates the α model assumption. Since the MLEs of the parameters πjk and πjk(i) are π`jk=xjk/n(j,k=0,1) and π`jk(i)=xjk(i)/xjk if j + k > 0, the closed-form MLE of niα under the α model is n^iα=nq^iα=x11(i)+x10(i)+x01(i)+x00x01(i)x10(i)x11d[∑ix01(i)x10(i)x11(i)]−1, (3) which is essentially the same as the equation (15) in Bohning and Patilea (2008) without the stability correction. In other words, the estimator obtained in equation (15) is the MLE under the α model assumption with the stability correction. Under the θ model θ=p1|1(i)p1+(i) is assumed to be identical for all i = 1, 2, …, d; by Bayes’ rule θ=p1|1(i)p1+(i)=P(T1=1|T2=1,D=i)P(T1=1|D=i)=P(D=i|T1=1,T2=1)P(T1=1,T2=1)P(T2=1,D=i)P(T1=1,D=i)×P(D=i)=π11π11(i)(π01π01(i)+π11π11(i))(π10π10(i)+π11π11(i))×P(D=i), Thus θ=[∑i(π10π10(i)π11π11(i)+1)(π01π01(i)+π11π11(i))]−1 and π00θ(i)=1π00{(π10π10(i)π11π11(i)+1)(π01π01(i)+π11π11(i))[∑i(π10π10(i)π11π11(i)+1)(π01π01(i)+π11π11(i))]−1−π11π11(i)−π10π10(i)−π01π01(i)} qiθ=(π10π10(i)π11π11(i)+1)(π01π01(i)+π11π11(i))[∑i(π10π10(i)π11π11(i)+1)(π01π01(i)+π11π11(i))]−1, where the subscript θ indicates the θ model assumption. Similarly, the closed-form MLE of under niθ under the θ model is n^iθ=nq^iθ=(x10(i)x11(i)+1)(x01(i)+x11(i))[∑i(x10(i)x11(i)+1)(x01(i)+x11(i))]−1=x1+(i)x+1(i)x11(i)[∑ix1+(i)x+1(i)x11(i)]−1, (4) which is essentially the same as the equation (10) in Bohning and Patilea (2008) without the stability correction. As a byproduct of this alternative parameterization, we can test the difference between ^qiθ and ^qiα (or equivalently, the difference between ^niθ and ^niα) to make inference on whether these two assumptions provide statistically significantly different predictions for the probability (or equivalently, the number) of individuals with certain disease class i. Although the formula for se(^qiθ − ^qiα) is tedious, its numerical value can be obtained easily through statistical software using the delta method. We note that the difference between estimated probabilities of disease classes under the α and θ models can be statistically different and potentially meaningful for the same study. For example, in the Health Insurance Plan Study for breast cancer screening in New York (Strax, Venet Shapiro and Gross 1967), the estimated probability of having cancer assuming the α model is 4.8% with a 95% confidence interval (CI) of 0.3% to 9.3%, while the estimated probability of having cancer assuming the θ model is 7.5% with 95% CI of 2.8% to 12.2%. The difference is 2.7% (95% CI: 1.4% to 4%) with a p-value less than 0.001. This difference can have a big impact on the cancer surveillance and prevention. Unfortunately, the data does not contain information to differentiate the α model versus the θ model. The alternative parameterization in (2) sheds lights on maximum likelihood approaches in the setting considered here; the corresponding closed-form ML estimators under the α and θ models allow tests of the difference between the estimated probabilities of a specific disease class using the α versus the θ model. Our results complements the estimators obtained in equations (10) and (15) by Bohning and Patilea (2008) using a capture-recapture approach, and ensure the usual MLE properties.

Estimating a positive normal mean

Let X 1, X 2, ..., X n be a random sample from a normal population with mean μ and variance σ 2. In many real life situations, specially in lifetime or reliability estimation, the parameter μ is known a priori to lie in an interval [a, ∞). This makes the usual maximum likelihood estimator (MLE) an inadmissible estimator of μ with respect to the squared error loss. This is due to the fact that it may take values outside the parameter space. Katz (1961) and Gupta and Rohatgi (1980) proposed estimators which lie completely in the given interval. In this paper we derive some new estimators for μ and present a comparative study of the risk performance of these estimators. Both the known and unknown variance cases have been explored. The new estimators are shown to have superior risk performance over the existing ones over large portions of the parameter space.

On estimation of the shape parameter of the gamma distribution

The problem of estimation of an unknown shape parameter under the sample drawn from the gamma distribution, where the scale parameter is also unknown, is considered. A new estimator, called the maximum likelihood scale invariant estimator, is proposed. It is established that both the bias and the variance of this estimator are less than that of the usual maximum likelihood estimator. A property of the psi function is also obtained.

Cox regression analysis in presence of collinearity: an application to assessment of health risks associated with occupational radiation exposure

This paper considers the analysis of time to event data in the presence of collinearity between covariates. In linear and logistic regression models, the ridge regression estimator has been applied as an alternative to the maximum likelihood estimator in the presence of collinearity. The advantage of the ridge regression estimator over the usual maximum likelihood estimator is that the former often has a smaller total mean square error and is thus more precise. In this paper, we generalized this approach for addressing collinearity to the Cox proportional hazards model. Simulation studies were conducted to evaluate the performance of the ridge regression estimator. Our approach was motivated by an occupational radiation study conducted at Oak Ridge National Laboratory to evaluate health risks associated with occupational radiation exposure in which the exposure tends to be correlated with possible confounders such as years of exposure and attained age. We applied the proposed methods to this study to evaluate the association of radiation exposure with all-cause mortality.

A third-order bias corrected estimate in generalized linear models

Cordeiro and McCullagh (J Roy Stat Soc Ser B 53:629–643, 1991) derived a second-order bias-corrected estimate, which displays smaller bias than the classical maximum likelihood estimate in generalized linear models. This estimate, although consistent, can display pronounced bias in small to moderately large samples, as shown by Monte Carlo simulations here. In this paper, we obtain a simple matrix formula for the bias of order n-2 of this estimate, where n is the sample size, and define a third-order bias-corrected estimate in this class of models, which displays much smaller bias in small samples. In particular, some Monte Carlo simulations show that our new estimate can deliver substantial improvements in terms of bias and mean squared errors over the usual maximum likelihood estimate and Cordeiro and McCullagh’s estimate.

Estimation of Order Restricted Scale Parameters of Two Pareto Distributions

SYNOPTIC ABSTRACTThe problem of estimating the scale parameters α1 and α2 of two Pareto distributions is investigated with common known shape parameter β and order restriction α1 ≤ α2 We prove the non-existence of unbiased estimators of α1 and α2 and propose several estimators which are better than natural estimators. An admissible class of estimators in scalar multiple class has been found using Brewster-Zidek technique. The bias and risks of these estimators are calculated. The minimax estimators with respect to absolute bias risk criterion are obtained for both ordered scale parameters. Simulation for biases and risks of all these estimators has been carried out and improvement has been shown over usual maximum likelihood estimators.

Robust Weighted Likelihood Estimation of Exponential Parameters

The problem of estimating the parameter of an exponential distribution when a proportion of the observations are outliers is quite important to reliability applications. The method of weighted likelihood is applied to this problem, and a robust estimator of the exponential parameter is proposed. Interestingly, the proposed estimator is an /spl alpha/-trimmed mean type estimator. The large-sample robustness properties of the new estimator are examined. Further, a Monte Carlo simulation study is conducted showing that the proposed estimator is, under a wide range of contaminated exponential models, more efficient than the usual maximum likelihood estimator in the sense of having a smaller risk, a measure combining bias & variability. An application of the method to a data set on the failure times of throttles is presented.

Estimation of Order Restricted Concentration Parameters of von Mises Distributions

ABSTRACT The von Mises distribution is a natural circular analog of the normal distribution on the real line, and is known as the “circular normal distribution”. This distribution has two parameters, known as the concentration parameter and the circular mean (or the mean direction). There are practical situations where it is of interest to estimate the concentration parameters of several von Mises distributions, when it is known apriori that the concentration parameters are subject to a simple order restriction. In this article, we discuss the restricted maximum likelihood estimation of the concentration parameters κ1, …, κ m of m( ≥ 2) von Mises distributions, when it is known apriori that 0 ≤ κ1 ≤ κ2 ≤ … ≤ κ m ≤ ∞. Using the theory of isotonic regression, we derive the restricted maximum likelihood estimators of the concentration parameters. Using approximations of some statistics based on a random sample from the von Mises distributions having large concentration parameters, we propose some more estimators for the order restricted concentration parameters of two von Mises distributions. Using Monte Carlo simulations, the restricted maximum likelihood estimators and the proposed estimators, based on the assumption of large concentration parameters, are compared with the usual (unrestricted) maximum likelihood estimators under the mean squared error criterion.

Estimation of parameters for exponentiated-Weibull family under type-II censoring scheme

Bayes and classical estimators have been obtained for two-parameter exponentiated-Weibull distribution when sample is available from type-II censoring scheme. Bayes estimators have been developed under squared error loss function as well as under LINEX loss function using non-informative type of priors for the parameters. Besides, the generalized maximum likelihood estimators and the usual maximum likelihood estimators have also been attempted. It has been seen that the estimators obtained are not available in nice closed forms, although they can be easily evaluated for the given sample by using suitable numerical methods. The performance of the proposed estimators have been compared on the basis of their simulated risks (average loss over the sample space) obtained under squared error as well as under LINEX loss functions.

An Empirical Bayes Group-Testing Approach to Estimating Small Proportions

Group testing has long been recognized as a safe and sensible alternative to one-at-a-time testing in applications wherein the prevalence rate p is small. In this article, we develop an empirical Bayes (EB) procedure to estimate p using a beta-type prior distribution and a squared-error loss function. We show that the EB estimator is preferred over the usual maximum likelihood estimator (MLE) for small group sizes and small p. In addition, we also discuss interval estimation and consider the use of other loss functions perhaps more appropriate in public health studies. The proposed methods are illustrated using group-testing data from a prospective hepatitis C virus study conducted in Xuzhou City, China.

Threshold estimation in the log-gamma model

The three-parameter log-gamma distribution is a versatile lifetime model. However, it has a quite unusual property that allows it to be used as a potential threshold model: namely, that though it is not left-limited in general, it includes the shifted exponential as a special case. Thus, it is a useful bridge between skewed models that are not explicitly left-limited, on the one hand, and a true threshold model on the other. It is shown that the likelihood always has a local maximum corresponding to the exponential model, even when this is not the true model, so that the usual maximum likelihood (ML) estimator is unsatisfactory. An estimator obtained by maximising a certain spacing-modified likelihood is proposed that does not suffer from this problem. Its distributional properties are derived showing it to be as efficient as ML when this does work and moreover showing the estimators of shape and location to be hyper-efficient when the exponential model is the true model. Simulation results and three numerical examples are given contrasting the behaviour of the proposed estimator with the ML estimator.

Penalized quasi-likelihood with spatially correlated data

This article discusses and evaluates penalized quasi-likelihood (PQL) estimation techniques for the situation where random effects are correlated, as is typical in mapping studies. This is an approximate fitting technique which uses a Laplace approximation to the integrated mixed model likelihood. It is much easier to implement than usual maximum likelihood estimation. Our results show that the PQL estimates are reasonably unbiased for analysis of mixed Poisson models when there is correlation in the random effects, except when the means are sufficiently small to yield sparse data. However, although the normal approximation to the distribution of the parameter estimates works fairly well for the parameters in the mean it does not perform as well for the variance components. In addition, when the mean mortality counts are small, the estimated standard errors of the variance components tend to become more biased than those for the mean. We illustrate our approaches by applying PQL for mapping mortality in British Columbia, Canada, over the five-year period 1985–1989.

The Use of a Distance Measure in Regularised Discriminant Analysis

Friedman (1989) proposed a regularised discriminant function (RDF) as a compromise between the normal-based linear and quadratic discriminant functions, by considering alternatives to the usual maximum likelihood estimates for the covariance matrices. These alternatives are characterised by two (regularisation) parameters, the values of which are customised to individual situations by jointly minimising a sample-based (cross-validated) estimate of future misclassification risk. This technique appears to provide considerable gains in classification accuracy in many circumstances, although it is computationally intensive. Because of the computational burden inherent in the RDF, and with regards to criticisms of the technique by Rayens et al. (1991), we investigated whether information about appropriate values of the two regularisation parameters could be obtained from examining the behaviour of the Bhattacharyya distance between the various populations. This distance measure is found to give information which leads to unique and generally appropriate values for the regularisation parameters being selected.

A modified score function estimator for multinomial logistic regression in small samples

Logistic regression modelling of mixed binary and continuous covariates is common in practice, but conventional estimation methods may not be feasible or appropriate for small samples. It is well known that the usual maximum likelihood estimates (MLEs) of the log-odds-ratio parameters are biased in finite samples, and there is a non-zero probability that an MLE is infinite, i.e., does not exist. In this paper, we extend the approach proposed by Firth (Biometrika 80 (1993) 27) for bias reduction of MLEs in exponential family models to the multinomial logistic regression model, and consider general regression covariate types. The method is based on a suitable modification of the score function that removes first order bias. We apply the method in the analysis of two datasets: one is a study of disease prognosis and the other is a disease prevention trial. In a series of simulation studies in small samples, the modified-score estimates for binomial and trinomial logistic regressions had mean bias closer to zero and smaller mean squared error than other approaches. The modified-score estimates have properties that make them attractive for routine application in logistic regressions of binary and continuous covariates, including the advantage that they can be obtained in samples in which the MLEs are infinite.

On the failure rate estimation of the inverse gaussian distribution

New estimators of the inverse Gaussian failure rate are proposed based on the maximum likelihood predictive densities derived by Yang (1999). These estimators are compared, via Monte Carlo simulation, with the usual maximum likelihood estimators of the failure rate and found to be superior in terms of bias and mean squared error. Sensitivity of the estimators against the departure from the inverse Gaussian distribution is studied.

A note on kernel assisted estimators in missing covariate regression

We investigate the asymptotic relationships among three kernel assisted semiparametric estimators in regression analysis when some covariates are missing or measured with error. Smoothing techniques are employed in estimating the selection probabilities and the conditionally expected scores, a step which is required to obtain the estimators of interest. The asymptotic distributional properties of these estimators are derived and their asymptotic equivalence is shown. Some important differences are also noted. Furthermore, the asymptotic efficiency of the estimators relative to the usual maximum likelihood estimator is obtained.

Estimation of the trace of the scaled covariance matrix of a multivariate t-model using a known information

The trace of the scaled covariance matrix of the multivariate t-distribution is considered for estimation using a power transformation. The proposed estimator always dominates the usual maximum likelihood estimator in the sense of having smaller risk under a quadratic loss function. The dominance behaviour is proved analytically as well as computationally by using Monte-Carlo simulation.