Usual Maximum Likelihood Estimator Research Articles

Multiple imputation with sequential penalized regression.

Missing data is a common issue that can cause problems in estimation and inference in biomedical, epidemiological and social research. Multiple imputation is an increasingly popular approach for handling missing data. In case of a large number of covariates with missing data, existing multiple imputation software packages may not work properly and often produce errors. We propose a multiple imputation algorithm called mispr based on sequential penalized regression models. Each variable with missing values is assumed to have a different distributional form and is imputed with its own imputation model using the ridge penalty. In the case of a large number of predictors with respect to the sample size, the use of a quadratic penalty guarantees unique estimates for the parameters and leads to better predictions than the usual Maximum Likelihood Estimation (MLE), with a good compromise between bias and variance. As a result, the proposed algorithm performs well and provides imputed values that are better even for a large number of covariates with small samples. The results are compared with the existing software packages mice, VIM and Amelia in simulation studies. The missing at random mechanism was the main assumption in the simulation study. The imputation performance of the proposed algorithm is evaluated with mean squared imputation error and mean absolute imputation error. The mean squared error ( ), parameter estimates with their standard errors and confidence intervals are also computed to compare the performance in the regression context. The proposed algorithm is observed to be a good competitor to the existing algorithms, with smaller mean squared imputation error, mean absolute imputation error and mean squared error. The algorithm's performance becomes considerably better than that of the existing algorithms with increasing number of covariates, especially when the number of predictors is close to or even greater than the sample size. Two real-life datasets are also used to examine the performance of the proposed algorithm using simulations.

Improved inference for the generalized Pareto distribution

The generalized Pareto distribution is commonly used to model exceedances over a threshold. In this paper, we obtain adjustments to the generalized Pareto profile likelihood function using the likelihood function modifications proposed by Barndorff-Nielsen (Biometrika 70 (1983) 343–365), Cox and Reid (J. R. Stat. Soc. Ser. B. Stat. Methodol. 55 (1993) 467–471), Fraser and Reid (Utilitas Mathematica 47 (1995) 33–53), Fraser, Reid and Wu (Biometrika 86 (1999) 249–264) and Severini (Biometrika 86 (1999) 235–247). We consider inference on the generalized Pareto distribution shape parameter, the scale parameter being a nuisance parameter. Bootstrap-based testing inference is also considered. Monte Carlo simulation results on the finite sample performances of the usual profile maximum likelihood estimator and profile likelihood ratio test and also their modified versions is presented and discussed. The numerical evidence favors the modified profile maximum likelihood estimators and tests we propose. Finally, we consider two real datasets as illustrations.

The bivariate alpha-skew-normal distribution

ABSTRACTIn this paper, we propose a new bivariate distribution, namely bivariate alpha-skew-normal distribution. The proposed distribution is very flexible and capable of generalizing the univariate alpha-skew-normal distribution as its marginal component distributions; it features a probability density function with up to two modes and has the bivariate normal distribution as a special case. The joint moment generating function as well as the main moments are provided. Inference is based on a usual maximum-likelihood estimation approach. The asymptotic properties of the maximum-likelihood estimates are verified in light of a simulation study. The usefulness of the new model is illustrated in a real benchmark data.

Combining reliability functions of a Weibull distribution

In this article, a large sample pooling procedure is considered for the reliability function of a Weibull distribution. Asymptotic properties of shrinkage estimation procedures based on the preliminary test are developed. It is shown that the proposed estimator has substantially smaller asymptoticmean squared error (AMSE) than the usual maximumlikelihood (ML) estimator inmost of the parameter space. Analytic AMSE expressions of the proposed estimators are obtained and the dominance picture of the estimators is presented by comparing them. It is shown that the suggested estimators yield a wider dominance range over theML estimator than the usual pretest estimator and give a meaningful size of the pretest. To appraise the small sample performance of the estimators, detailed Monte-Carlo simulation studies are also carried out.

An algorithm for estimating Box–Cox transformation parameter in ANOVA

ABSTRACTIn this study, we construct a feasible region, in which we maximize the likelihood function, by using Shapiro–Wilk and Bartlett's test statistics to obtain Box–Cox power transformation parameter for solving the issues of non-normality and/or heterogeneity of variances in analysis of variance (ANOVA). Simulation studies illustrate that the proposed approach is more successful in attaining normality and variance stabilization, and is at least as good as the usual maximum likelihood estimation (MLE) in estimating the transformation parameter for different conditions. Our proposed method is illustrated on two real-life datasets. Moreover, the proposed algorithm is released under R package AID under the name of “boxcoxfr” for implementation.

A note on comparison and improvement of estimators based on likelihood

The problem of estimation of a parameter of interest in the presence of a nuisance parameter, which is either location or scale, is considered. Three estimators are taken into account: usual maximum likelihood (ML) estimator, maximum integrated likelihood estimator and the bias-corrected ML estimator. General results on comparison of these estimators w.r.t. the second-order risk based on the mean-squared error are obtained. Possible improvements of basic estimators via the notion of admissibility and methodology given in Ghosh and Sinha [A necessary and sufficient condition for second order admissibility with applications to Berkson's bioassay problem. Ann Stat. 1981;9(6):1334–1338] are considered. In the recent paper by Tanaka et al. [On improved estimation of a gamma shape parameter. Statistics. 2014; doi:10.1080/02331888.2014.915842], this problem was considered for estimating the shape parameter of gamma distribution. Here, we perform more accurate comparison of estimators for this case as well as for some other cases.

Shrinkage Estimation of Log-odds Ratios for Comparing Mobility Tables

Statistical analysis of mobility tables has long played a pivotal role in comparative stratification research. This article proposes a shrinkage estimator of the log-odds ratio for comparing mobility tables. Building on an empirical Bayes framework, the shrinkage estimator improves estimation efficiency by “borrowing strength” across multiple tables while placing no restrictions on the pattern of association within tables. Numerical simulation shows that the shrinkage estimator outperforms the usual maximum likelihood estimator (MLE) in both the total squared error and the correlation with the true values. Moreover, the benefits of the shrinkage estimator relative to the MLE depend on both the variation in the true log-odds ratio and the variation in sample size among mobility regimes. To illustrate the effects of shrinkage, the author contrasts the shrinkage estimates with the usual estimates for the mobility data assembled by Hazelrigg and Garnier for 16 countries in the 1960s and 1970s. For mobility tables with more than two categories, the shrinkage estimates of log-odds ratios can also be used to calculate summary measures of association that are based on aggregations of log-odds ratios. Specifically, the author constructs an adjusted estimator of the Altham index and, with a set of calibrated simulations, demonstrates its usefulness in enhancing both the precision of individual estimates and the accuracy of cross-table comparisons. Finally, using two real data sets, the author shows that in gauging the overall degree of social fluidity, the adjusted estimator of the Altham index agrees more closely with results from the Unidiff model than does the direct estimator of the Altham index.

Correcting for bias in the selection and validation of informative diagnostic tests.

When developing a new diagnostic test for a disease, there are often multiple candidate classifiers to choose from, and it is unclear if any will offer an improvement in performance compared with current technology. A two-stage design can be used to select a promising classifier (if one exists) in stage one for definitive validation in stage two. However, estimating the true properties of the chosen classifier is complicated by the first stage selection rules. In particular, the usual maximum likelihood estimator (MLE) that combines data from both stages will be biased high. Consequently, confidence intervals and p-values flowing from the MLE will also be incorrect. Building on the results of Pepe et al. (SIM 28:762–779), we derive the most efficient conditionally unbiased estimator and exact confidence intervals for a classifier's sensitivity in a two-stage design with arbitrary selection rules; the condition being that the trial proceeds to the validation stage. We apply our estimation strategy to data from a recent family history screening tool validation study by Walter et al. (BJGP 63:393–400) and are able to identify and successfully adjust for bias in the tool's estimated sensitivity to detect those at higher risk of breast cancer. © 2015 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

Mathematical probit and logistic mortality models of the Khapra beetle fumigated with plant essential oils.

In the current study, probit and logistic models were employed to fit experimental mortality data of the Khapra beetle, Trogoderma granarium (Everts) (Coleoptera: Dermestidae), when fumigated with three plant oils of the gens Achillea. A generalized inverse matrix technique was used to estimate the mortality model parameters instead of the usual statistical iterative maximum likelihood estimation. As this technique needs to perturb the observed mortality proportions if the proportions include 0 or 1, the optimal perturbation in terms of minimum least squares (L2) error was also determined. According to our results, it was better to log-transform concentration and time as explanatory variables in modeling mortality of the test insect. Estimated data using the probit model were more accurate in terms of L2 errors, than the logistic one. Results of the predicted mortality revealed also that extending the fumigation period could be an effective control strategy, even, at lower concentrations. Results could help in using a relatively safe and effective strategy for the control of this serious pest using alternative control strategy to reduce the health and environmental drawbacks resulted from the excessive reliance on the broadly toxic chemical pesticides and in order to contribute safeguard world-wide grain supplies.

Parametric Estimation of Menarcheal Age Distribution Based on Recall Data

AbstractMenarche, the onset of menstruation, is an important maturational event of female childhood. Most of the studies of age at menarche make use of dichotomous (status quo) data. More information can be harnessed from recall data, but such data are often censored in a informative way. We show that the usual maximum likelihood estimator based on interval censored data, which ignores the informative nature of censoring, can be biased and inconsistent. We propose a parametric estimator of the menarcheal age distribution on the basis of a realistic model of the recall phenomenon. We identify the additional information contained in the recall data and demonstrate theoretically as well as through simulations the advantage of the maximum likelihood estimator based on recall data over that based on status quo data.

Maximum integrated likelihood estimator of the interest parameter when the nuisance parameter is location or scale

The problem of estimation of an interest parameter in the presence of a nuisance parameter, which is either location or scale, is studied. Two estimators are considered: the usual maximum likelihood estimator and the estimator based on maximization of the integrated likelihood function. The estimators are compared, asymptotically, with respect to the bias and with respect to the mean squared error. The examples are given.

Increased Statistical Efficiency in a Lognormal Mean Model

Within the context of clinical and other scientific research, a substantial need exists for an accurate determination of the point estimate in a lognormal mean model, given that highly skewed data are often present. As such, logarithmic transformations are often advocated to achieve the assumptions of parametric statistical inference. Despite this, existing approaches that utilize only a sample’s mean and variance may not necessarily yield the most efficient estimator. The current investigation developed and tested an improved efficient point estimator for a lognormal mean by capturing more complete information via the sample’s coefficient of variation. Results of an empirical simulation study across varying sample sizes and population standard deviations indicated relative improvements in efficiency of up to 129.47 percent compared to the usual maximum likelihood estimator and up to 21.33 absolute percentage points above the efficient estimator presented by Shen and colleagues (2006). The relative efficiency of the proposed estimator increased particularly as a function of decreasing sample size and increasing population standard deviation.

Shrinkage Estimation of the Memory Parameter in Stationary Gaussian Processes

The correct and efficient estimation of memory parameters in a stationary Gaussian processes is an important issue, since otherwise, forecasts based on the resulting time series would be misleading. On the other hand, if the memory parameters are suspected to fall in a smaller subspace through some hypothesis restrictions, it becomes a hard decision whether to use estimators based on the restricted spaces or to use unrestricted estimators over the full parameter space. In this article, we propose James-Stein-type estimators of the memory parameters of a stationary Gaussian times series process, which can efficiently incorporate the hypothetical restrictions. We show theoretically that the proposed estimators are more efficient than the usual unrestricted maximum likelihood estimators over the entire parameter space.

Modelling mortality of a stored grain insect pest with fumigation: Probit, logistic or Cauchy model?

Computer simulation models can provide a relatively fast, safe and inexpensive means to judge and weigh the merits of various pest control management options. However, the usefulness of such simulation models relies on the accurate estimation of important model parameters, such as the pest mortality under different treatments and conditions. Recently, an individual-based simulation model of population dynamics and resistance evolution has been developed for the stored grain insect pest Rhyzopertha dominica, based on experimental results showing that alleles at two different loci are involved in resistance to the grain fumigant phosphine. In this paper, we describe how we used three generalized linear models, probit, logistic and Cauchy models, each employing two- and four-parameter sub-models, to fit experimental data sets for five genotypes for which detailed mortality data was already available. Instead of the usual statistical iterative maximum likelihood estimation, a direct algebraic approach, generalized inverse matrix technique, was used to estimate the mortality model parameters. As this technique needs to perturb the observed mortality proportions if the proportions include 0 or 1, a golden section search approach was used to find the optimal perturbation in terms of minimum least squares (L2) error. The results show that the estimates using the probit model were the most accurate in terms of L2 errors between observed and predicted mortality values. These errors with the probit model ranged from 0.049% to 5.3%, from 0.381% to 8.1% with the logistic model and from 8.3% to 48.2% with the Cauchy model. Meanwhile, the generalized inverse matrix technique achieved similar results to the maximum likelihood estimation ones, but is less time consuming and computationally demanding. We also describe how we constructed a two-parameter model to estimate the mortalities for each of the remaining four genotypes based on realistic genetic assumptions.

BAYESIAN ESTIMATION OF POPULATION PROPORTION IN KIM AND WARDE MIXED RANDOMIZED RESPONSE TECHNIQUE

In this study, we have developed the Bayesian estimator of the population proportion of a sensitive characteristic when data are obtained through the Randomized Response Technique (RRT) proposed by Kim and Warde (2005). Superiority of the Bayesian estimators is established for a wide range of the values of the population proportion using simple Beta prior information. It is observed that Bayesian estimators are better than the usual Maximum Likelihood Estimator (MLE) for small as well as moderate samples. The Proposed estimator is also compared with the Warner (1965), Kim and warde (2005) and Kim et al. (2006) estimators.

Ridge estimation for multinomial logit models with symmetric side constraints

In multinomial logit models, the identifiability of parameter estimates is typically obtained by side constraints that specify one of the response categories as reference category. When parameters are penalized, shrinkage of estimates should not depend on the reference category. In this paper we investigate ridge regression for the multinomial logit model with symmetric side constraints, which yields parameter estimates that are independent of the reference category. In simulation studies the results are compared with the usual maximum likelihood estimates and an application to real data is given.

A theoretical view of the envelope model for multivariate linear regression as response dimension reduction

The envelope model recently developed for the classical multivariate linear regression have potential gain in efficiency in estimating unknown parameters over usual maximum likelihood estimation. In this paper, we theoretically investigate the envelope model as dimension reduction for response variables and connect them to existing methods.

Shrinkage estimation with a matrix loss function

Consider estimating an n×p matrix of means Θ, say, from an n×p matrix of observations X, where the elements of X are assumed to be independently normally distributed with E(xij)=θij and constant variance, and where the performance of an estimator is judged using a p×p matrix quadratic error loss function. A matrix version of the James-Stein estimator is proposed, depending on a tuning constant a. It is shown to dominate the usual maximum likelihood estimator for some choices of a when n≥3. This result also extends to other shrinkage estimators and settings.

Bayesian Estimation Using Warner's Randomized Response Model through Simple and Mixture Prior Distributions

Bayesian estimation of population proportion of a sensitive characteristic is proposed by adopting a simple beta distribution and a mixture of Beta distributions as quantification of prior information using simple random sampling with replacement. In the sequel application of the stratified random sampling is also studied in Bayesian scenario. It is assumed that data are collected through Warner (1965) randomized response technique. To study the performance of Bayesian estimators we have used Mean Squared Error (MSE) and/or Relative Efficiency (RE) as performance criterion. Further, comparison of the suggested estimator is made with Kim et al. (2006) stratified estimator and usual maximum likelihood estimator in case of stratified random sampling. It is observed that unlike the moment and maximum likelihood methods, proposed Bayesian estimation method is free of the problems of having an estimate of population proportion outside the interval (0, 1) and large variance when the sample proportion of yes responses is very low or very high.

Being Naive about Naive Diversification: Can Investment Theory be Consistently Useful

The modern portfolio theory pioneered by Markowitz (1952) is widely used in practice and taught in MBA texts. DeMiguel, Garlappi and Uppal (2007), however, show that, due to estimation errors, existing theory-based portfolio strategies are not as good as we once thought, and the estimation window needed for them to beat the naive $1/N$ strategy (that invests equally across N risky assets) is 'around 3000 months for a portfolio with 25 assets and about 6000 months for a portfolio with 50 assets.' In this paper, we modify the modern portfolio theory to account for estimation errors, so that the theory becomes more relevant in practice to yield positive gains over the naive 1/N strategy under realistic estimation windows. In particular, we provide new portfolio strategies that not only perform as well as the 1/N strategy in an exact one-factor model that favors the 1/N, but also outperform it substantially in a one-factor model with mispricing, in multi-factor models with and without mispricing, and in models calibrated from real data without any factor structures. We also find that the usual maximum likelihood (ML) estimator of the true portfolio rule can have Sharpe ratios higher than the 1/N in many cases, and hence, if one is concerned only about Sharpe ratios, the ML estimator is not as bad as one might have once believed.