Unrestricted Maximum Likelihood Estimators Research Articles

Estimation of Heterogeneous Agent Models: A Likelihood Approach

Using a Bewley‐Hugget‐Aiyagari model we show how to use the Fokker‐Planck equation for likelihood inference in heterogeneous agent (HA) models. We study the finite sample properties of the maximum likelihood estimator (MLE) in Monte Carlo experiments using cross‐sectional data on wealth and income. We use the Kullback–Leibler divergence to investigate identification problems that may affect inference. Unrestricted MLE leads to considerable biases of some parameters. Calibrating weakly identified parameters is shown to be useful to pin down the remaining structural parameters. We illustrate our approach by estimating the model for the US economy using the Survey of Consumer Finances.

Exact likelihood-ratio tests for joint type-II censored exponential data

ABSTRACTSeveral results based on the likelihood-ratio test have been discussed in the literature for testing the scale parameters of two exponential distributions under complete and different censored forms of data, many of which are asymptotic in nature. However, exact likelihood-ratio tests have been derived only for the case of complete and independent Type-II censored samples. In this paper, we first develop exact inference using the likelihood-ratio test based on two exponential samples under joint Type-II censoring. We then obtain the exact critical value as well as the power function by using both unrestricted and restricted maximum likelihood estimators (MLEs) in the likelihood-ratio test statistics. Based on these results, we discuss the procedure for designing an optimal life-testing experiment in this set-up. Finally, some numerical examples are presented for illustrating all the results developed here.

Estimation of Heterogeneous Agent Models: A Likelihood Approach

We study the statistical properties of heterogeneous agent models. Using a Bewley-Hugget-Aiyagari model we compute the density function of wealth and income and use it for likelihood inference. We study the finite sample properties of the maximum likelihood estimator (MLE) using Monte Carlo experiments on artificial cross-sections of wealth and income. We propose to use the Kullback-Leibler divergence to investigate identification problems that may affect inference. Our results suggest that the unrestricted MLE leads to considerable biases of some parameters. Calibrating weakly identified parameters allows to pin down the other unidentified parameter without compromising the estimation of the remaining parameters. We illustrate our approach by estimating the model for the U.S. economy using wealth and income data from the Survey of Consumer Finances.

Stein-type shrinkage estimators in gamma regression model with application to prostate cancer data.

Gamma regression is applied in several areas such as life testing, forecasting cancer incidences, genomics, rainfall prediction, experimental designs, and quality control. Gamma regression models allow for a monotone and no constant hazard in survival models. Owing to the broad applicability of gamma regression, we propose some novel and improved methods to estimate the coefficients of gamma regression model. We combine the unrestricted maximum likelihood (ML) estimators and the estimators that are restricted by linear hypothesis, and we present Stein-type shrinkage estimators (SEs). We then develop an asymptotic theory for SEs and obtain their asymptotic quadratic risks. In addition, we conduct Monte Carlo simulations to study the performance of the estimators in terms of their simulated relative efficiencies. It is evident from our studies that the proposed SEs outperform the usual ML estimators. Furthermore, some tabular and graphical representations are given as proofs of our assertions. This study is finally ended by appraising the performance of our estimators for a real prostate cancer data.

Exact Likelihood-Ratio Tests for a Simple Step-Stress Cumulative Exposure Model with Censored Exponential Data

An accelerated life test (ALT) is commonly used in reliability studies of high quality products. A step-stress experiment is one such ALT that is often used in practice. In this paper, we develop exact inference based on likelihood-ratio test for a simple step-stress model with exponential lifetimes under Type-II censoring. We consider both unrestricted and order-restricted maximum likelihood estimators (MLEs) for this purpose. We also derive the exact power function of this test, which we then use for designing an optimal step-stress testing experiment. By using a similar approach, we extend these results to other forms of censoring. Monte Carlo simulations are then carried out to evaluate the performance of the inferential methods developed here. Finally, an illustrative example is presented for demonstrating all the inferential results.

Shrinkage estimation in linear mixed models for longitudinal data

This paper is concerned with the selection and estimation of fixed effects in linear mixed models while the random effects are treated as nuisance parameters. We propose the non-penalty James–Stein shrinkage and pretest estimation methods based on linear mixed models for longitudinal data when some of the fixed effect parameters are under a linear restriction. We establish the asymptotic distributional biases and risks of the proposed estimators, and investigate their relative performance with respect to the unrestricted maximum likelihood estimator (UE). Furthermore, we investigate the penalty (LASSO and adaptive LASSO) estimation methods and compare their relative performance with the non-penalty pretest and shrinkage estimators. A simulation study for various combinations of the inactive covariates shows that the shrinkage estimators perform better than the penalty estimators in certain parts of the parameter space. This particularly happens when there are many inactive covariates in the model. It also shows that the pretest, shrinkage, and penalty estimators all outperform the UE. We further illustrate the proposed procedures via a real data example.

Estimation of Order Restricted Normal Means when the Variances Are Unknown and Unequal

<p dir="ltr">In the present paper, two normal distributions with parameters <span style="font-family: Calibri;"><span style="font-size: medium;">μ</span><sub><span style="font-size: small;">i </span></sub></span>and <span style="font-family: Calibri;"><span style="font-size: medium;">σ</span><span style="font-size: small;"><sub>i</sub><sup>2 </sup></span></span>where there is an order restriction on the means when the variances are unknown and unequal are considered. Under the squared error loss function, a necessary and sufficient condition for the plug-in estimators to improve upon the unrestricted maximum likelihood estimators uniformly is given. Also under the modified Pitman nearness criterion; a class of estimators is considered that reduce to the estimators of a common mean when the unbiased estimators violate the order restriction. It is shown that the most critical case for uniform improvement with regard to the unbiased estimators is the one when two means are equal. To illustrate the results, two numerical examples are presented.</p>

Model selection and post estimation based on a pretest for logistic regression models

ABSTRACTThis article addresses the problem of parameter estimation of the logistic regression model under subspace information via linear shrinkage, pretest, and shrinkage pretest estimators along with the traditional unrestricted maximum likelihood estimator and restricted estimator. We developed an asymptotic theory for the linear shrinkage and pretest estimators and compared their relative performance using the notion of asymptotic distributional bias and asymptotic quadratic risk. The analytical results demonstrated that the proposed estimation strategies outperformed the classical estimation strategies in a meaningful parameter space. Detailed Monte-Carlo simulation studies were conducted for different combinations and the performance of each estimation method was evaluated in terms of simulated relative efficiency. The results of the simulation study were in strong agreement with the asymptotic analytical findings. Two real-data examples are also given to appraise the performance of the estimators.

Design and analysis of three-arm trials with negative binomially distributed endpoints.

A three-arm clinical trial design with an experimental treatment, an active control, and a placebo control, commonly referred to as the gold standard design, enables testing of non-inferiority or superiority of the experimental treatment compared with the active control. In this paper, we propose methods for designing and analyzing three-arm trials with negative binomially distributed endpoints. In particular, we develop a Wald-type test with a restricted maximum-likelihood variance estimator for testing non-inferiority or superiority. For this test, sample size and power formulas as well as optimal sample size allocations will be derived. The performance of the proposed test will be assessed in an extensive simulation study with regard to type I error rate, power, sample size, and sample size allocation. For the purpose of comparison, Wald-type statistics with a sample variance estimator and an unrestricted maximum-likelihood estimator are included in the simulation study. We found that the proposed Wald-type test with a restricted variance estimator performed well across the considered scenarios and is therefore recommended for application in clinical trials. The methods proposed are motivated and illustrated by a recent clinical trial in multiple sclerosis. The R package ThreeArmedTrials, which implements the methods discussed in this paper, is available on CRAN.

On the Properties of Estimates of Monotonic Mean Vectors for Multivariate Normal Distributions

Problems concerning estimation of parameters and determination the statistic, when it is known a priori that some of these parameters are subject to certain order restrictions, are of considerable interest. In the present paper, we consider the estimators of the monotonic mean vectors for two dimensional normal distributions and compare those with the unrestricted maximum likelihood estimators under two different cases. One case is that covariance matrices are known, the other one is that covariance matrices are completely unknown and unequal. We show that when the covariance matrices are known, under the squared error loss function which is similar to the mahalanobis distance, the obtained multivariate isotonic regression estimators, motivated by estimators given in Robertson et al. (1988), which are the estimators given by Sasabuchi et al. (1983) and Sasabuchi et al. (1992), have the smaller risk than the unrestricted maximum likelihood estimators uniformly, but when the covariance matrices are unknown and unequal, the estimators have the smaller risk than the unrestricted maximum likelihood estimators only over some special sets which are defined on the covariance matrices. To illustrate the results two numerical examples are presented.

Shrinkage, pretest, and penalty estimators in generalized linear models

We consider estimation in generalized linear models when there are many potential predictors and some of them may not have influence on the response of interest. In the context of two competing models where one model includes all predictors and the other restricts variable coefficients to a candidate linear subspace based on subject matter or prior knowledge, we investigate the relative performances of Stein type shrinkage, pretest, and penalty estimators (L1GLM, adaptive L1GLM, and SCAD) with respect to the unrestricted maximum likelihood estimator (MLE). The asymptotic properties of the pretest and shrinkage estimators including the derivation of asymptotic distributional biases and risks are established. In particular, we give conditions under which the shrinkage estimators are asymptotically more efficient than the unrestricted MLE. A Monte Carlo simulation study shows that the mean squared error (MSE) of an adaptive shrinkage estimator is comparable to the MSE of the penalty estimators in many situations and in particular performs better than the penalty estimators when the dimension of the restricted parameter space is large. The Steinian shrinkage and penalty estimators all improve substantially on the unrestricted MLE. A real data set analysis is also presented to compare the suggested methods.

Preliminary test and Bayes Estimation of a Location Parameter Under Blinex Loss

In this article, the preliminary test estimator is considered under the BLINEX loss function. The problem under consideration is the estimation of the location parameter from a normal distribution. The risk under the null hypothesis for the preliminary test estimator, the exact risk function for restricted maximum likelihood and approximated risk function for the unrestricted maximum likelihood estimator, are derived under BLINEX loss and the different risk structures are compared to one another both analytically and computationally. As a motivation on the use of BLINEX rather than LINEX, the risk for the preliminary test estimator under BLINEX loss is compared to the risk of the preliminary test estimator under LINEX loss and it is shown that the LINEX expected loss is higher than BLINEX expected loss. Furthermore, two feasible Bayes estimators are derived under BLINEX loss, and a feasible Bayes preliminary test estimator is defined and compared to the classical preliminary test estimator.

On shrinkage estimators in matrix variate elliptical models

This paper derives the risk functions of a class of shrinkage estimators for the mean parameter matrix of a matrix variate elliptically contoured distribution. It is showed that the positive rule shrinkage estimator outperformed the shrinkage and unrestricted (maximum likelihood) estimators. To illustrate the findings of the paper, the relative risk functions for different degrees of freedoms are given for a multivariate t distribution. Shrinkage estimators for the matrix variate regression model under matrix normal, matrix t or Pearson VII error distributions would be special cases of this paper.

Shrinkage Estimation of the Memory Parameter in Stationary Gaussian Processes

The correct and efficient estimation of memory parameters in a stationary Gaussian processes is an important issue, since otherwise, forecasts based on the resulting time series would be misleading. On the other hand, if the memory parameters are suspected to fall in a smaller subspace through some hypothesis restrictions, it becomes a hard decision whether to use estimators based on the restricted spaces or to use unrestricted estimators over the full parameter space. In this article, we propose James-Stein-type estimators of the memory parameters of a stationary Gaussian times series process, which can efficiently incorporate the hypothetical restrictions. We show theoretically that the proposed estimators are more efficient than the usual unrestricted maximum likelihood estimators over the entire parameter space.

Sample size calculations for noninferiority trials with Poisson distributed count data

Clinical trials with Poisson distributed count data as the primary outcome are common in various medical areas such as relapse counts in multiple sclerosis trials or the number of attacks in trials for the treatment of migraine. In this article, we present approximate sample size formulae for testing noninferiority using asymptotic tests which are based on restricted or unrestricted maximum likelihood estimators of the Poisson rates. The Poisson outcomes are allowed to be observed for unequal follow-up schemes, and both the situations that the noninferiority margin is expressed in terms of the difference and the ratio are considered. The exact type I error rates and powers of these tests are evaluated and the accuracy of the approximate sample size formulae is examined. The test statistic using the restricted maximum likelihood estimators (for the difference test problem) and the test statistic that is based on the logarithmic transformation and employs the maximum likelihood estimators (for the ratio test problem) show favorable type I error control and can be recommended for practical application. The approximate sample size formulae show high accuracy even for small sample sizes and provide power values identical or close to the aspired ones. The methods are illustrated by a clinical trial example from anesthesia.

On exact computation of minimum sample size for restricted estimation of a binomial parameter

The problem of determining minimum sample size for the estimation of a binomial parameter with prescribed margin of error and confidence level is considered. It is assumed that available auxiliary information allows to restrict the parameter space to some interval whose left boundary is above zero. A range-preserving estimator resulting from the conditional maximization of the likelihood function is considered. A method for exact computation of minimum sample size controlling for the relative error is proposed. Several tables of minimum sample sizes for typical situations are also presented. The range-preserving estimator achieves the same precision and confidence level as the unrestricted maximum likelihood estimator but with a smaller sample.

Estimation of order-restricted means of two normal populations under the LINEX loss function

In this paper, the estimation of order-restricted means of two normal distributions is studied under the LINEX loss function, when the variances are unknown and possibly unequal. Under certain sufficient conditions to be described in this paper, the proposed plug-in estimators uniformly perform better than the existing unrestricted maximum likelihood estimators. Further, the restricted maximum likelihood estimators are compared with the unrestricted maximum likelihood estimators under the Pitman nearness criterion. A simulation study is conducted and it is shown that our proposed plug-in estimators perform better than the unrestricted maximum likelihood estimators. An illustrative example of real data analysis is also given to compare the estimators.

Estimation of means of multivariate normal populations with order restriction

Multivariate isotonic regression theory plays a key role in the field of statistical inference under order restriction for vector valued parameters. Two cases of estimating multivariate normal means under order restricted set are considered. One case is that covariance matrices are known, the other one is that covariance matrices are unknown but are restricted by partial order. This paper shows that when covariance matrices are known, the estimator given by this paper always dominates unrestricted maximum likelihood estimator uniformly, and when covariance matrices are unknown, the plug-in estimator dominates unrestricted maximum likelihood estimator under the order restricted set of covariance matrices. The isotonic regression estimators in this paper are the generalizations of plug-in estimators in unitary case.

The Wald-Type Test of a Normalization of Cointegrating Vectors

This paper proposes a test for the normalization of cointegrating vectors. Our test is constructed using the unrestricted maximum likelihood estimator and then it may be seen as a Wald-type test. The test statistic is shown to be asymptotically bounded above by a chi-square distribution with one degree of freedom (χ 2) and then we can conduct a conservative test using critical values of χ 2.

James–Stein type estimators for ordered normal means

Suppose independent observations X 1, X 2, …, X k are available from k (≥2) normal populations having means θ1, θ2, …, θ k , respectively, and common variance unity. The means are known to be ordered, that is, θ1≤θ2≤···≤θ k . Two commonly used estimators for simultaneous estimation of θ=(θ1, θ2, …, θ k ) are δ MLE, the order restricted maximum likelihood estimator (MLE), and δ p, the generalized Bayes estimator of θ with respect to the uniform prior on the restricted space Ω={θ∈R k : θ1≤θ2≤···≤θ k }, where R k denotes the k-dimensional Euclidean space. Both δ MLE and δ p improve the usual unrestricted MLE X=(X 1, X 2, …, X k ) and are minimax. But δ MLE is inadmissible for k≥2 and δ p is inadmissible for k≥3. However, no dominating estimators are yet known. Using Brown's [Brown, L.D., 1979, A heuristic method for determining admissibility of estimators—with applications. Annals of Statistics, 7, 960–994] heuristic approach for proving admissibility or inadmissibility of estimators, we propose some classes of James–Stein type estimators and show, through a simulation study, that many of these estimators dominate δ p and δ MLE.