Constrained Maximum Likelihood Estimation Research Articles

A comparison of nonparametric shape constrained bioassay estimators

Abstract The nonparametric Bayesian shape-constrained bioassay estimator of Ramgopal et al. [Ramgopal, P., Laud, P., Smith, A.F.M., 1993. Nonparametric Bayesian bioassay with prior constraints on the shape of the potency curve. Biometrika 80 (3), 489–498.] is compared with the shape constrained maximum likelihood estimator through simulations. The purpose is to illustrate that, in the absence of prior information other than the shape constraints, the less computationally intensive MLE might be an acceptable alternative. Data are simulated from an underlying potency function and both estimates are computed with the assumption that the shape is known (either concave or convex–concave, both nondecreasing over the support). Several different sampling functions and sample sizes are chosen to investigate the behavior of the estimators in varying conditions. An efficient algorithm for computing the shape-constrained MLE is presented. A theorem provides the theoretical justification for this new algorithm.

Non-Parametric Likelihood Enhancements to Parametric Graduations

ABSTRACTParametric graduation may fail to achieve satisfactory results without overparameterisation. Whittaker-Henderson graduation tends to constrain the graduated values towards a low-order polynomial. Non-parametric methods do not generally make direct use of true likelihood functions. This paper suggests a method of enhancing the likelihood of a parametric graduation by means of non-parametric methods, thus reducing the disadvantages of both methods.The parametric graduation is taken to be ideally smooth by definition and is adjusted by using constrained maximum likelihood estimation to obtain better fidelity to the experience. The constraint imposes a minimum sacrifice of smoothness, in terms of a quantitative smoothness criterion, from the initial ideal. The method is not entirely objective in that, in some cases, professional judgement is required in order to assess the degree of smoothness that can be imposed. In other cases the method provides an objective optimum. In either case, by quantifying the degrees of departure from perfect fidelity and from ideal smoothness, the suggested method provides useful and theoretically sound criteria for the purposes of the optimisation process. In particular, by inverting the parametric graduation formula for the purposes of defining the smoothness criterion, the method ensures that the smoothness criterion is consistent over the whole age range, thus resolving the main objection to non-parametric graduation.The method is applied to the 1979-82 experience for life office pensioners in the United Kingdom with positive results.

Theory and method for constrained estimation in structural equation models with incomplete data

The statistical properties and a practical procedure for constrained maximum likelihood estimation in covariance structure analysis with incomplete data are studied. We show that the constrained maximum likelihood estimator possesses nice statistical properties, such as, consistency and normality, and we provide tests of the model structure and of a subset of restrictions. Computationally, a so-called restricted EM algorithm is proposed to obtain the constrained ML estimates. A simulated incomplete data set is used as an illustrative example. The degrees of freedom of the model change substantially with increases in the number of missing data response patterns. Possible effects of missing data on power, and of violation of distributional assumptions, are discussed.

Maximum likelihood track-before-detect matched-field beamforming with SWELLEX data

Moving source localization methods are seriously limited at low signal-to-noise ratios because the signal energy becomes spatially spread over long observation times. In this paper, the integration time available to matched-field beamformers is extended by incorporating a priori statistical knowledge of the source dynamics. This is accomplished by first showing that the well-known minimum variance beamformer output power is essentially the log-likelihood function with respect to source location under a model consisting of a single target in an unknown but structured noise field. This likelihood function for a single frame of data may be combined with the conditional distribution of the current target position given its previous location to obtain a maximum-likelihood (ML) track-before-detect processor. In cases where the sample covariance matrix is ill-conditioned, derivation of a constrained ML estimate assuming known diffuse noise level is given. Results are presented using vertical hydrophone array data for an underwater acoustic source off the coast of southern California. Comparisons with conventional MV matched-field beamforming demonstrate that incorporating source dynamics into the processor improves its ability to maintain the target track even in the presence of interference.

Log-linear models for a binary response with nonignorable nonresponse

We focus on the analysis of contingency tables that contain both completely and partially cross-classified data where one of the variables is a binary response variable subject to nonignorable nonresponse and the other variables are always observed. Log-linear models can be used to adjust for nonresponse in an augmented table where one index corresponds to whether or not the subject is a respondent. The maximum likelihood estimators (MLEs) of the expected cell frequencies in the augmented table can be obtained iteratively using the EM algorithm. ML estimation may yield boundary estimators for the expected frequencies of the nonrespondents' cells. As a result, the odds ratio which represents the nonresponse mechanism is estimated either as 0 or ∞ and the estimate of the corresponding log-linear model parameter is infinite. Since such an extreme estimate is not anticipated, it is desirable to use a constraint which limits the range of the odds ratio. We propose a constrained maximum likelihood estimator which maximizes the likelihood function subject to a constraint on the odds ratio. Through a simulation study, the proposed estimators of the expected cell frequencies of the nonrespondents are shown to have smaller mean square errors than those of the MLEs. This estimation procedure is illustrated using polling data in Baker and Laird (1988). In addition, we provide some recommendations on how to choose constraints for practical applications.

Constrained maximum likelihood estimators for superimposed exponential signals

Recently Kundu(1993a) has proposed a non-linear eigenvalue method for finding the maximum likelihood estimators (MLE) of the parameters of undamped exponential signals.It is known to perform better than the previously existing methods like FBLP of Tufts and Kumaresan or Pisarenko's method (Pisarenko;),in the sense of lower mean squared errors.The solution in general depends on the roots of a polynomial equation.It is observed that the coefficients of the polynomial exhibit a certain symmetry.Since it is known (Crowder; 1984) that the MLE with constraints is more efficient than the unconstrained MLE,modified maximum likelihood method.

Tests of Independence, Treatment Heterogeneity Dose-Related Trend with Exchangeable Binary Data

Abstract Existing methods of testing for treatment effects for clustered binary data include the beta-binomial, quasilikelihood GEE procedures. All of these methods revolve around the mean response and the second-order correlation. However, these two parameters alone do not fully determine the effect of treatment. This article develops nonparametric likelihood ratio procedures to test for independence, heterogeneity dose-related trend in dose—response studies involving exchangeable binary data. The hypotheses of independence, heterogeneity trend are expressed in terms of joint probabilities of similar responses among cluster mates. Constrained maximum likelihood estimates of these probabilities are computed and used to construct test statistics. Unlike the test statistics for independence and heterogeneity, the asymptotic distribution of the likelihood ratio test for trend is not exactly a chi-square. However, an upper bound of its p value is obtained by using a chi-squared distribution. A set of clustere...

Testing for ordered group effects with repeated measurements

SUMMARY In the repeated measures experiment, testing procedures for ordered group effects are proposed using the likelihood ratio principle. The constrained maximum likelihood estimators of ordered group effects are approximated by the isotonic regression of the unconstrained maximum likelihood estimators. Based on these constrained maximum likelihood estimators, test statistics are constructed and are used for detecting ordered group effects. The limiting distributions of the proposed test statistics are shown to be mixtures of chi-squared distributions. A simulation study shows that the proposed tests have improved powers over the usual chi-squared statistic for testing equality of group effects. A real data set is analysed to illustrate the proposed statistics.

On information matrices in case-control studies

In this paper we investigate the properties of the information matrices and asymptotic covariances of the constrained maximum likelihood estimates (MLE) in case-control studies. Since in a case-control study samples are stratified on case-control status, the sampling scheme is different from the usual simple random sampling. It is shown that the relationship between the information and asymptotic covariances of the MLEs in the usual unconstrained likelihood theory applies to case-control studies. Furthermore, this property holds in nondifferential measurement error models.

Explicit solutions for constrained maximum likelihood estimators in survival/sacrifice experiments

SUMMARY Regulatory agencies routinely base conclusions regarding carcinogenicity of compounds tested in long-term animal studies on tests which are known to be biased under conditions of treatment lethality and tumour lethality. The recognition of these biases has led to a variety of proposed approaches which rely on survival and sacrifice data. Williams & Portier (1992) derived analytic expressions for maximum likelihood estimators of the tumour incidence rate and discrete death rates based on a discrete multistate model. A disadvantage of the proposed estimators was that they sometimes resulted in negative estimates of the tumour incidence rate. In this paper, explicit solutions for constrained estimators are derived under the imposition of boundary conditions for a study design with one or two interim sacrifices and a terminal sacrifice. For study designs with more than two interim sacrifices, alternative estimators of the tumour incidence rate and discrete death rates are developed heuristically by pooling data together from adjacent

On the Fitting of Generalized Linear Models with Nonnegativity Parameter Constraints

SUMMARY We consider the problem of finding maximum likelihood estimates of a generalized linear model when some or all of the regression parameters are constrained to be nonnegative. The Kuhn-Tucker conditions of nonlinear programming can be used to characterize the solution of this constrained estimation problem when the maximum likelihood estimates exist and are unique. For the case of a generalized linear model with nonnegativity parameter constraints, the Kuhn-Tucker conditions are derived and utilized to provide a stopping rule for search algorithms for the constrained maximum likelihood estimates. Two examples are discussed.

Modified lagrange multiplier tests for problems with one-sided alternatives

Abstract A modified Lagrange multiplier test statistic is proposed which takes explicit account of the one-sided nature of the alternative in problems where the null hypothesis specifies that the true value of the parameter vector lies on the boundary of the parameter space. Computation of this statistic requires only the constrained maximum likelihood estimator. Conditions for the consistency of tests based on this statistic are examined and it is shown that the distribution of the statistic is not affected of nuisance parameters are allowed to lie on the boundary of the parameter space.

Algebraic equivalences among Wald, LM and Hausman's tests in the linear regression model

Under certain conditions, the Hausman's statistic is identical to the Wald statistic, if we use the unconstrained maximum likelihood estimate for the variance of disturbances, while it becomes identical to the LM statistic, if we use the constrained maximum likelihood estimate for the variance.

On constrained maximum likelihood estimation with non-i.i.d. observations

Consistency and asymptotic normality of the m.l.e. are examined in the non-i.i.d. case when the parameters are constrained. Inequality constraints are considered as an application.

The likelihood ratio test statistic of mean-variance efficiency without a riskless asset

Abstract The question whether a given porfolio is mean-variance efficient is a basic problem of investment analysis. Mean-variance efficiency is also the basis of the Capital Asset Pricing Model. This paper presents the explicit form of the likelihood ratio test of the hypothesis that a given portfolio, or a particular market index, is ex-ante mean-variance efficient in the case where there is no riskless asset. Geometric relations are illustrated to provide intuition about the constrained maximum likelihood estimators and the test statistic, and two simple economic interpretations of the test are given.

A Truncated Maximum Likelihood Estimator of a Constrained Bivariate Linear Regression Coefficient

Abstract In a bivariate linear regression model, the constrained maximum likelihood estimator (MLE) of a regression coefficient usually has smaller variance than the unconstrained MLE. This situation can be reversed if both the sample size n and the correlation coefficient ρ of disturbances between two regression equations are small. That is, when both n and ρ are small, the variance of the unconstrained MLE is smaller than the constrained MLE. In this article, a truncated MLE is considered for the justification of using either the constrained or the unconstrained MLE as an appropriate estimator for the regression coefficient.

A Truncated Maximum Likelihood Estimator of a Constrained Bivariate Linear Regression Coefficient

Abstract In a bivariate linear regression model, the constrained maximum likelihood estimator (MLE) of a regression coefficient usually has smaller variance than the unconstrained MLE. This situation can be reversed if both the sample size n and the correlation coefficient ρ of disturbances between two regression equations are small. That is, when both n and ρ are small, the variance of the unconstrained MLE is smaller than the constrained MLE. In this article, a truncated MLE is considered for the justification of using either the constrained or the unconstrained MLE as an appropriate estimator for the regression coefficient.

Inequality Constraints, Multicollinearity and Models of Police Expenditure

Inequality constraints have been used very infrequently in the analysis of linear statistical models. This is not due to an inability to construct estimators. Rather, it is a result of an inability to say very much about the properties of the inequality constrained estimator. Hanson [15] proved the existence of inequality constrained maximum likelihood estimators. A year later Judge and Takayama [17] demonstrated the value of quadratic programming for obtaining estimates of the parameters in a model constrained by inequalities. More recently, Liew [20; 21] and Klemm and Sposito [19] have suggested closed form estimates. The work by Liew makes some reference to the properties of the inequality constrained least squares (ICLS) estimator. A more formal effort to determine the properties of the ICLS estimator was made by Lovell and Prescott [23]. Their paper considered only the case of one location parameter being restricted to either the positive or negative half of the real line. In a more recent paper Wardle [32] has considered a Bayesian approach to the problem of determining the small sample properties of the ICLS estimator. Our more classical results are a confirmation of his work.

Statistical analysis of ECC bypass data using a nonlinear constrained maximum likelihood estimation technique

Recently much attention has been focused on the problem of developing models which describe the partial penetration of ECC water into the lower plenum of a pressurized water reactor (PWR) during a postulated loss-of-coolant accident (LOCA). To date model parameters have been estimated using inappropriate classical statistical techniques which assume that only one of the experimental variables is subject to error and that it is functionally related to a set of independent experimental variables which are not subject to error. A nonlinear constrained maximum likelihood estimation (MLE) technique is discussed and illustrated which incorporates several nonstandard features, including errors in all variables, into the estimation procedure. This technique places the statistical analysis of ECC bypass data on a solid theoretical foundation. The technique is illustrated by sample calculations of estimates of the parameters in a modified form of the Wallis flooding correlation.

Constrained estimation in covariance structure analysis

SUMMARY In covariance structure analysis, the method of weighted least squares for estimating parameters that are subject to functional constraints is developed. Statistical properties of the estimator are studied and a goodness-of-fit statistic is presented. Based on the penalty function technique, an algorithm is developed. This algorithm is able to produce not only the constrained weighted least squares estimator but also the constrained maximum likelihood estimator, if we choose an appropriate weight matrix iteratively. The feasibility of the proposed algorithm is demonstrated by an example in factor analysis.