Constrained Maximum Likelihood Estimation Research Articles

Testing for generalized linear mixed models with cluster correlated data under linear inequality constraints

AbstractGeneralized linear mixed models (GLMMs) are often used for analyzing cluster correlated data, including longitudinal data and repeated measurements. Full unrestricted maximum likelihood (ML) approaches for inference on both fixed‐and random‐effects parameters in GLMMs have been extensively studied in the literature. However, parameter orderings or constraints may occur naturally in practice, and in such cases, the efficiency of a statistical method is improved by incorporating the parameter constraints into the ML estimation and hypothesis testing. In this paper, inference for GLMMs under linear inequality constraints is considered. The asymptotic properties of the constrained ML estimators and constrained likelihood ratio tests for GLMMs have been studied. Simulations investigated the empirical properties of the constrained ML estimators, compared to their unrestricted counterparts. An application to a recent survey on Canadian youth smoking patterns is also presented. As these survey data exhibit natural parameter orderings, a constrained GLMM has been considered for data analysis.The Canadian Journal of Statistics40: 243–258; 2012 © 2012 Crown in the right of Canada

Sparse Estimation of Conditional Graphical Models With Application to Gene Networks

In many applications the graph structure in a network arises from two sources: intrinsic connections and connections due to external effects. We introduce a sparse estimation procedure for graphical models that is capable of isolating the intrinsic connections by removing the external effects. Technically, this is formulated as a conditional graphical model, in which the external effects are modeled as predictors, and the graph is determined by the conditional precision matrix. We introduce two sparse estimators of this matrix using the reproduced kernel Hilbert space combined with lasso and adaptive lasso. We establish the sparsity, variable selection consistency, oracle property, and the asymptotic distributions of the proposed estimators. We also develop their convergence rate when the dimension of the conditional precision matrix goes to infinity. The methods are compared with sparse estimators for unconditional graphical models, and with the constrained maximum likelihood estimate that assumes a known graph structure. The methods are applied to a genetic data set to construct a gene network conditioning on single-nucleotide polymorphisms.

Constrained nonparametric maximum likelihood estimation of stochastically ordered survivor functions

AbstractThis paper considers estimators of survivor functions subject to a stochastic ordering constraint based on right censored data. We present the constrained nonparametric maximum likelihood estimator (C‐NPMLE) of the survivor functions in one‐and two‐sample settings where the survivor distributions could be discrete or continuous and discuss the non‐uniqueness of the estimators. We also present a computationally efficient algorithm to obtain the C‐NPMLE. To address the possibility of non‐uniqueness of the C‐NPMLE of $S_1(t)$ when $S_1(t)\le S_2(t)$, we consider the maximum C‐NPMLE (MC‐NPMLE) of $S_1(t)$. In the one‐sample case with arbitrary upper bound survivor function $S_2(t)$, we present a novel and efficient algorithm for finding the MC‐NPMLE of $S_1(t)$. Dykstra (1982) also considered constrained nonparametric maximum likelihood estimation for such problems, however, as we show, Dykstra's method has an error and does not always give the C‐NPMLE. We corrected this error and simulation shows improvement in efficiency compared to Dykstra's estimator. Confidence intervals based on bootstrap methods are proposed and consistency of the estimators is proved. Data from a study on larynx cancer are analysed to illustrate the method. The Canadian Journal of Statistics 40: 22–39; 2012 © 2012 Statistical Society of Canada

A two-stage estimation for screening studies using two diagnostic tests with binary disease status verified in test positives only

This article considers the statistical estimation and inference for screening studies in which two binary tests are used for screening with a binary disease status verified only for those subjects with at least one positive test result. The challenge encountered in these studies is the non-identifiability because the disease rate is not identifiable for subjects with negative results from both tests without additional assumptions. Different homogeneous association models have been proposed in the literature to circumvent the non-identifiability problem, which were solved using numerical methods. We propose to formulate the problem as a constrained maximum likelihood estimation (MLE) problem. The MLE has a closed-form in general, which can be solved using a unified two-stage estimation approach. We demonstrate the application of the proposed method on a set of homogeneous association models. The homogeneous association assumptions are generally not testable as all models are saturated. Therefore, we propose an association-ratio plot as a visualization tool for model comparisons. The methods are illustrated through three examples.

Statistical analysis of achievable resolution limit in the near field source localization context

In this fast communication, we derive the statistical resolution limit (SRL), characterizing the minimal parameter separation, to resolve two closely spaced known near-field sources impinging on a linear array. Toward this goal, we conduct on the first-order Taylor expansion of the observation model a Generalized Likelihood Ratio Test (GLRT) based on a Constrained Maximum Likelihood Estimator (CMLE) of the SRL. More precisely, the minimum separation between two near-field sources, that is detectable for a given probability of false alarm and a given probability of detection, is derived herein. Finally, numerical simulations are done to quantify the impact of the array geometry of the signal sources power distribution and of the array aperture on the statistical resolution limit.

A Constrained Maximum Likelihood Approach to the Estimation of Meta-Frontiers

We demonstrate that the estimation of meta-frontier parameters by minimizing the sum of the absolute values or squares of the distances between the meta-frontier and the individual group frontiers, which is an established practice in the literature, is equivalent to maximizing a constrained likelihood function that corresponds to a meta-frontier model treating those distances as a non-negative random variable distributed either exponentially or half-normally. We confirm our claim by empirical results based on the world’s agricultural production data. Our procedure of estimating the meta-frontier parameters by constrained maximum likelihood allows for both the statistical inference on the meta-frontier parameters and on the choice of the most preferred specification. Not only the constrained maximum likelihood estimation allows for the statistical inference that is not straightforward in case of the linear or quadratic programming approach, it also expands the variety of the possible meta-frontiers, each corresponding to a particular distributional assumption on the distances between meta- and the group frontiers.

Inference for comparing a multinomial distribution with a known standard

We propose a new type of stochastic ordering which imposes a monotone tendency in differences between one multinomial probability and a known standard one. An estimation procedure is proposed for the constrained maximum likelihood estimate, and then the asymptotic null distribution is derived for the likelihood ratio test statistic for testing equality of two multinomial distributions against the new stochastic ordering. An alternative test is also discussed based on Neyman modified minimum chi-square estimator. These tests are illustrated with a set of heart disease data.

On the superposition of overlapping Poisson processes and nonparametric estimation of their intensity function

It is shown that, when measuring time in the Total Time on Test scale, the superposition of overlapping realizations of a nonhomogeneous Poisson process is also a Poisson process and is sufficient for inferential purposes. Hence, many nonparametric procedures which are available when only one realization is observed can be easily extended for the overlapping realizations setup. These include, for instance, the constrained maximum likelihood estimator of a monotonic intensity and bootstrap confidence bands based on Kernel estimates of the intensity. The kernel estimate proposed here performs the smoothing in the Total Time on Test scale and it is shown to behave approximately as a usual kernel estimate but with a variable bandwidth which is inversely proportional to the number of realizations at-risk. Likewise, bootstrap samples can be obtained from the single realization of the superimposed process. The methods are illustrated using a real data set consisting of the failure histories of 40 electrical power transformers.

Estimating the dose–toxicity curve in completed phase I studies

While there is an extensive amount of literature covering prospective designs for phase I trials, the methodology for analyzing these data is limited. Prospective designs select the maximum tolerated dose (MTD) through a dose escalation scheme based on a model or on empirical rules. For example, the '3 + 3' method (standard method: SM) assigns patients in cohorts of three and expands to six if one toxicity is observed. It has been shown previously that the MTD chosen by the SM may be low, possibly leading to a non-efficacious dose. Additionally, when deviation from the original trial design occurs, the rules for determining MTD might not be applicable. We hypothesize that a retrospective analysis would suggest an MTD that is more accurate than the one obtained by the SM. A weighted Continual Reassessment Method (CRM-w) has been suggested (Biometrics 2005; 61:749-756) for analyzing data obtained from designs other than the prospective Continual Reassessment Method (CRM). However, CRM-w has not been evaluated in trials that follow the SM design. In this study, we propose a method to analyze completed phase I trials and possibly confirm or amend the recommended phase II dose, based on a constrained maximum likelihood estimation (CMLE). A comparison of CRM-w, isotonic regression, and CMLE in analyzing simulated SM trials shows that CMLE more accurately selects the true MTD than SM, and is better or comparable to isotonic regression and CRM-w. Confidence intervals around the toxicity probabilities at each dose level are estimated using the cumulative toxicity data. A programming code is included.

Partially monotone tensor spline estimation of the joint distribution function with bivariate current status data

The analysis of the joint cumulative distribution function (CDF) with bivariate event time data is a challenging problem both theoretically and numerically. This paper develops a tensor spline-based sieve maximum likelihood estimation method to estimate the joint CDF with bivariate current status data. The I-splines are used to approximate the joint CDF in order to simplify the numerical computation of a constrained maximum likelihood estimation problem. The generalized gradient projection algorithm is used to compute the constrained optimization problem. Based on the properties of B-spline basis functions it is shown that the proposed tensor spline-based nonparametric sieve maximum likelihood estimator is consistent with a rate of convergence potentially better than $n^{1/3}$ under some mild regularity conditions. The simulation studies with moderate sample sizes are carried out to demonstrate that the finite sample performance of the proposed estimator is generally satisfactory.

An MM Algorithm for Constrained Estimation in a Road Safety Measure Modeling

We model the effect of a road safety measure on a set of target sites with a control area for each site, and we suppose that the accident data recorded at each site are classified in different mutually exclusive types. We adopt the before–after technique and we assume that at any one target site the total number of accidents recorded is multinomially distibuted between the periods and types of accidents. In this article, we propose a minorization–majorization (MM) algorithm for obtaining the constrained maximum likelihood estimates of the parameter vector. We compare it with a gradient projection–expectation maximization (GP-EM) algorithm, based on gradient projections. The performance of the algorithms is examined through a simulation study of road safety data.

Unified generalized iterative scaling and its applications

Generalized iterative scaling (GIS) has become a popular method for getting the maximum likelihood estimates for log-linear models. It is basically a sequence of successive I -projections onto sets of probability vectors with some given linear combinations of probability vectors. However, when a sequence of successive I -projections are applied onto some closed and convex sets (e.g., marginal stochastic order), they may not lead to the actual solution. In this manuscript, we present a unified generalized iterative scaling (UGIS) and the convergence of this algorithm to the optimal solution is shown. The relationship between the UGIS and the constrained maximum likelihood estimation for log-linear models is established. Applications to constrained Poisson regression modeling and marginal stochastic order are used to demonstrate the proposed UGIS.

Improved Prediction Intervals and Distribution Functions

Abstract. The plug‐in solution is usually not entirely adequate for computing prediction intervals, as their coverage probability may differ substantially from the nominal value. Prediction intervals with improved coverage probability can be defined by adjusting the plug‐in ones, using rather complicated asymptotic procedures or suitable simulation techniques. Other approaches are based on the concept of predictive likelihood for a future random variable. The contribution of this paper is the definition of a relatively simple predictive distribution function giving improved prediction intervals. This distribution function is specified as a first‐order unbiased modification of the plug‐in predictive distribution function based on the constrained maximum likelihood estimator. Applications of the results to the Gaussian and the generalized extreme‐value distributions are presented.

Bayesian estimation of hispanic fertility hazards from survey and population data*

Previous studies have demonstrated both large gains in efficiency and reductions in bias by incorporating population information in regression estimation with sample survey data. These studies, however, assumed that the population values are exact. This assumption is relaxed here through a Bayesian extension of constrained maximum likelihood estimation applied to U.S. Hispanic fertility. The Bayesian approach allows for the use of both auxiliary survey data and expert judgment in making adjustments to published Hispanic Population fertility rates, and for the estimation of uncertainty about these adjustments. Compared with estimation from sample survey data only, the Bayesian constrained estimator results in much greater precision in the age pattern of the baseline fertility hazard and therefore of the predicted values for any given combination of socioeconomic variables. The use of population data in combination with survey data may therefore be highly advantageous even when the population data are known to have significant levels of nonsampling error

Statistical analysis of non-inferiority via non-zero risk difference in stratified matched-pair studies

A stratified study is often designed for adjusting several independent trials in modern medical research. We consider the problem of non-inferiority tests and sample size determinations for a nonzero risk difference in stratified matched-pair studies, and develop the likelihood ratio and Wald-type weighted statistics for testing a null hypothesis of non-zero risk difference for each stratum in stratified matched-pair studies on the basis of (1) the sample-based method and (2) the constrained maximum likelihood estimation (CMLE) method. Sample size formulae for the above proposed statistics are derived, and several choices of weights for Wald-type weighted statistics are considered. We evaluate the performance of the proposed tests according to type I error rates and empirical powers via simulation studies. Empirical results show that (1) the likelihood ratio and the Wald-type CMLE test based on harmonic means of the stratum-specific sample size (SSIZE) weight (the Cochran's test) behave satisfactorily in the sense that their significance levels are much closer to the prespecified nominal level; (2) the likelihood ratio test is better than Nam's [2006. Non-inferiority of new procedure to standard procedure in stratified matched-pair design. Biometrical J. 48, 966–977] score test; (3) the sample sizes obtained by using SSIZE weight are smaller than other weighted statistics in general; (4) the Cochran's test statistic is generally much better than other weighted statistics with CMLE method. A real example from a clinical laboratory study is used to illustrate the proposed methodologies.

Maximum Likelihood Estimation of Linear Models for Longitudinal Data with Inequality Constraints

Maximum likelihood (ML) estimation for linear models with longitudinal data under inequality restrictions is investigated. Within-subject correlations are modeled by parametric structure. Asymptotic properties of constrained ML estimates, including strong consistency, approximate representation and asymptotic distribution, are derived. Finally, the ML estimators with and without constraints are compared in terms of sample bias, sample mean-square error MSE and sample variance of the estimators by a simulation.

Maximum-Likelihood Estimation, the CramÉr–Rao Bound, and the Method of Scoring With Parameter Constraints

Maximum-likelihood (ML) estimation is a popular approach to solving many signal processing problems. Many of these problems cannot be solved analytically and so numerical techniques such as the method of scoring are applied. However, in many scenarios, it is desirable to modify the ML problem with the inclusion of additional side information. Often this side information is in the form of parametric constraints, which the ML estimate (MLE) must now satisfy. We unify the asymptotic constrained ML (CML) theory with the constrained Cramer-Rao bound (CCRB) theory by showing the CML estimate (CMLE) is asymptotically efficient with respect to the CCRB. We also generalize the classical method of scoring using the CCRB to include the constraints, satisfying the constraints after each iterate. Convergence properties and examples verify the usefulness of the constrained scoring approach. As a particular example, an alternative and more general CMLE is developed for the complex parameter linear model with linear constraints. A novel proof of the efficiency of this estimator is provided using the CCRB.

A note on constrained estimation in the simple linear measurement error model

It is well known that for any sample size the estimator for the slope in the simple linear measurement error model does not possess even a first moment. Addressing this issue constrained maximum likelihood estimators (MLEs) are developed using three different approaches, under a number of identifiability assumptions. The constrained MLEs have finite moments of all orders. They are asymptotically equivalent to the MLE, which is known to be consistent and asymptotically normal. Numerical investigation shows that the constrained MLEs perform well under a wide variety of experimental settings.

A new non‐randomized model for analysing sensitive questions with binary outcomes

We propose a new non-randomized model for assessing the association of two sensitive questions with binary outcomes. Under the new model, respondents only need to answer a non-sensitive question instead of the original two sensitive questions. As a result, it can protect a respondent's privacy, avoid the usage of any randomizing device, and be applied to both the face-to-face interview and mail questionnaire. We derive the constrained maximum likelihood estimates of the cell probabilities and the odds ratio for two binary variables associated with the sensitive questions via the EM algorithm. The corresponding standard error estimates are then obtained by bootstrap approach. A likelihood ratio test and a chi-squared test are developed for testing association between the two binary variables. We discuss the loss of information due to the introduction of the non-sensitive question, and the design of the co-operative parameters. Simulations are performed to evaluate the empirical type I error rates and powers for the two tests. In addition, a simulation is conducted to study the relationship between the probability of obtaining valid estimates and the sample size for any given cell probability vector. A real data set from an AIDS study is used to illustrate the proposed methodologies.

Designs of Superiority and Noninferiority Trials for Binary Responses are Noninterchangeable

To better understand the design of noninferiority trials for binary data, we identify analogies and contrasts between this and the more familiar superiority trial design. We restrict attention to the problem of detecting a difference between experimental and control response rates in the setting where there is no difference (piE - piC = 0) under the noninferiority alternative hypothesis and under the superiority null, and a matching difference between groups under the complementary hypotheses (/piE - piC/ = delta). Our derivation of the constrained maximum likelihood estimates (MLEs) reveals that superiority and noninferiority trials have different nuisance parameters--the marginal response rate and the control-group response rate, respectively. Our empirical results show that when individuals are allocated to treatment groups in the ratio that minimizes the overall sample size, balanced allocation is optimal only for superiority trials when the error rates are equal; otherwise imbalanced allocation is optimal. Different allocation ratios between trial types lead to different variances, and thus to different sample sizes. Finally, since the value of the marginal response rate--a design parameter in noninferiority trials--typically cannot be obtained from preliminary or published studies, we suggest a means of identifying a value that can be used. We conclude that full documentation of the design of a trial requires specification not only of the design parameters but also of the allocation ratio and the nuisance parameter, the value of which is not obvious under unequal allocation.