Semiparametric Maximum Likelihood Estimation Research Articles

On the Breslow–Holubkov estimator

Breslow and Holubkov (J Roy Stat Soc B 59:447-461 1997a) developed semiparametric maximum likelihood estimation for two-phase studies with a case-control first phase under a logistic regression model and noted that, apart for the overall intercept term, it was the same as the semiparametric estimator for two-phase studies with a prospective first phase developed in Scott and Wild (Biometrica 84:57-71 1997). In this paper we extend the Breslow-Holubkov result to general binary regression models and show that it has a very simple relationship with its prospective first-phase counterpart. We also explore why the design of the first phase only affects the intercept of a logistic model, simplify the calculation of standard errors, establish the semiparametric efficiency of the Breslow-Holubkov estimator and derive its asymptotic distribution in the general case.

Semiparametric Regression with Kernel Error Model

Abstract. We propose and study a class of regression models, in which the mean function is specified parametrically as in the existing regression methods, but the residual distribution is modelled non‐parametrically by a kernel estimator, without imposing any assumption on its distribution. This specification is different from the existing semiparametric regression models. The asymptotic properties of such likelihood and the maximum likelihood estimate (MLE) under this semiparametric model are studied. We show that under some regularity conditions, the MLE under this model is consistent (when compared with the possibly pseudo‐consistency of the parameter estimation under the existing parametric regression model), is asymptotically normal with rate and efficient. The non‐parametric pseudo‐likelihood ratio has the Wilks property as the true likelihood ratio does. Simulated examples are presented to evaluate the accuracy of the proposed semiparametric MLE method.

Semiparametric Regression with Kernel Error Model

We propose and study a class of regression models, in which the mean function is specified parametrically as in the existing regression methods, but the residual distribution is modeled nonparametrically by a kernel estimator, without imposing any assumption on its distribution. This specification is different from the existing semiparametric regression models. The asymptotic properties of such likelihood and the maximum likelihood estimate (MLE) under this semiparametric model are studied. We show that under some regularity conditions, the MLE under this model is consistent (as compared to the possibly pseudo consistency of the parameter estimation under the existing parametric regression model), and is asymptotically normal with rate sqrt{n} and efficient. The nonparametric pseudo-likelihood ratio has the Wilks property as the true likelihood ratio does. Simulated examples are presented to evaluate the accuracy of the proposed semiparametric MLE method.

An EM algorithm for regression analysis with incomplete covariate information

Regression analysis is often challenged by the fact that some covariates are not completely observed. Among other approaches is a newly developed semiparametric maximum likelihood (SML) method that requires no parametric specification of the selection mechanism or the covariate distribution and that yields efficient inference, at least in some specific models. In this paper, we propose an EM algorithm for finding the SML estimate and for variance estimation. Simulation results suggest that the SML method performs reasonably well in moderate-sized samples. In contrast, the analogous parametric maximum likelihood method is subject to severe bias under model mis-specification, even in large samples.

Semiparametric Mixtures of Generalized Exponential Families

A semiparametric mixture model is characterized by a non-parametric mixing distribution Q (with respect to a parameter 0) and a structural parameter β common to all components. Much of the literature on mixture models has focused on fixing β and estimating Q. However, this can lead to inconsistent estimation of both Q and the order of the model m. Creating a framework for consistent estimation remains an open problem and is the focus of this article. We formulate a class of generalized exponential family (GEF) models and establish sufficient conditions for the identifiability of finite mixtures formed from a GEF along with sufficient conditions for a nesting structure. Finite identifiability and nesting structure lead to the central result that semiparametric maximum likelihood estimation of Q and β fails. However, consistent estimation is possible if we restrict the class of mixing distributions and employ an information-theoretic approach. This article provides a foundation for inference in semiparametric mixture models, in which GEFs and their structural properties play an instrumental role.

Semiparametric Maximum Likelihood Variance Component Estimation Using Mixture Moment Structure Models

AbstractNonnormal phenotypic distributions introduce significant problems in the estimation and selection of genetic models. Here, a semiparametric maximum likelihood approach to analyzing non-normal phenotypes is described. In this approach, distributions are explicitly modeled together with genetic and environmental effects. Distributional parameters are introduced through mixture constraints, where the distribution of effects are discretized and freely estimated rather than assumed to be normal. Semiparametric maximum likelihood estimation can be used with a variety of genetic models, can be extended to a variety of pedigree structures, and has various advantages over other approaches to modeling nonnormal data.

Semiparametric Maximum Likelihood Variance Component Estimation Using Mixture Moment Structure Models

Nonnormal phenotypic distributions introduce significant problems in the estimation and selection of genetic models. Here, a semiparametric maximum likelihood approach to analyzing nonnormal phenotypes is described. In this approach, distributions are explicitly modeled together with genetic and environmental effects. Distributional parameters are introduced through mixture constraints, where the distribution of effects are discretized and freely estimated rather than assumed to be normal. Semiparametric maximum likelihood estimation can be used with a variety of genetic models, can be extended to a variety of pedigree structures, and has various advantages over other approaches to modeling nonnormal data.

Semiparametric Maximum Likelihood for Missing Covariates in Parametric Regression

We consider parameter estimation in parametric regression models with covariates missing at random. This problem admits a semiparametric maximum likelihood approach which requires no parametric specification of the selection mechanism or the covariate distribution. The semiparametric maximum likelihood estimator (MLE) has been found to be consistent. We show here, for some specific models, that the semiparametric MLE converges weakly to a zero-mean Gaussian process in a suitable space. The regression parameter estimate, in particular, achieves the semiparametric information bound, which can be consistently estimated by perturbing the profile log-likelihood. Furthermore, the profile likelihood ratio statistic is asymptotically chi-squared. The techniques used here extend to other models.

Selfinformative limits of Bayes estimates and generalized maximum likelihood

A definition of selfinformative Bayes carriers or limits is given as a description of an approach to non-informative Bayes estimation in non- and semiparametric models. It takes the posterior w.r.t. a prior as a new prior and repeats this procedure again and again. A main objective of this article is to clarify the relation between selfinformative carriers or limits and maximum likelihood estimates (MLEs). For a model with dominated probability distributions, we state sufficient conditions under which the set of MLEs is a selfinformative carrier or in the case of a unique MLE its selfinformative limit property. Mixture models are covered. The result on carriers is extended to more general models without dominating measure. Selfinformative limits, in the case of estimation, of hazard functions based in censored observations and in the case of normal linear models with possibly non-identifiable parameters are shown to be identical to the generalized MLEs in the sense of Gill [Gill, R.D., 1989, Non- and semi-parametric maximum likelihood estimators and the von Mises method. I. Scandinanian Journal of Statistics, 16(2), 97–128.] and Kiefer and Wolfowitz [Kiefer, J. and Wolfowitz, J., 1956, Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters. Annals of Mathematical Statistics, 27, 887–906.]. Selfinformative limits are given for semiparametric linear models. For a location model, they are identical to generalized MLEs, while this is not true in general.

Fitting logistic regression models with contaminated case–control data

Errors in measurement frequently occur in observing responses. If case–control data are based on certain reported responses, which may not be the true responses, then we have contaminated case–control data. In this paper, we first show that the ordinary logistic regression analysis based on contaminated case–control data can lead to very serious biased conclusions. This can be concluded from the results of a theoretical argument, one example, and two simulation studies. We next derive the semiparametric maximum likelihood estimate (MLE) of the risk parameter of a logistic regression model when there is a validation subsample. The asymptotic normality of the semiparametric MLE will be shown along with consistent estimate of asymptotic variance. Our example and two simulation studies show these estimates to have reasonable performance under finite sample situations.

Semiparametric maximum likelihood estimation exploiting gene-environment independence in case-control studies

We consider the problem of maximum-likelihood estimation in case-control studies of gene-environment associations with disease when genetic and environmental exposures can be assumed to be independent in the underlying population. Traditional logistic regression analysis may not be efficient in this setting. We study the semiparametric maximum likelihood estimates of logistic regression parameters that exploit the gene-environment independence assumption and leave the distribution of the environmental exposures to be nonparametric. We use a profile-likelihood technique to derive a simple algorithm for obtaining the estimator and we study the asymptotic theory. The results are extended to situations where genetic and environmental factors are independent conditional on some other factors. Simulation studies investigate small-sample properties. The method is illustrated using data from a case-control study designed to investigate the interplay of BRCA1/2 mutations and oral contraceptive use in the aetiology of ovarian cancer.

Maximum Likelihood Estimation in a Semiparametric Logistic/Proportional-Hazards Mixture Model

Abstract. We consider large sample inference in a semiparametric logistic/proportional-hazards mixture model. This model has been proposed to model survival data where there exists a positive portion of subjects in the population who are not susceptible to the event under consideration. Previous studies of the logistic/proportional-hazards mixture model have focused on developing point estimation procedures for the unknown parameters. This paper studies large sample inferences based on the semiparametric maximum likelihood estimator. Specifically, we establish existence, consistency and asymptotic normality results for the semiparametric maximum likelihood estimator. We also derive consistent variance estimates for both the parametric and non-parametric components. The results provide a theoretical foundation for making large sample inference under the logistic/proportional-hazards mixture model.

Modified Semiparametric Maximum Likelihood Estimator in Linear Regression Analysis With Complete Data or Right-Censored Data

Consider a linear regression model where the response variable may be right-censored. The standard maximum likelihood estimator (MLE)-based parametric approach to estimation of regression coefficients requires that the parametric form of the error distribution be known. Given a dataset, we may not be able to find a valid parametric form for the error distribution. In such a case the error distribution is unknown and arbitrary, and a semiparametric approach is plausible. A special modified semiparametric MLE (MSMLE) of the regression coefficients is proposed. Simulation suggests that the MSMLE is consistent is asymptotically normally distributed and may be efficient. The new procedure is applied to engineering data.

Estimation of the Asymptotic Variance of Semiparametric Maximum Likelihood Estimators in the Cox Model with a Missing Time-Dependent Covariate

The relationship between a time-to-event and a time-dependent or time-independent covariate is usually assessed using the Cox model. A frequently encountered problem however is occurrence of missing covariate values. A recent approach for estimating the Cox model with a missing covariate jointly models the time-to-event and covariate. In the case of a time-dependent covariate, Dupuy and Mesbah [Dupuy, J.-F., Mesbah, M. (2002). Joint modeling of event time and nonignorable missing longitudinal data. Lifetime D. Anal. 8:99–115] have proposed a joint model and have obtained a semiparametric maximum likelihood estimator of the regression parameter of the Cox model that is consistent and asymptotically normal. Furthermore, an explicit expression was obtained for the asymptotic variance of this estimator. In this paper, we examine the problem of estimating this variance. We propose a computationally simple estimator and we show its consistency. We illustrate the approach by applications to real data sets.

Robust semiparametric M-estimation and the weighted bootstrap

M-estimation is a widely used technique for statistical inference. In this paper, we study properties of ordinary and weighted M-estimators for semiparametric models, especially when there exist parameters that cannot be estimated at the n convergence rate. Results on consistency, rates of convergence for all parameters, and n consistency and asymptotic normality for the Euclidean parameters are provided. These results, together with a generic paradigm for studying semiparametric M-estimators, provide a valuable extension to previous related research on semiparametric maximum-likelihood estimators (MLEs). Although penalized M-estimation does not in general fit in the framework we discuss here, it is shown for a great variety of models that many of the forgoing results still hold, including the n consistency and asymptotic normality of the Euclidean parameters. For semiparametric M-estimators that are not likelihood based, general inference procedures for the Euclidean parameters have not previously been developed. We demonstrate that our paradigm leads naturally to verification of the validity of the weighted bootstrap in this setting. For illustration, several examples are investigated in detail. The new M-estimation framework and accompanying weighted bootstrap technique shed light on a universal way of investigating semiparametric models.

On semiparametric [formula omitted]-estimation in single-index regression

In this paper we analyze a large class of semiparametric M -estimators for single-index models, including semiparametric quasi-likelihood and semiparametric maximum likelihood estimators. Some possible applications to robustness are also mentioned. The definition of these estimators involves a kernel regression estimator for which a bandwidth rule is necessary. Given the semiparametric M -estimation problem, we propose a natural bandwidth choice by joint maximization of the M -estimation criterion with respect to the parameter of interest and the bandwidth. In this way we extend a methodology first introduced by Härdle et al. (Ann. Statist. 21 (1993) 157) for semiparametric least-squares. We prove asymptotic normality for our semiparametric estimator. We derive the asymptotic equivalence between our bandwidth and the optimal bandwidth obtained through weighted cross-validation. Empirical evidence obtained from simulations suggests that our bandwidth improves the higher order asymptotics of the semiparametric M -estimator when it replaces the usual bandwidth chosen by cross-validation.

Goodness-of-fit tests for semiparametric biased sampling models

Consider an s-sample biased sampling model in which the distribution function for each of the first s−1 samples is related to the unknown distribution function G of the sth sample by a known parametric selection bias weight function. Gilbert et al. (Biometrika 86 (1999) 27) gave a procedure for semiparametric maximum likelihood estimation of the parameters in this model. In many applications, information are scarce for basing the choice of the parametric weight function(s), motivating the need for goodness-of-fit tests of the hypothesis that the weight functions are correctly specified. Cramér–von Mises-type, Anderson–Darling-type, and Kolmogorov–Smirnov-type test statistics are studied which compare discrepancies between the empirical distribution of G and the semiparametric maximum likelihood estimator of G. Finite-sample properties of the tests are evaluated with simulations and with a real example of HIV genetic sequence data.

Theory & Methods: Semi‐parametric modelling and likelihood estimation with estimating equations

This paper proposes a semi‐parametric modelling and estimating method for analysing censored survival data. The proposed method uses the empirical likelihood function to describe the information in data, and formulates estimating equations to incorporate knowledge of the underlying distribution and regression structure. The method is more flexible than the traditional methods such as the parametric maximum likelihood estimation (MLE), Cox’s (1972) proportional hazards model, accelerated life test model, quasi‐likelihood (Wedderburn, 1974) and generalized estimating equations (Liang & Zeger, 1986). This paper shows the existence and uniqueness of the proposed semi‐parametric maximum likelihood estimates (SMLE) with estimating equations. The method is validated with known cases studied in the literature. Several finite sample simulation and large sample efficiency studies indicate that when the sample size is larger than 100 the SMLE is compatible with the parametric MLE; and in all case studies, the SMLE is about 15% better than the parametric MLE with a mis‐specified underlying distribution.

Semiparametric maximum likelihood for nonlinear regression with measurement errors.

This article demonstrates semiparametric maximum likelihood estimation of a nonlinear growth model for fish lengths using imprecisely measured ages. Data on the species corvina reina, found in the Gulf of Nicoya, Costa Rica, consist of lengths and imprecise ages for 168 fish and precise ages for a subset of 16 fish. The statistical problem may therefore be classified as nonlinear errors-in-variables regression with internal validation data. Inferential techniques are based on ideas extracted from several previous works on semiparametric maximum likelihood for errors-in-variables problems. The illustration of the example clarifies practical aspects of the associated computational, inferential, and data analytic techniques.

The analysis of retrospective family studies

Case‐control samples allow straightforward calculation of estimates of the association between covariates and disease status by fitting a prospective logistic regression model. In genetic studies of disease, investigators often gather additional information on response and covariate variables from family members of cases and controls. The objective is to model the responses of all the family members in terms of the covariate data. Whittemore (1995) has discussed maximum likelihood methods for fitting a special class of logistic models to family data collected according to a particular design. In the present paper, we show that we can obtain efficient semiparametric maximum likelihood estimates for an arbitrary multivariate binary regression model by fitting a modified prospective model for a wide class of retrospective designs. However, in contrast to the situation with simple case‐control studies, the prospective model will differ from the original model even when the model is logistic.