Residual Maximum Likelihood Estimation Research Articles

Influence diagnostics for random effect survival models: Application to a recurrent infection study for kidney patients on portable dialysis

In modeling multivariate failure time data, a class of survival model with random effects is applicable. It incorporates the random effect terms in the linear predictor and includes various random effect survival models as special cases, such as the random effect model assuming Cox's proportional hazards, with Weibull baseline hazards and with power family of transformation in the relative risk function. Residual maximum likelihood (REML) estimation of parameters is achieved by adopting the generalised linear mixed models (GLMM) approach. Accordingly, influence diagnostics are developed as sensitivity measures for the REML estimation of model parameters. A data set of recurrent infections of kidney patients on portable dialysis illustrates the usefulness of the influence diagnostics. A simulation study is carried out to examine the performance of the proposed influence diagnostics.

Extension of the concept of kinship, relationship, and inbreeding to account for linked epistatic complexes

Abstract Although epistatic effects are well defined and, in principle, can be exploited in quantitative-genetic selection theory, they often are ignored or even treated as nuisance parameters in practical applications. Traditionally, epistasis is considered as an interaction between genes at unspecified loci. Inspired by the observation that functional genes are often organised in physical clusters, we developed a model to combine additive effects and additive × additive interactions in linked gene clusters of defined length. Malecot's kinship concept is extended to identity by descent probabilities for chromosome segments of a given length in Morgan units, called epistatic kinship. Using the analogy of Malecot's kinship and Wright's relationship and inbreeding coefficients, epistatic relationship coefficients and epistatic inbreeding coefficients are defined. Simple rules are given to set up the epistatic numerator relationship matrix and its inverse directly from a pedigree list. The well-known single locus parameters and algorithms to set up the additive numerator relationship matrix and its inverse are a special case of the suggested methodology for a chromosome segment length of null Morgan. A proof of concept of the suggested method is given with a small simulation study. Assuming additive, linked epistatic and residual variance components, 100 replicated data sets for 1000 individuals are generated. From these data, residual maximum likelihood estimates of the variance components and of the chromosome segment size are obtained. Potential applications of the methodology are discussed. Given that a substantial variance component is attributed to this effect, the expected genetic gain can be increased on the short term if selection is on additive and epistatic effects, the latter comprising additive × additive interaction effect of loci in linkage disequilibrium. This extra benefit, however, will diminish through crossing over in subsequent generations. Despite some practical problems yet to be solved, the suggested model and algorithms open new perspectives to use a higher proportion of genetic variability in selection and breeding.

High-Breakdown Inference for Mixed Linear Models

Mixed linear models are used to analyze data in many settings. These models have a multivariate normal formulation in most cases. The maximum likelihood estimator (MLE) or the residual MLE (REML) is usually chosen to estimate the parameters. However, the latter are based on the strong assumption of exact multivariate normality. Welsh and Richardson have shown that these estimators are not robust to small deviations from multivariate normality. This means that in practice a small proportion of data (even only one) can drive the value of the estimates on their own. Because the model is multivariate, we propose a high-breakdown robust estimator for very general mixed linear models that include, for example, covariates. This robust estimator belongs to the class of S-estimators, from which we can derive asymptotic properties for inference. We also use it as a diagnostic tool to detect outlying subjects. We discuss the advantages of this estimator compared with other robust estimators proposed previously and illustrate its performance with simulation studies and analysis of three datasets. We also consider robust inference for multivariate hypotheses as an alternative to the classical F-test by using a robust score-type test statistic proposed by Heritier and Ronchetti, and study its properties through simulations and analysis of real data.

On spatial prediction of soil properties in the presence of a spatial trend: the empirical best linear unbiased predictor (E‐BLUP) with REML

SummaryGeostatistical estimates of a soil property by kriging are equivalent to the best linear unbiased predictions (BLUPs). Universal kriging is BLUP with a fixed‐effect model that is some linear function of spatial co‐ordinates, or more generally a linear function of some other secondary predictor variable when it is called kriging with external drift. A problem in universal kriging is to find a spatial variance model for the random variation, since empirical variograms estimated from the data by method‐of‐moments will be affected by both the random variation and that variation represented by the fixed effects.The geostatistical model of spatial variation is a special case of the linear mixed model where our data are modelled as the additive combination of fixed effects (e.g. the unknown mean, coefficients of a trend model), random effects (the spatially dependent random variation in the geostatistical context) and independent random error (nugget variation in geostatistics). Statisticians use residual maximum likelihood (REML) to estimate variance parameters, i.e. to obtain the variogram in a geostatistical context. REML estimates are consistent (they converge in probability to the parameters that are estimated) with less bias than both maximum likelihood estimates and method‐of‐moment estimates obtained from residuals of a fitted trend. If the estimate of the random effects variance model is inserted into the BLUP we have the empirical BLUP or E‐BLUP. Despite representing the state of the art for prediction from a linear mixed model in statistics, the REML–E‐BLUP has not been widely used in soil science, and in most studies reported in the soils literature the variogram is estimated with methods that are seriously biased if the fixed‐effect structure is more complex than just an unknown constant mean (ordinary kriging). In this paper we describe the REML–E‐BLUP and illustrate the method with some data on soil water content that exhibit a pronounced spatial trend.

Modelling bivariate count series with excess zeros

Bivariate time series of counts with excess zeros relative to the Poisson process are common in many bioscience applications. Failure to account for the extra zeros in the analysis may result in biased parameter estimates and misleading inferences. A class of bivariate zero-inflated Poisson autoregression models is presented to accommodate the zero-inflation and the inherent serial dependency between successive observations. An autoregressive correlation structure is assumed in the random component of the compound regression model. Parameter estimation is achieved via an EM algorithm, by maximizing an appropriate log-likelihood function to obtain residual maximum likelihood estimates. The proposed method is applied to analyze a bivariate series from an occupational health study, in which the zero-inflated injury count events are classified as either musculoskeletal or non-musculoskeletal in nature. The approach enables the evaluation of the effectiveness of a participatory ergonomics intervention at the population level, in terms of reducing the overall incidence of lost-time injury and a simultaneous decline in the two mean injury rates.

Conditional Akaike information for mixed-effects models

SUMMARY This paper focuses on the Akaike information criterion, AIC, for linear mixed-effects models in the analysis of clustered data. We make the distinction between questions regarding the population and questions regarding the particular clusters in the data. We show that the AIC in current use is not appropriate for the focus on clusters, and we propose instead the conditional Akaike information and its corresponding criterion, the conditional AIC, CAIC. The penalty term in CAIC is related to the effective degrees of freedom p for a linear mixed model proposed by Hodges & Sargent (2001); p reflects an intermediate level of complexity between a fixed-effects model with no cluster effect and a corresponding model with fixed cluster effects. The CAIC is defined for both maximum likelihood and residual maximum likelihood estimation. A pharmacokinetics data appli cation is used to illuminate the distinction between the two inference settings, and to illustrate the use of the conditional AIC in model selection.

On measuring the variability of small area estimators under a basic area level model

In this paper based on a basic area level model we obtain second-order accurate approximations to the mean squared error of model-based small area estimators, using the Fay & Herriot (1979) iterative method of estimating the model variance based on weighted residual sum of squares. We also obtain mean squared error estimators unbiased to second order. Based on simulations, we compare the finite-sample performance of our mean squared error estimators with those based on method-of-moments, maximum likelihood and residual maximum likelihood estimators of the model variance. Our results suggest that the Fay–Herriot method performs better, in terms of relative bias of mean squared error estimators, than the other methods across different combinations of number of areas, pattern of sampling variances and distribution of small area effects. We also derive a noninformative prior on the model parameters for which the posterior variance of a small area mean is second-order unbiased for the mean squared error. The posterior variance based on such a prior possesses both Bayesian and frequentist interpretations.

Exploring scale‐dependent correlation of soil properties by nested sampling

SummaryThe value of nested sampling for exploring the spatial structure of univariate variation of the soil has been demonstrated in several studies and applied to practical problems. This paper shows how the method can be extended to the multivariate case. While the extension is simple in theory, in practice the direct estimation of covariance components by equating mean‐square matrices with their expectation will often lead to estimates that are not positive semidefinite. This paper discusses solutions to this problem for balanced and unbalanced sample designs. In the balanced case there is a residual maximum likelihood (REML) estimator that will find estimates of covariance components that maximize an overall likelihood on the condition that all components are positive semidefinite (p.s.d.). This is possible because the condition is met if the differences of successive mean‐square matrices are positive semidefinite, and this constraint can be incorporated into an algorithm. This does not hold for unbalanced designs. In this paper the problem was solved for unbalanced designs by scaling covariance components that were not p.s.d. to the nearest p.s.d. matrix according to a Euclidean distance.These methods were applied to data from three surveys, two with balanced and one with unbalanced sampling. Different patterns of scale‐dependence of the correlation of soil properties were found. For example, at Ginninderra Experimental Station in Australia the soil water content and bulk density were correlated significantly, with the correlation increasing with distance to 56 m, but at longer distances the properties were not significantly correlated. By contrast, the pH of the soil and the available P content showed correlation that increased with distance. The implications of these results for planning more detailed sampling, both for prediction and for investigation of processes, are discussed.

Modeling zero-inflated count series with application to occupational health

A zero-inflated Poisson mixed autoregression model is presented for analyzing time series of count events with excess zeros. The model is motivated by the evaluation of a participatory ergonomics intervention intended to reduce manual handling workplace injuries over a specified time period. Random effects are introduced into the linear predictor of the model to account for serial correlation between successive observations. Parameter estimation is achieved by maximizing an appropriate log-likelihood function to obtain approximate residual maximum likelihood estimates. The method enables the evaluation of occupational intervention using population level aggregated count data series containing extra zeros.

Fixed or random contemporary groups in genetic evaluation for litter size in pigs using a single trait repeatability animal model

SummaryThe importance of using fixed or random contemporary groups in the genetic evaluation of litter size in pigs was analysed by using farm and simulated data. Farm data were from four Spanish pig breeding populations, two Landrace (13 084 and 13 619 records) and two Large White (2762 and 8455 records). A simulated population (200 sows and 10 boars) selected for litter size, in which litter size was simulated using a repeatability animal model with random herd–year–season (HYS), was used to obtain simulated data. With farm data, the goodness‐of‐fit and the predictive ability of a repeatability animal model were calculated under several definitions of the HYS effect. A residual maximum likelihood estimator of the HYS variance in each population was obtained as well. In this sense, HYS was considered as either fixed or random with different number of individuals per level. Results from farm data showed that HYS variance was small in relation to the total variance (ranging from 0.01 to 0.04). The treatment of HYS effect as fixed, reduced the residual variance but the size of HYS levels does not explain by itself the goodness‐of‐fit of the model. The results obtained by simulation showed that the predictive ability of the model is better for random than for fixed HYS models. However, the improvement of predictive ability does not lead to a significant increase of the genetic response. Finally, results showed that random HYS models biased the estimates of genetic response when there is an environmental trend.

The estimation of density dependence using census data from several sites.

The discussion of density dependence in the ecological literature has tended to concentrate on significance testing, whereas estimation is often of more importance. In this paper we use simulation to investigate the bias and precision of a number of methods for estimating density dependence using census data from several sites. The estimation methods examined were a modification of an existing maximum likelihood approach, a regression method, and a new approach using restricted (or residual) maximum likelihood (REML). Simulations indicated that the REML method produced the most accurate estimates, with negligible bias for most parameter combinations. A further advantage of the REML method is that it can be easily implemented using standard statistical software. Using a second series of simulations we investigated the relationship between accuracy of estimation and sample size for the REML method. The results indicated that using more than one site gave substantial improvement in accuracy, but that using more than five sites gave little further improvement unless in excess of 10years of data were available for each site. Where a standard error is required for the density dependence estimate we suggest using bootstrapping at the site level. Where this is not possible, a parametric bootstrap or a randomisation test may be used instead. The REML method is demonstrated using bag totals of red-legged partridges Alectoris rufa and grey partridges Perdix perdix for shoots on estates in East Anglia in the United Kingdom. This paper shows that combining information from several sites can give improved estimation of density dependence, particularly if REML estimation is adopted.

Analysis of unbalanced mixed model data: A case study comparison of ANOVA versus REML/GLS

Major transition has occurred in recent years in statistical methods for analysis of linear mixed model data from analysis of variance (ANOVA) to likelihood-based methods. Prior to the early 1990s, most applications used some version of analysis of variance because computer software was either not available or not easy to use for likelihood-based methods. ANOVA is based on ordinary least squares computations, with adoptions for mixed models. Computer programs for such methodology were plagued with technical problems of estimability, weighting, and handling missing data. Likelihood-based methods mainly use a combination of residual maximum likelihood (REML) estimation of covariance parameters and generalized least squares (GLS) estimation of mean parameters. Software for REML/GLS methods became readily available early in the 1990s, but the methodology still is not universally embraced. Although many of the computational inadequacies have been overcome, conceptual problems remain. Also, technical problems with REML/GLS have emerged, such as the need for adjustments for effects due to estimating covariance parameters. This article attempts to identify the major problems with ANOVA, describe the problems which remain with REML/GLS, and discuss new problems with REML/GLS.

A unified jackknife theory for empirical best prediction with M-estimation

The paper presents a unified jackknife theory for a fairly general class of mixed models which includes some of the widely used mixed linear models and generalized linear mixed models as special cases. The paper develops jackknife theory for the important, but so far neglected, prediction problem for the general mixed model. For estimation of fixed parameters, a jackknife method is considered for a general class of M-estimators which includes the maximum likelihood, residual maximum likelihood and ANOVA estimators for mixed linear models and the recently developed method of simulated moments estimators for generalized linear mixed models. For both the prediction and estimation problems, a jackknife method is used to obtain estimators of the mean squared errors (MSE). Asymptotic unbiasedness of the MSE estimators is shown to hold essentially under certain moment conditions. Simulation studies undertaken support our theoretical results.

Finite mixture regression model with random effects: application to neonatal hospital length of stay

A two-component mixture regression model that allows simultaneously for heterogeneity and dependency among observations is proposed. By specifying random effects explicitly in the linear predictor of the mixture probability and the mixture components, parameter estimation is achieved by maximising the corresponding best linear unbiased prediction type log-likelihood. Approximate residual maximum likelihood estimates are obtained via an EM algorithm in the manner of generalised linear mixed model (GLMM). The method can be extended to a g-component mixture regression model with the component density from the exponential family, leading to the development of the class of finite mixture GLMM. For illustration, the method is applied to analyse neonatal length of stay (LOS). It is shown that identification of pertinent factors that influence hospital LOS can provide important information for health care planning and resource allocation.

A zero-inflated Poisson mixed model to analyze diagnosis related groups with majority of same-day hospital stays

With increasing trend of same-day procedures and operations performed for hospital admissions, it is important to analyze those Diagnosis Related Groups (DRGs) consisting of mainly same-day separations. A zero-inflated Poisson (ZIP) mixed model is presented to identify health- and patient-related characteristics associated with length of stay (LOS) and to model variations in LOS within such DRGs. Random effects are introduced to account for inter-hospital variations and the dependence of clustered LOS observations via the generalized linear mixed models (GLMM) approach. Parameter estimation is achieved by maximizing an appropriate log-likelihood function using the EM algorithm to obtain approximate residual maximum likelihood (REML) estimates. An S-Plus macro is developed to provide a unified ZIP modeling approach. The determination of pertinent factors would benefit hospital administrators and clinicians to manage LOS and expenditures efficiently.

Multilevel models for survival analysis with random effects.

A method for modeling survival data with multilevel clustering is described. The Cox partial likelihood is incorporated into the generalized linear mixed model (GLMM) methodology. Parameter estimation is achieved by maximizing a log likelihood analogous to the likelihood associated with the best linear unbiased prediction (BLUP) at the initial step of estimation and is extended to obtain residual maximum likelihood (REML) estimators of the variance component. Estimating equations for a three-level hierarchical survival model are developed in detail, and such a model is applied to analyze a set of chronic granulomatous disease (CGD) data on recurrent infections as an illustration with both hospital and patient effects being considered as random. Only the latter gives a significant contribution. A simulation study is carried out to evaluate the performance of the REML estimators. Further extension of the estimation procedure to models with an arbitrary number of levels is also discussed.

STATISTICAL PERSPECTIVE ON ADJUSTING FOR CENTER EFFECT IN A 2 × 2 CROSSOVER DESIGN WITH APPLICATION TO CLINICAL TRIALS

Patients for large-scale clinical trials are typically recruited from multiple centers to ensure adequate patient population and timely enrollment. In this note, carryover, treatment, and period effects for continuous data in multicenter studies with a 2 × 2 crossover design are examined using two types of covariate analysis adjusted for center effect proposed by Castellana and Patel [1]. The correspondence among F-tests for various sources of variation in the two approaches is derived in detail. An example from a clinical study with complete and incomplete data is given to illustrate the use of the two types of covariate analysis. Also, the impact of complete and incomplete data on the covariate analyses is discussed along with the weighted squares of means analysis used in SAS procedure PROC GLM and residual maximum likelihood (REML) estimation employed in SAS procedure PROC MIXED.

ML and REML Estimation of Matusita's Measure for Two Bivariate Normal Distributions with Missing Observations

SYNOPTIC ABSTRACTThe measures of niche overlap are used to assess the similarity or dissimilarity of two populations. Matusita's measure is one of the commonly used niche overlap measures. We discuss the problem of estimating Matusita's measure when the niches are bivariate normal distributions with missing observations. Under the assumption of equal variance of two variates in each population, we consider four cases depending on whether the variances and correlations for the two populations are common or different. The plug-in estimates of Matusita's measure by the Maximum Likelihood (ML) estimates and the REstricted or REsidual Maximum Likelihood (REML) estimates for dispersion parameters are considered, their asymptotic variances and bias are derived, and bias correction methods are proposed. Simulation study shows that the plug-in estimate by the REMLE tends to have smaller MSE than that by the MLE and the bias correction reduces MSE considerably.

Teaching Statistics to Biological Research Scientists

This paper describes a programme of modules for training scientists from biological research organizations. It consists of one- and two-day modules on introductory statistics, advanced methods such as residual maximum likelihood estimation and specialized applications such as the analysis of molecular sequence data and mathematical modelling. The philosophy behind the programme is discussed, and the contents of modules are summarized with illustrations of how concepts are explained. An assessment of the effect of the programme and future directions are also considered.

A Simulation Study for the Binomial-Logit Model with Correlated Random Effects

A simulation study of the binomial-logit model with correlated random effects is carried out based on the generalized linear mixed model (GLMM) methodology. Simulated data with various numbers of regression parameters and different values of the variance component are considered. The performance of approximate maximum likelihood (ML) and residual maximum likelihood (REML) estimators is evaluated. For a range of true parameter values, we report the average biases of estimators, the standard error of the average bias and the standard error of estimates over the simulations. In general, in terms of bias, the two methods do not show significant differences in estimating regression parameters. The REML estimation method is slightly better in reducing the bias of variance component estimates.