Probability Of Missingness Research Articles

A Bayesian hybrid method for the analysis of generalized linear models with missing-not-at-random covariates

Missing data handling is one of the main problems in modelling, particularly if the missingness is of type missing-not-at-random (MNAR) where missingness occurs due to the actual value of the observation. The focus of the current article is generalized linear modelling of fully observed binary response variables depending on at least one MNAR covariate. For the traditional analysis of such models, an individual model for the probability of missingness is assumed and incorporated in the model framework. However, this probability model is untestable, as the missingness of MNAR data depend on their actual values that would have been observed otherwise. In this article, we consider creating a model space that consist of all possible and plausible models for probability of missingness and develop a hybrid method in which a reversible jump Markov chain Monte Carlo (RJMCMC) algorithm is combined with Bayesian Model Averaging (BMA). RJMCMC is adopted to obtain posterior estimates of model parameters as well as probability of each model in the model space. BMA is used to synthesize coefficient estimates from all models in the model space while accounting for model uncertainties. Through a validation study with a simulated data set and a real data application, the performance of the proposed methodology is found to be satisfactory in accuracy and efficiency of estimates.

How estimating nuisance parameters can reduce the variance (with consistent variance estimation).

We often estimate a parameter of interest when the identifying conditions involve a finite-dimensional nuisance parameter . Examples from causal inference are inverse probability weighting, marginal structural models and structural nested models, which all lead to unbiased estimating equations. This article presents a consistent sandwich estimator for the variance of estimators that solve unbiased estimating equations including which is also estimated by solving unbiased estimating equations. This article presents four additional results for settings where solves (partial) score equations and does not depend on . This includes many causal inference settings where describes the treatment probabilities, missing data settings where describes the missingness probabilities, and measurement error settings where describes the error distribution. These four additional results are: (1) Counter-intuitively, the asymptotic variance of is typically smaller when is estimated. (2) If estimating is ignored, the sandwich estimator for the variance of is conservative. (3) A consistent sandwich estimator for the variance of . (4) If with the true plugged in is efficient, the asymptotic variance of does not depend on whether is estimated. To illustrate we use observational data to calculate confidence intervals for (1) the effect of cazavi versus colistin on bacterial infections and (2) how the effect of antiretroviral treatment depends on its initiation time in HIV-infected patients.

Instability of inverse probability weighting methods and a remedy for nonignorable missing data.

Inverse probability weighting (IPW) methods are commonly used to analyze nonignorable missing data (NIMD) under the assumption of a logistic model for the missingness probability. However, solving IPW equationsnumerically may involve nonconvergence problems when the sample size is moderate and the missingness probability is high. Moreover, those equationsoften have multiple roots, and identifying the best root is challenging. Therefore, IPW methods may have low efficiency or even produce biased results. We identify the pitfall in these methods pathologically: they involve the estimation of a moment-generating function (MGF), and such functions are notoriously unstable in general. As a remedy, we model the outcome distribution given the covariates of the completely observed individuals semiparametrically. After forming an induced logistic regression (LR) model for the missingness status of the outcome and covariate, we develop a maximum conditional likelihood method to estimate the underlying parameters. The proposed method circumvents the estimation of an MGF and hence overcomes the instability of IPW methods. Our theoretical and simulation results show that the proposed method outperforms existing competitors greatly. Two real data examples are analyzed to illustrate the advantages of our method. We conclude that if only a parametric LR is assumed but the outcome regression model is left arbitrary, then one has to be cautious in using any of the existing statistical methods in problems involvingNIMD.

A multiple imputation-based sensitivity analysis approachfor regression analysis with a missing notatrandom covariate.

Missing covariate problems are common in biomedical and electrical medical record data studies while evaluating the relationship between a biomarker and certain clinical outcome, when biomarker data are not collected for all subjects. However, missingness mechanism is unverifiable based on observed data. If there is a suspicion of missing not at random (MNAR), researchers often perform sensitivity analysis to evaluate the impact of various missingness mechanisms. Under the selection modeling framework, we propose a sensitivity analysis approach with a standardized sensitivity parameter using a nonparametric multiple imputation strategy. The proposed approach requires fitting two working models to derive two predictive scores: one for predicting missing covariate values and the other for predicting missingness probabilities. For each missing covariate observation, the two predictive scores along with the pre-specified sensitivity parameter are used to define an imputing set. The proposed approach is expected to be robust against mis-specifications of the selection model and the sensitivity parameter since the selection model and the sensitivity parameter are not directly used to impute missing covariate values. A simulation study is conducted to study the performance of the proposed approach when MNAR is induced by Heckman's selection model. Simulation results show the proposed approach can produce plausible regression coefficient estimates. The proposed sensitivity analysis approach is also applied to evaluate the impact of MNAR on the relationship between post-operative outcomes and incomplete pre-operative Hemoglobin A1c level for patients who underwent carotid intervetion for advanced atherosclerotic disease.

Response to 'Effects of parity on lifestyle changes: from a methodological perspective to societal applications'.

We appreciate the interest of Anamege and Perez-Carreno in our article titled ‘Assessing patterns of change in lifestyle behaviours by parity: a longitudinal cohort study’.1 We are also grateful to the editor for the opportunity to respond and to provide further details about our work. We excluded all respondents with implausible energy intake (n = 286) from all analyses (these were not part of the 4927 participants included in the final analysis). We only excluded the 573 women who were pregnant in sensitivity analysis to explore the impact of pregnancy including confounding (these were included in the 4927 participants in the final analysis). Missing data on weight and body mass index (BMI) reported in Table 1 of the published manuscript were from the baseline in 2003 (Survey 3).1 At Survey 5 in 2009 (the endpoint in our analysis), missing data for both weight and BMI in parous women was lower, at 1%. Hence previous weight and BMI measurements did not seem to affect women’s willingness to provide data in subsequent surveys. The pattern of missingness in weight and BMI data is consistent with decisions made for data cleaning in the Australian Longitudinal Study of Women’s Health (ALSWH) data.2 In 2002, limits were set for plausible weights, and the derived BMI variable and values outside these limits were set to missing.2 This decision was reviewed in July 2006 in light of analytical issues arising and it was decided that extreme values should no longer be deleted.2 Hence survey data from Survey 4 in 2006 onwards had less missingness in the weight and BMI data. The probability of missingness in our weight and BMI data is independent of the missing values given the observed data (i.e. it can be predicted based on the observed data). This satisfies the ‘missing at random’ mechanism and validates our analysis.3,4

Inference about quantitative traits under selection: a Bayesian revisitation for the post-genomic era

BackgroundSelection schemes distort inference when estimating differences between treatments or genetic associations between traits, and may degrade prediction of outcomes, e.g., the expected performance of the progeny of an individual with a certain genotype. If input and output measurements are not collected on random samples, inferences and predictions must be biased to some degree. Our paper revisits inference in quantitative genetics when using samples stemming from some selection process. The approach used integrates the classical notion of fitness with that of missing data. Treatment is fully Bayesian, with inference and prediction dealt with, in an unified manner. While focus is on animal and plant breeding, concepts apply to natural selection as well. Examples based on real data and stylized models illustrate how selection can be accounted for in four different situations, and sometimes without success.ResultsOur flexible “soft selection” setting helps to diagnose the extent to which selection can be ignored. The clear connection between probability of missingness and the concept of fitness in stylized selection scenarios is highlighted. It is not realistic to assume that a fixed selection threshold t holds in conceptual replication, as the chance of selection depends on observed and unobserved data, and on unequal amounts of information over individuals, aspects that a “soft” selection representation addresses explicitly. There does not seem to be a general prescription to accommodate potential distortions due to selection. In structures that combine cross-sectional, longitudinal and multi-trait data such as in animal breeding, balance is the exception rather than the rule. The Bayesian approach provides an integrated answer to inference, prediction and model choice under selection that goes beyond the likelihood-based approach, where breeding values are inferred indirectly.ConclusionsThe approach used here for inference and prediction under selection may or may not yield the best possible answers. One may believe that selection has been accounted for diligently, but the central problem of whether statistical inferences are good or bad does not have an unambiguous solution. On the other hand, the quality of predictions can be gauged empirically via appropriate training-testing of competing methods.

High-dimensional Changepoint Estimation with Heterogeneous Missingness

AbstractWe propose a new method for changepoint estimation in partially observed, high-dimensional time series that undergo a simultaneous change in mean in a sparse subset of coordinates. Our first methodological contribution is to introduce a ‘MissCUSUM’ transformation (a generalisation of the popular cumulative sum statistics), that captures the interaction between the signal strength and the level of missingness in each coordinate. In order to borrow strength across the coordinates, we propose to project these MissCUSUM statistics along a direction found as the solution to a penalised optimisation problem tailored to the specific sparsity structure. The changepoint can then be estimated as the location of the peak of the absolute value of the projected univariate series. In a model that allows different missingness probabilities in different component series, we identify that the key interaction between the missingness and the signal is a weighted sum of squares of the signal change in each coordinate, with weights given by the observation probabilities. More specifically, we prove that the angle between the estimated and oracle projection directions, as well as the changepoint location error, are controlled with high probability by the sum of two terms, both involving this weighted sum of squares, and representing the error incurred due to noise and the error due to missingness respectively. A lower bound confirms that our changepoint estimator, which we call MissInspect, is optimal up to a logarithmic factor. The striking effectiveness of the MissInspect methodology is further demonstrated both on simulated data, and on an oceanographic data set covering the Neogene period.

Semiparametric marginal regression for clustered competing risks data with missing cause of failure.

Clustered competing risks data are commonly encountered in multicenter studies. The analysis of such data is often complicated due to informative cluster size (ICS), a situation where the outcomes under study are associated with the size of the cluster. In addition, the cause of failure is frequently incompletely observed in real-world settings. To the best of our knowledge, there is no methodology for population-averaged analysis with clustered competing risks data with an ICS and missing causes of failure. To address this problem, we consider the semiparametric marginal proportional cause-specific hazards model and propose a maximum partial pseudolikelihood estimator under a missing at random assumption. To make the latter assumption more plausible in practice, we allow for auxiliary variables that may be related to the probability of missingness. The proposed method does not impose assumptions regarding the within-cluster dependence and allows for ICS. The asymptotic properties of the proposed estimators for both regression coefficients and infinite-dimensional parameters, such as the marginal cumulative incidence functions, are rigorously established. Simulation studies show that the proposed method performs well and that methods that ignore the within-cluster dependence and the ICS lead to invalid inferences. The proposed method is applied to competing risks data from a large multicenter HIV study in sub-Saharan Africa where a significant portion of causes of failure is missing.

Return-to-baseline multiple imputation for missing values in clinical trials.

Return-to-baseline is an important method to impute missing values or unobserved potential outcomes when certain hypothetical strategies are used to handle intercurrent events in clinical trials. Current return-to-baseline approaches seen in literature and in practice inflate the variability of the "complete" dataset after imputation and lead to biased mean estimators when the probability of missingness depends on the observed baseline and/or postbaseline intermediate outcomes. In this article, we first provide a set of criteria a return-to-baseline imputation method should satisfy. Under this framework, we propose a novel return-to-baseline imputation method. Simulations show the completed data after the new imputation approach have the proper distribution, and the estimators based on the new imputation method outperform the traditional method in terms of both bias and variance, when missingness depends on the observed values. The new method can be implemented easily with the existing multiple imputation procedures in commonly used statistical packages.

A doubly robust method to handle missing multilevel outcome data with application to the China Health and Nutrition Survey.

Missing data are common in longitudinal cohort studies and can lead to bias, particularly in studies with informative missingness. Many common methods for handling informatively missing data in survey samples require correctly specifying a model for missingness. Although doubly robust methods exist to provide unbiased regression coefficients in the presence of missing outcome data, these methods do not account for correlation due to clustering inherent in longitudinal or cluster-sampled studies. In this work, we developed a doubly robust method to estimate the regression of an outcome on a predictor in the presence of missing multilevel data on the outcome, which results in consistent estimation of regression coefficients assuming correct specification of either (1) the probability of missingness or (2) the outcome model. This method involves specification of separate hierarchical models for missingness and for the outcome, conditional on observed auxiliary variables and cluster-specific random effects, to account for correlation among observations. We showed this proposed estimator is doubly robust and derived its asymptotic distribution, conducted simulation studies to compare the method to an existing doubly robust method developed for independent data, and applied the method to data from the China Health and Nutrition Survey, an ongoing multilevel longitudinal cohort study.

1223Handling missing data for causal effect estimation in cohort studies using Targeted Maximum Likelihood Estimation

Abstract Background Causal inference from cohort studies is central to epidemiological research. Targeted Maximum Likelihood Estimation (TMLE) is an appealing doubly robust method for causal effect estimation, but it is unclear how missing data should be handled when it is used in conjunction with machine learning approaches for the exposure and outcome models. This is problematic because missing data are ubiquitous and can result in biased estimates and loss of precision if handled inappropriately. Methods Based on a motivating example from the Victorian Adolescent Health Cohort Study, we conducted a simulation study to evaluate the performance of available approaches for handling missing data when using TMLE with machine learning. These included complete-case analysis; an extended TMLE approach incorporating an outcome missingness probability model; the missing indicator approach for missing covariate data (MCMI); and multiple imputation (MI) using standard parametric approaches or machine learning algorithms. We considered 11 missingness mechanisms typical in cohort studies, and a simple and a complex setting, in which exposure and outcome generation models included two-way and higher-order interactions. Results MI using regression with no interactions and MI with random forest yielded estimates with the highest bias. MI with regression including two-way interactions was the best performing method overall. Of the non-MI approaches, MCMI performed the worst Conclusions When using TMLE with machine learning to estimate the average causal effect, avoiding standard MI with no interactions and MCMI is recommended. Key messages We provide novel guidance for handling missing data for causal effect estimation using TMLE.

A randomized controlled trial evaluating the effects of a family-centered HIV care model on viral suppression and retention in care of HIV-positive children in Eswatini.

IntroductionA family-centered care model (FCCM) providing family-based HIV services, rather than separate adult/pediatric services, has been proposed to increase pediatric retention and treatment adherence.Materials and methodsEight health-care facilities in the Hhohho region of Eswatini were randomized to implement FCCM (n = 4) or continue standard-of-care (SOC) separate adult/pediatric clinics (n = 4). HIV-positive children and caregivers were enrolled; caregiver interview and child/caregiver chart abstraction were done at enrollment and every three months; pediatric viral load was evaluated at enrollment and every six months through 12 months. Because of study group differences in 12-month viral load data availability (89.4% FCCM and 72.0% SOC children had 12-month viral load), we used three separate analyses to evaluate the effects of FCCM on children’s viral suppression (<1,000 copies/mL) and undetectable virus (<400 copies/mL) at 12 months. In the first analysis, all children with missing viral outcome data were excluded from the analysis (modified intent to treat, mITT). The second analysis used inverse probability of missingness weighted logistic regression to estimate the effect of FCCM on 12-month viral outcomes compared to SOC (weighted mITT). For the third approach, missing virologic outcome data were imputed as virologic failure (imputed ITT). We also examined factors associated with viral suppression at 12 months using multivariable logistic regression.ResultsWe enrolled 379 HIV-positive children and 363 caregivers. Among all children at enrollment, viral suppression and undetectability was 78.4% and 73.9%, respectively, improving to 90.2% and 87.3% at 12 months. In mITT and weighted mITT analyses, there was no significant difference in children’s 12-month viral suppression between FCCM and SOC groups (89.2% and 91.6%, respectively). Using imputed ITT, there was a modest increase in 12-month viral suppression in FCCM versus SOC children (79.7% and 69.8%, respectively, p = 0.051) and 12-month undetectability (78.7% and 65.7%, respectively, p = 0.015). Among the 255 children suppressed at enrollment, more FCCM versus SOC children (98.0% versus 95.3%) were suppressed at 12-months, but this was not statistically significant in mITT or weighted mITT analyses, with a marginally significant difference using imputed mITT analysis (p = 0.042). A higher proportion of children suppressed at enrollment had undetectable viral load at 12 months in FCCM versus SOC children (98.0% versus 92.5%), a statistically significant difference across analytical methods. Among the 61 children unsuppressed at enrollment, achieving suppression was higher among SOC versus FCCM children, but this difference was not statistically significant and included only 38 children; and there were no significant differences in detectable viral load at 12 months.There were no significant differences between study groups in retention or ART adherence at 12 months for children or caregivers. Factors associated with lack of viral suppression/detectability at 12 months included lack of viral suppression at enrollment and having a younger caregiver (age <25 years).ConclusionsFCCM in Eswatini was associated with a modest increase in viral suppression/undetectability at 12-months; 12-month retention and adherence did not differ by study group for children or caregivers. High levels of suppression and retention in both groups may have limited our ability to detect a difference.Trial registrationNCT03397420; ClinicalTrials.gov.

Linear Mixed Models for Longitudinal Data with Nonrandom Dropouts

Longitudinal studies represent one of the principal research strategies employed in medical and social research. These studies are the most appropriate for studying individual change over time. The prematurely withdrawal of some subjects from the study (dropout) is termed nonrandom when the probability of missingness depends on the missing value. Nonrandom dropout is common phenomenon associated with longitudinal data and it complicates statistical inference. Linear mixed effects model is used to fit longitudinal data in the presence of nonrandom dropout. The stochastic EM algorithm is developed to obtain the model parameter estimates. Also, parameter estimates of the dropout model have been obtained. Standard errors of estimates have been calculated using the developed Monte Carlo method. All these methods are applied to two data sets.

A shared parameter mixture model for longitudinal income data with missing responses and zero rounding

SummaryThe analysis of longitudinal income data is often made challenging for several reasons. For example, in a national Australian survey on income over time, a non‐negligible proportion of responses are missing, and it is believed the missingness mechanism is non‐ignorable. Also, there are a large number of reported zero incomes, some of which may be true zeros (corresponding to individuals who legitimately do not earn an income), while some may be false zeros (corresponding to individuals choosing to round their income to zero). We propose a new shared parameter mixture (SPM) model for analysing semi‐continuous longitudinal income data, which addresses the two challenges of income non‐response and zero rounding. This is accomplished by jointly modelling an individual's underlying income together with the probability of missingness and rounding to zero, where both probabilities are permitted to vary in a smooth manner with their underlying non‐zero income. Applying the SPM model to the Australian income survey reveals that on average, older female individuals and individuals with a long‐term health condition are considerably less likely to earn an income, while income tended to be highest for male individuals on fixed‐term/permanent job contracts between ages 50 and 60. Furthermore there is evidence of both zero rounding, and conditional on the assumed missingness mechanism, individuals with incomes at the higher and lower ends are more likely to not report their income.

A Selection Model for Longitudinal Data with Non-Ignorable Non-Monotone Missing Values

Missing values are not uncommon in longitudinal data studies. Missingness could be due to withdrawal from the study (dropout) or intermittent. The missing data mechanism is termed non-ignorable if the probability of missingness depends on the unobserved (missing) observations. This paper presents a model for continuous longitudinal data with non-ignorable non-monotone missing values. Two separate models, for the response and missingness, are assumed. The response is modeled as multivariate nor mal whereas the binomial model for missingness process. Parameters in the adopted model are estimated using the stochastic EM algorithm. The proposed model (approach) is then applied to an example from the International Breast Cancer Study Group.

Semiparametric regression on cumulative incidence function with interval-censored competing risks data and missing event types.

SummaryCompeting risk data are frequently interval-censored, that is, the exact event time is not observed but only known to lie between two examination time points such as clinic visits. In addition to interval censoring, another common complication is that the event type is missing for some study participants. In this article, we propose an augmented inverse probability weighted sieve maximum likelihood estimator for the analysis of interval-censored competing risk data in the presence of missing event types. The estimator imposes weaker than usual missing at random assumptions by allowing for the inclusion of auxiliary variables that are potentially associated with the probability of missingness. The proposed estimator is shown to be doubly robust, in the sense that it is consistent even if either the model for the probability of missingness or the model for the probability of the event type is misspecified. Extensive Monte Carlo simulation studies show good performance of the proposed method even under a large amount of missing event types. The method is illustrated using data from an HIV cohort study in sub-Saharan Africa, where a significant portion of events types is missing. The proposed method can be readily implemented using the new function ciregic_aipw in the R package intccr.

A multiple imputation-based sensitivity analysis approach for data subject to missing not at random.

Missingness mechanism is in theory unverifiable based only on observed data. If there is a suspicion of missing not at random, researchers often perform a sensitivity analysis to evaluate the impact of various missingness mechanisms. In general, sensitivity analysis approaches require a full specification of the relationship between missing values and missingness probabilities. Such relationship can be specified based on a selection model, a pattern-mixture model or a shared parameter model. Under the selection modeling framework, we propose a sensitivity analysis approach using a nonparametric multiple imputation strategy. The proposed approach only requires specifying the correlation coefficient between missing values and selection (response) probabilities under a selection model. The correlation coefficient is a standardized measure and can be used as a natural sensitivity analysis parameter. The sensitivity analysis involves multiple imputations of missing values, yet the sensitivity parameter is only used to select imputing/donor sets. Hence, the proposed approach might be more robust against misspecifications of the sensitivity parameter. For illustration, the proposed approach is applied to incomplete measurements of level of preoperative Hemoglobin A1c, for patients who had high-grade carotid artery stenosisa and were scheduled for surgery. A simulation study is conducted to evaluate the performance of the proposed approach.

A tractable method to account for high-dimensional nonignorable missing data in intensive longitudinal data.

Despite the need for sensitivity analysis to nonignorable missingness in intensive longitudinal data (ILD), such analysis is greatly hindered by novel ILD features, such as large data volume and complex nonmonotonic missing-data patterns. Likelihood of alternative models permitting nonignorable missingness often involves very high-dimensional integrals, causing curse of dimensionality and rendering solutions computationally prohibitive to obtain. We aim to overcome this challenge by developing a computationally feasible method, nonlinear indexes of local sensitivity to nonignorability (NISNI). We use linear mixed effects models for the incomplete outcome and covariates. We use Markov multinomial models to describe complex missing-data patterns and mechanisms in ILD, thereby permitting missingness probabilities to depend directly on missing data. Using a second-order Taylor series to approximate likelihood under nonignorability, we develop formulas and closed-form expressions for NISNI. Our approach permits the outcome and covariates to be missing simultaneously, as is often the case in ILD, and can capture U-shaped impact of nonignorability in the neighborhood of the missing at random model without fitting alternative models or evaluating integrals. We evaluate performance of this method using simulated data and real ILD collected by the ecological momentary assessment method.

A multiply robust Mann-Whitney test for non-randomised pretest-posttest studies with missing data

ABSTRACTPretest-posttest studies are a commonly used design by social scientists, medical and health researchers to examine the effect of a treatment or an intervention. We propose an empirical likelihood based Mann-Whitney test on the equality of the response distribution functions of the treatment and control arms for non-randomised pretest-posttest studies with missing responses. The proposed test is multiply robust in the sense that multiple working models can be postulated for the propensity score of treatment assignment, the missingness probability and the outcome regression, and the validity of the test only requires certain combinations of the working models to be correctly specified. Performances of the proposed test are examined through an application to the dataset from AIDS Clinical Trials Group Protocol 175 and simulation studies.

An improvement on the efficiency of complete-case-analysis with nonignorable missing covariate data

This paper develops a weighted composite quantile regression method for linear models where some covariates are missing not at random but the missingness is conditionally independent of the response variable. It is known that complete case analysis (CCA) is valid under these missingness assumptions. By fully utilizing the information from incomplete data, empirical likelihood-based weights are obtained to conduct the weighted composite quantile regression. Theoretical results show that the proposed estimator is more efficient than the CCA one if the probability of missingness on the fully observed variables is correctly specified. Besides, the proposed algorithm is computationally simple and easy to implement. The methodology is illustrated on simulated data and a real data set.