Nonignorable Nonresponse Research Articles

An Improved Product Estimator for Finite Population Mean in the Presence of Nonignorable Nonresponse

.

Rejoinder: Building a Paradigm That Allows for the Possibility of Non-Ignorable Nonresponse

Rejoinder: Building a Paradigm That Allows for the Possibility of Non-Ignorable Nonresponse

Learning from missing data with the binary latent block model

Missing data can be informative. Ignoring this information can lead to misleading conclusions when the data model does not allow information to be extracted from the missing data. We propose a co-clustering model, based on the binary Latent Block Model, that aims to take advantage of this nonignorable nonresponses, also known as Missing Not At Random data. A variational expectation–maximization algorithm is derived to perform inference and a model selection criterion is presented. We assess the proposed approach on a simulation study, before using our model on the voting records from the lower house of the French Parliament, where our analysis brings out relevant groups of MPs and texts, together with a sensible interpretation of the behavior of non-voters.

Identification and Estimation of Graphical Models with Nonignorable Nonresponse

We study the identification and estimation of graphical models with nonignorable nonresponse. An observable variable correlated to nonresponse is added to identify the mean of response for the unidentifiable model. An approach to estimating the marginal mean of response is proposed, based on simulation imputation methods which are introduced for a variety of models including linear, generalized linear, and monotone nonlinear models. The proposed mean estimators are N -consistent, where N is the sample size. Finite sample simulations confirm the effectiveness of the proposed method. Sensitivity analysis for the untestable assumption on our augmented model is also conducted. A real data example is employed to illustrate the use of the proposed methodology.

Bayesian Scalar on Image Regression With Nonignorable Nonresponse

Medical imaging has become an increasingly important tool in screening, diagnosis, prognosis, and treatment of various diseases given its information visualization and quantitative assessment. The aim of this article is to develop a Bayesian scalar-on-image regression model to integrate high-dimensional imaging data and clinical data to predict cognitive, behavioral, or emotional outcomes, while allowing for nonignorable missing outcomes. Such a nonignorable nonresponse consideration is motivated by examining the association between baseline characteristics and cognitive abilities for 802 Alzheimer patients enrolled in the Alzheimer’s Disease Neuroimaging Initiative 1 (ADNI1), for which data are partially missing. Ignoring such missing data may distort the accuracy of statistical inference and provoke misleading results. To address this issue, we propose an imaging exponential tilting model to delineate the data missing mechanism and incorporate an instrumental variable to facilitate model identifiability followed by a Bayesian framework with Markov chain Monte Carlo algorithms to conduct statistical inference. This approach is validated in simulation studies where both the finite sample performance and asymptotic properties are evaluated and compared with the model with fully observed data and that with a misspecified ignorable missing mechanism. Our proposed methods are finally carried out on the ADNI1 dataset, which turns out to capture both of those clinical risk factors and imaging regions consistent with the existing literature that exhibits clinical significance. Supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement.

Empirical likelihood and Wilks phenomenon for data with nonignorable missing values

AbstractWilks's theorem is useful for constructing confidence regions. When applying the popular empirical likelihood to data with nonignorable nonresponses, Wilks's phenomenon does not hold. This paper unveils that this is caused by the extra estimation of the nuisance parameter in the nonignorable nonresponse propensity. Motivated by this result, we propose an adjusted empirical likelihood for which Wilks's theorem holds. Asymptotic results are presented and supplemented by simulation results for finite sample performance of the point estimators and confidence regions. An analysis of a data set is included for illustration.

Estimated conditional score function for missing mechanism model with nonignorable nonresponse

Missing data mechanism often depends on the values of the responses, which leads to nonignorable nonresponses. In such a situation, inference based on approaches that ignore the missing data mechanism could not be valid. A crucial step is to model the nature of missingness. We specify a parametric model for missingness mechanism, and then propose a conditional score function approach for estimation. This approach imputes the score function by taking the conditional expectation of the score function for the missing data given the available information. Inference procedure is then followed by replacing unknown terms with the related nonparametric estimators based on the observed data. The proposed score function does not suffer from the non-identifiability problem, and the proposed estimator is shown to be consistent and asymptotically normal. We also construct a confidence region for the parameter of interest using empirical likelihood method. Simulation studies demonstrate that the proposed inference procedure performs well in many settings. We apply the proposed method to a data set from research in a growth hormone and exercise intervention study.

A Propensity-score-adjustment Method for Nonignorable Nonresponse

A Propensity-score-adjustment Method for Nonignorable Nonresponse

Power-linear models for incomplete contingency tables with nonignorable non-responses

For categorical data exhibiting nonignorable non-responses, it is well known that maximum likelihood (ML) estimates with a boundary solution are implausible and do not provide a perfect fit to the observed data even for saturated models. We provide the conditions under which ML estimates for the generalized linear model (GLM) with the usual log/logit link function have a boundary solution. These conditions introduce a new GLM with appropriately defined power link functions where its ML estimates resolve the problems arising from a boundary solution and offer useful statistics for identifying the non-response mechanism. This model is applied to a real dataset and compared with Bayesian models.

Partial identification in binary response models with nonignorable nonresponses

This study investigates the identification of parameters in semiparametric binary response models of the form y=1(x′β+v+ε>0) when there are nonignorable nonresponses. We propose an estimation procedure for the identified set, the set of parameters that are observationally indistinguishable from the true value β, based on the special regressor approach of Lewbel (2000). We show that the estimator for the identified set is consistent in the Hausdorff metric.

Imputation of non-ignorable nonresponses for income: analysis of a panel study on Taiwan

In this paper, the two-step generalized estimating equations (GEE) approach developed by Wang and Fitzmaurice (Biom J 2:302–318, 2006) is employed to handle income non-responses in the Panel Study of Family Dynamics survey conducted in Taiwan. In our analysis, we first construct a conditional logit model of the paid work equation by taking the missing patterns into account. We then use the estimation results to impute whether or not the nonresponses were working for pay. For those who were imputed or observed to work for pay, we adopt the two-step GEE method to estimate the income equation. Compared to simply deleting the missing cases, the two-step imputation procedure is found to improve the estimation results.

Test of independence in a [formula omitted] contingency table with nonignorable nonresponse via constrained EM algorithm

Test of independence for 2×2 contingency tables with nonignorable nonresponses is discussed. Dependency assumption between two observed outcomes is required to achieve identification in many models with nonignorable nonresponses in the analysis of 2×2 contingency tables (e.g., [Ma, W.-Q., Geng, Z., Li, X.-T., 2003. Identification of nonresponse mechanisms for two-way contingency tables. Behaviormetrika 30, 125–144]). The assumption is, however, violated under the null hypothesis when implementing the test of independence. In this article, we introduce a new simple assumption to achieve identification. The assumption involves pre-specified parameters. EM algorithms for finding the MLE are numerically unstable when there are nonlinear constraints, which are created by models treating nonignorable nonresponses. In the analysis of contingency tables, estimated values often fall outside the admissible region. We propose a new EM type algorithm to stably calculate the constrained MLE, and apply it to make the test of independence for a real data set (crime data). We compare empirical performance among several testing procedures for independence. It turns out that the new EM type algorithm works well to calculate the MLE, and that the nonignorable model with the correctly specified parameters performs best while the conventional chi-square test of independence works fairly well.

A Simple Imputation Method for Longitudinal Studies with Non‐ignorable Non‐responses

Missing data are a common problem in longitudinal studies in the health sciences. Motivated by data from the Muscatine Coronary Risk Factor (MCRF) study, a longitudinal study of obesity, we propose a simple imputation method for handling non-ignorable non-responses (i.e., when non-response is related to the specific values that should have been obtained) in longitudinal studies with either discrete or continuous outcomes. In the proposed approach, two regression models are specified; one for the marginal mean of the response, the other for the conditional mean of the response given non-response patterns. Statistical inference for the model parameters is based on the generalized estimating equations (GEE) approach. An appealing feature of the proposed method is that it can be readily implemented using existing, widely-available statistical software. The method is illustrated using longitudinal data on obesity from the MCRF study.

Using Number of Failed Contact Attempts to Adjust for Non-Ignorable Non-Response

SummaryWe present a general method of adjustment for non-ignorable non-response in studies where one or more further attempts are made to contact initial non-responders. A logistic regression model relates the probability of response at each contact attempt to covariates and outcomes of interest. We assume that the effect of these covariates and outcomes on the probability of response is the same at all contact attempts. Knowledge of the number of contact attempts enables estimation of the model by using only information from the respondents and the number of non-responders. Three approaches for fitting the response models and estimating parameters of substantive interest and their standard errors are compared: a modified conditional likelihood method in which the fitted inverse probabilities of response are used in weighted analyses for the outcomes of interest, an EM procedure with the Louis formula and a Bayesian approach using Markov chain Monte Carlo methods. We further propose the creation of several sets of weights to incorporate uncertainty in the probability weights in subsequent analyses. Our methods are applied as a sensitivity analysis to a postal survey of symptoms in Persian Gulf War veterans and other servicemen.

Interval Estimation for Log-Linear Models with One Variable Subject to Non-Ignorable Non-Response

SummaryLog-linear models for multiway contingency tables where one variable is subject to non-ignorable non-response will often yield boundary solutions, with the probability of non-respondents being classified in some cells of the table estimated as 0. The paper considers the effect of this non-standard behaviour on two methods of interval estimation based on the distribution of the maximum likelihood estimator. The first method relies on the estimator being approximately normally distributed with variance equal to the inverse of the information matrix. It is shown that the information matrix is singular for boundary solutions, but intervals can be calculated after a simple transformation. For the second method, based on the bootstrap, asymptotic results suggest that the coverage properties may be poor for boundary solutions. Both methods are compared with profile likelihood intervals in a simulation study based on data from the British General Election Panel Study. The results of this study indicate that all three methods perform poorly for a parameter of the non-response model, whereas they all perform well for a parameter of the margin model, irrespective of whether or not there is a boundary solution.

A Bayesian Hierarchical Model for Categorical Data with Nonignorable Nonresponse

Log-linear models have been shown to be useful for smoothing contingency tables when categorical outcomes are subject to nonignorable nonresponse. A log-linear model can be fit to an augmented data table that includes an indicator variable designating whether subjects are respondents or nonrespondents. Maximum likelihood estimates calculated from the augmented data table are known to suffer from instability due to boundary solutions. Park and Brown (1994, Journal of the American Statistical Association 89, 44-52) and Park (1998, Biometrics 54, 1579-1590) developed empirical Bayes models that tend to smooth estimates away from the boundary. In those approaches, estimates for nonrespondents were calculated using an EM algorithm by maximizing a posterior distribution. As an extension of their earlier work, we develop a Bayesian hierarchical model that incorporates a log-linear model in the prior specification. In addition, due to uncertainty in the variable selection process associated with just one log-linear model, we simultaneously consider a finite number of models using a stochastic search variable selection (SSVS) procedure due to George and McCulloch (1997, Statistica Sinica 7, 339-373). The integration of the SSVS procedure into a Markov chain Monte Carlo (MCMC) sampler is straightforward, and leads to estimates of cell frequencies for the nonrespondents that are averages resulting from several log-linear models. The methods are demonstrated with a data example involving serum creatinine levels of patients who survived renal transplants. A simulation study is conducted to investigate properties of the model.

A pattern-mixture model for longitudinal binary responses with nonignorable nonresponse.

Longitudinal studies frequently incur outcome-related nonresponse. In this article, we discuss a likelihood-based method for analyzing repeated binary responses when the mechanism leading to missing response data depends on unobserved responses. We describe a pattern-mixture model for the joint distribution of the vector of binary responses and the indicators of nonresponse patterns. Specifically, we propose an extension of the multivariate logistic model to handle nonignorable nonresponse. This method yields estimates of the mean parameters under a variety of assumptions regarding the distribution of the unobserved responses. Because these models make unverifiable identifying assumptions, we recommended conducting sensitivity analyses that provide a range of inferences, each of which is valid under different assumptions for nonresponse. The methodology is illustrated using data from a longitudinal study of obesity in children.

Analysis of Multivariate Longitudinal Outcomes With Nonignorable Dropouts and Missing Covariates

This article analyzes changes in treatment practices in outpatient methadone treatment units from a national panel study. The analysis of this dataset is challenging due to several difficulties, including multiple longitudinal outcomes, nonignorable nonresponses, and missing covariates. Specifically, the data included several variables that measure the effectiveness of methadone treatment practices for each unit. A substantial percentage of units (33%%) did not respond during the follow-up. These dropout units tended to be units with less effective treatment practices; the dropout mechanism thus may be nonignorable. Finally, the time-varying covariates for the units that dropped out were missing at the time of dropout. A valid analysis hence needs to address these three issues simultaneously. Our approach assumes that the observed outcomes measure a latent variable (e.g., treatment practice effectiveness) with error. We model the relationship between this latent variable and covariates using a linear mixed model. To account for nonignorable dropouts, we apply a selection model in which the dropout probability depends on the latent variable. Finally, we accommodate missing time-varying covariates by modeling them using a transition model. In view of multidimensional integration in full-likelihood estimation, we develop the EM algorithm to estimate the model parameters. We apply the proposed approach to the methadone treatment practices data. Our results show that methadone treatment practices have improved in the last decade. Our results are also useful for identifying the types of methadone treatment units that need improvement.

Allowing for Non-Ignorable Non-Response in the Analysis of Voting Intention Data

SUMMARY We apply some log-linear modelling methods, which have been proposed for treating non-ignorable non-response, to some data on voting intention from the British General Election Survey. We find that, although some non-ignorable non-response models fit the data very well, they may generate implausible point estimates and predictions. Some explanation is provided for the extreme behaviour of the maximum likelihood estimates for the most parsimonious model. We conclude that point estimates for such models must be treated with great caution. To allow for the uncertainty about the non-response mechanism we explore the use of profile likelihood inference and find the likelihood surfaces to be very flat and the interval estimates to be very wide. To reduce the width of these intervals we propose constraining confidence regions to values where the parameters governing the non-response mechanism are plausible and study the effect of such constraints on inference. We find that the widths of these intervals are reduced but remain wide.

An Approach to Categorical Data with Nonignorable Nonresponse

Log-linear models have been used to adjust for nonresponse when categorical outcomes are subject to nonignorable nonresponse. The log models are fitted to the data in an augmented frequency table in which one index corresponds to whether the subject is a respondent. Park and Brown (1994, Journal of the American Statistical Association 89, 44-52) proposed a pseudo-Bayesian method that has the effect of smoothing the unobserved cell frequencies. Their approach assigns prior observations only to the unobserved cells. Their method was shown to perform better than the maximum likelihood method, which can produce unstable boundary estimates. Generalizing their approach, we propose a new approach that assigns prior observations to both observed and unobserved cells. Through a simulation study, we compare the proposed approach with Park and Brown's and the maximum likelihood approach.