Censored Data Problems Research Articles

COMPASS: An Open-Source, General-Purpose Software Toolkit for Computational Psychiatry

Mathematical modeling of behavior during a psychophysical task, referred to as “computational psychiatry,” could greatly improve our understanding of mental disorders. One barrier to the broader adoption of computational methods, is that they often require advanced statistical modeling and mathematical skills. Biological and behavioral signals often show skewed or non-Gaussian distributions, and very few toolboxes and analytical platforms are capable of processing such signal categories. We developed the Computational Psychiatry Adaptive State-Space (COMPASS) toolbox, an open-source MATLAB-based software package. This toolbox is easy to use and capable of integrating signals with a variety of distributions. COMPASS has the tools to process signals with continuous-valued and binary measurements, or signals with incomplete—missing or censored—measurements, which makes it well-suited for processing those signals captured during a psychophysical task. After specifying a few parameters in a small set of user-friendly functions, COMPASS allows users to efficiently apply a wide range of computational behavioral models. The model output can be analyzed as an experimental outcome or used as a regressor for neural data and can also be tested using the goodness-of-fit measurement. Here, we demonstrate that COMPASS can replicate two computational behavioral analyses from different groups. COMPASS replicates and can slightly improve on the original modeling results. We also demonstrate the use of COMPASS application in a censored-data problem and compare its performance result with naïve estimation methods. This flexible, general-purpose toolkit should accelerate the use of computational modeling in psychiatric neuroscience.

Toward improved analysis of concentration data: Embracing nondetects.

Various statistical tests on concentration data serve to support decision-making regarding characterization and monitoring of contaminated media, assessing exposure to a chemical, and quantifying the associated risks. However, the routine statistical protocols cannot be directly applied because of challenges arising from nondetects or left-censored observations, which are concentration measurements below the detection limit of measuring instruments. Despite the existence of techniques based on survival analysis that can adjust for nondetects, these are seldom taken into account properly. A comprehensive review of the literature showed that managing policies regarding analysis of censored data do not always agree and that guidance from regulatory agencies may be outdated. Therefore, researchers and practitioners commonly resort to the most convenient way of tackling the censored data problem by substituting nondetects with arbitrary constants prior to data analysis, although this is generally regarded as a bias-prone approach. Hoping to improve the interpretation of concentration data, the present article aims to familiarize researchers in different disciplines with the significance of left-censored observations and provides theoretical and computational recommendations (under both frequentist and Bayesian frameworks) for adequate analysis of censored data. In particular, the present article synthesizes key findings from previous research with respect to 3 noteworthy aspects of inferential statistics: estimation of descriptive statistics, hypothesis testing, and regression analysis. Environ Toxicol Chem 2018;37:643-656. © 2017 SETAC.

Machine Learning for Predictive Maintenance: A Multiple Classifier Approach

In this paper, a multiple classifier machine learning (ML) methodology for predictive maintenance (PdM) is presented. PdM is a prominent strategy for dealing with maintenance issues given the increasing need to minimize downtime and associated costs. One of the challenges with PdM is generating the so-called “health factors,” or quantitative indicators, of the status of a system associated with a given maintenance issue, and determining their relationship to operating costs and failure risk. The proposed PdM methodology allows dynamical decision rules to be adopted for maintenance management, and can be used with high-dimensional and censored data problems. This is achieved by training multiple classification modules with different prediction horizons to provide different performance tradeoffs in terms of frequency of unexpected breaks and unexploited lifetime, and then employing this information in an operating cost-based maintenance decision system to minimize expected costs. The effectiveness of the methodology is demonstrated using a simulated example and a benchmark semiconductor manufacturing maintenance problem.

Multivariate Gaussian Mixture Model Based Clustering with Truncated and Censored Data

Gaussian mixture model with the standard EM algorithm is a widely applicable approach for science computing and research. The truncation or censoring phenomenon is most frequently appearing in data collection work of science research. So this motivated interest in solving the truncated or censored data problem. This paper studied a truncated and censored EM algorithm for fltting multivariate Gaussian mixture models to data that is truncated and censored simultaneously. A split and merge operation was proposed to improve its sensitive to initialization. We verifled this method on some synthetic and real data sets and got some good result.

Instrumental variable additive hazards models.

Instrumental variable (IV) methods are popular in non-experimental studies to estimate the causal effects of medical interventions. These approaches allow for the consistent estimation of treatment effects even if important confounding factors are unobserved. Despite the increasing use of these methods, there have been few extensions of IV methods to censored data problems. In this article, we discuss challenges in applying IV techniques to the proportional hazards model and demonstrate the utility of the additive hazards formulation for IV analyses with censored data. Assuming linear structural equation models for the hazard function, we develop a closed-form, two-stage estimator for the causal effect in the additive hazard model. The methods permit both continuous and discrete exposures, and enable the estimation of causal relative survival measures. The asymptotic properties of the estimators are derived and the resulting inferences are shown to perform well in simulation studies and in an application to a data set on the effectiveness of a novel chemotherapeutic agent for colon cancer.

Bayesian Survival Analysis of Head and Neck Cancer Data Using Lognormal Model

The paper considers a lognormal model for the survival times and obtains a Bayes solution by means of Gibbs sampler algorithm when the priors for the parameters are vague. The formulation given in the paper is mainly focused for censored data problems though it is equally well applicable for complete data scenarios as well. For the purpose of numerical illustration, we considered two real data sets on head and neck cancer patients when they have been treated using either radiotherapy or chemotherapy followed by radiotherapy. The paper not only compares the survival functions for the two therapies assuming a lognormal model but also provides a model compatibility study based on predictive simulation results so that the choice of lognormal model can be justified for the two data sets. The ease of our analysis as compared to an earlier approach is certainly an advantage.

Research on a Kind of Distribution Selection Method for Right Censored Data Based on Data Expansion Algorithm

A kind of distribution selection method is researched for the right censored data situation. It can be widely applied in life sciences, medicine and health, reliability engineering, aerospace , social sciences and other related fields. By using a common PL estimation curve as a basic curve, a kind of distribution selection method is proposed to solve the parameter distribution problem of right censored data. This method can meet the selection requirement of data distribution in reliability assessment and other data analysis. It can guide the engineer and technician to choose a reasonable choice of the fitting of the data distribution, and it can improve the accuracy and efficiency of the reliability assessment.

Estimation of colorectal adenoma recurrence with dependent censoring

BackgroundDue to early colonoscopy for some participants, interval-censored observations can be introduced into the data of a colorectal polyp prevention trial. The censoring could be dependent of risk of recurrence if the reasons of having early colonoscopy are associated with recurrence. This can complicate estimation of the recurrence rate.MethodsWe propose to use midpoint imputation to convert interval-censored data problems to right censored data problems. To adjust for potential dependent censoring, we use information from auxiliary variables to define risk groups to perform the weighted Kaplan-Meier estimation to the midpoint imputed data. The risk groups are defined using two risk scores derived from two working proportional hazards models with the auxiliary variables as the covariates. One is for the recurrence time and the other is for the censoring time. The method described here is explored by simulation and illustrated with an example from a colorectal polyp prevention trial.ResultsWe first show that midpoint imputation under an assumption of independent censoring will produce an unbiased estimate of recurrence rate at the end of the trial, which is often the main interest of a colorectal polyp prevention trial, and then show in simulations that the weighted Kaplan-Meier method using the information from auxiliary variables based on the midpoint imputed data can improve efficiency in a situation with independent censoring and reduce bias in a situation with dependent censoring compared to the conventional methods, while estimating the recurrence rate at the end of the trial.ConclusionThe research in this paper uses midpoint imputation to handle interval-censored observations and then uses the information from auxiliary variables to adjust for dependent censoring by incorporating them into the weighted Kaplan-Meier estimation. This approach can handle a situation with multiple auxiliary variables by deriving two risk scores from two working PH models. Although the idea of this approach might appear simple, the results do show that the weighted Kaplan-Meier approach can gain efficiency and reduce bias due to dependent censoring.

Using integrated weighted survival difference for the two-sample censored data problem

For the two-sample censored data problem, Pepe and Fleming [Pepe, M.S., Fleming, T.R., 1989. Weighted Kaplan–Meier statistics: A class of distance tests for censored survival data. Biometrics 45, 497–507] introduced the weighted Kaplan–Meier (WKM) statistics. From these statistics we define stochastic processes which can be approximated by zero-mean martingales. Conditional distributions of the processes, given data, can be easily approximated through simulation techniques. Based on comparison of these processes, we construct a supremum test to assess the model adequacy. Monte Carlo simulations are conducted to evaluate and compare the size and power properties of the proposed test to the WKM and the log-rank tests. The procedures are illustrated using real data.

Multiple imputation for interval censored data with auxiliary variables

We propose a non-parametric multiple imputation scheme, NPMLE imputation, for the analysis of interval censored survival data. Features of the method are that it converts interval-censored data problems to complete data or right censored data problems to which many standard approaches can be used, and that measures of uncertainty are easily obtained. In addition to the event time of primary interest, there are frequently other auxiliary variables that are associated with the event time. For the goal of estimating the marginal survival distribution, these auxiliary variables may provide some additional information about the event time for the interval censored observations. We extend the imputation methods to incorporate information from auxiliary variables with potentially complex structures. To conduct the imputation, we use a working failure-time proportional hazards model to define an imputing risk set for each censored observation. The imputation schemes consist of using the data in the imputing risk sets to create an exact event time for each interval censored observation. In simulation studies we show that the use of multiple imputation methods can improve the efficiency of estimators and reduce the effect of missing visits when compared to simpler approaches. We apply the approach to cytomegalovirus shedding data from an AIDS clinical trial, in which CD4 count is the auxiliary variable.

A Bayesian analysis of mouse infectivity data to evaluate the effectiveness of using ultraviolet light as a drinking water disinfectant

Modelling disinfectant performance using Bayesian hierarchical methods can overcome problems with traditional methods and lead to improved estimates. Animal and cell-culture assays are used to estimate the degree of inactivation of a microorganism produced by a given disinfectant dose. Assay data traditionally are analyzed with logistic model or most probable number (MPN) method. These methods are limited particularly when assays show all (or no) animals or cells to be infected—estimates are reported as greater than (or less than) a measurement limit (i.e., censored data). The proposed Bayesian approach (1) properly models the propagation of uncertainty through the data analysis/modelling process, resulting in reduced model uncertainty, and (2) uses appropriate probability distribution models for the response variables, avoiding the censored data problem and more accurately describing statistical error when estimating dose–response behavior. This paper applies the Bayesian hierarchical models to logistic and MPN data from published papers for the ultraviolet (UV) inactivation of Cryptosporidium. Results are compared to those from three alternative models. The Bayesian model estimates a significantly lower UV dose for a given level of Cryptosporidium inactivation than the alternative models, due mainly to the reduced model uncertainty.

A Stochastic Filtering Approach To Survival Analysis

We connect some basic issues in survival analysis in biostatistics with estimation and convergence theories in stochastic filtering. Viewing censored data problems through a filtering perspective, we can derive estimators expressed using stochastic integral/differential equations. We then study statistical asymptotic using convergence theory of stochastic equations. We illustrate the effectiveness of such a program by revisiting the right censored and the doubly censored data problems.

Estimating medical costs with censored data

Incompleteness of follow-up data is a common problem in estimating medical costs. Naive analysis using summary statistics on the collected data can result in severely misleading statistical inference. This paper focuses on the problem of estimating the mean medical cost from a sample of individuals whose medical costs may be right censored. A class of weighted estimators which account appropriately for censoring are introduced. Our estimators are shown to be consistent and asymptotically normal with easily estimated variances. The efficiency of these estimators is studied with the goal of finding as efficient an estimator for the mean medical cost as is feasible. Extensive simulation studies are used to show that our estimators perform well in finite samples, even with heavily censored data, for a variety of circumstances. The methods are applied to a set of cost data from a cardiology trial conducted by the Duke University Medical Center. Extensions to other censored data problems are also discussed.

A Method for Testing Nested Point Null Hypotheses Using Multiple Bayes Factor

As a flexible Bayesian test criterion for nested point null hypotheses, asymmetric and multiple Bayes factors are introduced in the form of a modified Savage-Dickey density ratio. This leads to a simple method for obtaining pairwise comparisons of hypotheses in a statistical experiment with a partition on the parameter space. The method is derived from the fact that in general, the asymmetric Bayes factor can be written as the product of the Savage-Dickey ratio and a correction factor where both terms are easily estimated by means of posterior simulation. Analyses of a censored data problem and a serial correlation problem are illustrated for the method. For these cases, the method is straightforward for specifying distributionally and to implement computationally, with output readily adapted for required tests.

Analysis of structural equation models with censored or truncated data via EM algorithm

In this article, we delineate the analysis of structural equation models with censored data or truncated data with the number of unmeasured subjects under the framework of the generalized censoring mechanism (GCM), one of the most popular non-ignorable missing mechanisms. An EM algorithm for maximum-likelihood estimation in structural equation models with censored data will be presented. By reformulating the censored-data problem into non-ignorable missing-data problem, the maximization procedure is shown to be simplified. A simulated data set will be used as an illustrative example.

Weighted Mean Survival Test Statistics: A Class of Distance Tests for Censored Survival Data

Summary A class of test statistics is introduced which is sensitive against the alternative of stochastic ordering in the two-sample censored data problem. The test statistics for evaluating a cumulative weighted difference in survival distributions are developed while taking into account the imbalances in base-line covariates between two groups. This procedure can be used to test the null hypothesis of no treatment effect, especially when base-line hazards cross and prognostic covariates need to be adjusted. The statistics are semiparametric, not rank based, and can be written as integrated weighted differences in estimated survival functions, where these survival estimates are adjusted for covariate imbalances. The asymptotic distribution theory of the tests is developed, yielding test procedures that are shown to be consistent under a fixed alternative. The choice of weight function is discussed and relies on stability and interpretability considerations. An example taken from a clinical trial for acquired immune deficiency syndrome is presented.

Improving Efficiency of Estimation and Testing for Censored Data Using Auxiliary Covariates

This research improves upon standard nonparametric survival estimation and testing in the two sample censored data problem. First, a survival estimator which uses prognostic covariates to more precisely get at the underlying survival curves in the presence of censoring is defined. This estimator has a smaller asymptotic variance than the usual Kaplan-Meier estimator and reduces to the Kaplan-Meier when the covariate is not prognostic or no censoring occurs. Also, this estimate remains consistent when the incorporated covariate contains information about both the censoring process and survival. Since the Kaplan-Meier is biased in this situation the author's estimate is recommended. Using this more efficient survival estimator, a more powerful test statistic for detecting differences in clinical trials is then defined. Sequential methods for using this test are also discussed. This research allows researchers to study primary endpoints rather than taking the risk of substituting a surrogate marker which may...

Identifiability of Bivariate Survival Curves from Censored Data

Abstract We show that the survival curve is identifiable in bivariate censored data problems under weaker independence assumptions than have commonly been made. The common assumption has been mutual independence of (T 1, T 2) and (Z 1, Z 2), where (T 1, T 2) is the true survival vector, (Z 1, Z 2) is a nuisance censoring vector, and bivariate right-censored data is observed. We show that the distribution of (T 1, T 2) is identifiable under weaker, conditional independence assumptions for distributions with full support. Bivariate survival analysis is a more powerful analysis tool than univariate analysis if multiple, possibly related, times are of interest. The mutual independence model has become popular as a nonparametric way of analyzing such data. Analysis of the bivariate problem and analogy with univariate models are used to show that the conditional independence model is more widely applicable as a general nonparametric model for bivariate survival data.

Testing unbiasedness in futures markets: A clarification

where 5,,^ is the spot price at maturity time, m, for the j-th futures contract, and Fi,m-n is the futures price of the i-th contract n days prior to maturity time, m. The empirical test of the unbiasedness involves the use of an OLS estimate of the eq. (1) to perform an F test on the joint null hypothesis: a = 0 and b = 1. The collective evidence from previous studies suggest that a > 0 and b < I; i.e., the unbiasedness hypothesis is rejected in the futures markets. In a recent article of this Journal, Maberly (1985) questions the validity of an OLS procedure estimating eq. (1). Maberly argues that the OLS estimates are biased as a result of an ex-post censored data problem related to the forecast error. Ex-post the forecast error and the forecast are negatively correlated. More recently, Elam and Dixon (1988) disagree with this characterization of the source of the problem. They argue that if the spot/futures price changes follow a random walk, eq. (1) turns in a regression model with a lagged dependent variable and it is then this lagged dependent variable which has caused an excessively higher number of rejections of the unbiasedness hypothesis in previous studies. Based on Elam and Dixon (1988), if the spot price is described by the following nonstationary process,

Weighted Kaplan-Meier Statistics: A Class of Distance Tests for Censored Survival Data

A class of statistics based on the integrated weighted difference in Kaplan-Meier estimators is introduced for the two-sample censored data problem. With positive weight functions these statistics are intuitive for and sensitive against the alternative of stochastic ordering. The standard weighted log-rank statistics are not always sensitive against this alternative, particularly if the hazard functions cross. Qualitative comparisons are made between the weighted log-rank statistics and these weighted Kaplan-Meier (WKM) statistics. A statement of null asymptotic distribution theory is given and the choice of weight function is discussed in some detail. Results from small-sample simulation studies indicate that these statistics compare favorably with the log-rank procedure even under the proportional hazards alternative, and may perform better than it under the crossing hazards alternative.