Left-censored Data Research Articles

High-Dimensional Censored Regression via the Penalized Tobit Likelihood

High-dimensional regression and regression with a left-censored response are each well-studied topics. In spite of this, few methods have been proposed which deal with both of these complications simultaneously. The Tobit model—long the standard method for censored regression in economics—has not been adapted for high-dimensional regression at all. To fill this gap and bring up-to-date techniques from high-dimensional statistics to the field of high-dimensional left-censored regression, we propose several penalized Tobit models. We develop a fast algorithm which combines quadratic majorization with coordinate descent to compute the penalized Tobit solution path. Theoretically, we analyze the Tobit lasso and Tobit with a folded concave penalty, bounding the estimation loss for the former and proving that a local linear approximation estimator for the latter possesses the strong oracle property. Through an extensive simulation study, we find that our penalized Tobit models provide more accurate predictions and parameter estimates than other methods on high-dimensional left-censored data. We use a penalized Tobit model to analyze high-dimensional left-censored HIV viral load data from the AIDS Clinical Trials Group and identify potential drug resistance mutations in the HIV genome. A supplementary file contains intermediate theoretical results and technical proofs.

Statistical Power of Goodness-of-Fit Tests for Type~I Left-Censored Data

Type I doubly left-censored data often arise in environmental studies. In this paper, the power of the most frequently used goodness-of-fit tests (Kolmogorov-Smirnov, Cramér-von Mises, Anderson-Darling) is studied considering various sample sizes and degrees of censoring. Attention is paid to testing of the composite hypothesis that the data has a specific distribution with unknown parameters, which are estimated using the maximum likelihood method. Performance of the tests is assessed by means of Monte Carlo simulations for several distributions, specifically the Weibull, lognormal and gamma distributions, which are among the most frequently used distributions for modelling of environmental data. Finally, the tests are used for identification of the distribution of musk concentrations if fish tissue.

Risk assessment of N-nitrosamines in food.

EFSA was asked for a scientific opinion on the risks to public health related to the presence of N-nitrosamines (N-NAs) in food. The risk assessment was confined to those 10 carcinogenic N-NAs occurring in food (TCNAs), i.e. NDMA, NMEA, NDEA, NDPA, NDBA, NMA, NSAR, NMOR, NPIP and NPYR. N-NAs are genotoxic and induce liver tumours in rodents. The in vivo data available to derive potency factors are limited, and therefore, equal potency of TCNAs was assumed. The lower confidence limit of the benchmark dose at 10% (BMDL10) was 10μg/kg body weight (bw) perday, derived from the incidence of rat liver tumours (benign and malignant) induced by NDEA and used in a margin of exposure (MOE) approach. Analytical results on the occurrence of N-NAs were extracted from the EFSA occurrence database (n=2,817) and the literature (n=4,003). Occurrence data were available for five food categories across TCNAs. Dietary exposure was assessed for two scenarios, excluding (scenario 1) and including (scenario 2) cooked unprocessed meat and fish. TCNAs exposure ranged from 0 to 208.9ng/kg bw perday across surveys, age groups and scenarios. 'Meat and meat products' is the main food category contributing to TCNA exposure. MOEs ranged from 3,337 to 48 at the P95 exposure excluding some infant surveys with P95 exposure equal to zero. Two major uncertainties were (i) the high number of left censored data and (ii) the lack of data on important food categories. The CONTAM Panel concluded that the MOE for TCNAs at the P95 exposure is highly likely (98-100% certain) to be less than 10,000 for all age groups, which raises a health concern.

Robust smoothing of left-censored time series data with a dynamic linear model to infer SARS-CoV-2 RNA concentrations in wastewater

<abstract><p>Wastewater sampling for the detection and monitoring of SARS-CoV-2 has been developed and applied at an unprecedented pace, however uncertainty remains when interpreting the measured viral RNA signals and their spatiotemporal variation. The proliferation of measurements that are below a quantifiable threshold, usually during non-endemic periods, poses a further challenge to interpretation and time-series analysis of the data. Inspired by research in the use of a custom Kalman smoother model to estimate the true level of SARS-CoV-2 RNA concentrations in wastewater, we propose an alternative left-censored dynamic linear model. Cross-validation of both models alongside a simple moving average, using data from 286 sewage treatment works across England, allows for a comprehensive validation of the proposed approach. The presented dynamic linear model is more parsimonious, has a faster computational time and is represented by a more flexible modelling framework than the equivalent Kalman smoother. Furthermore we show how the use of wastewater data, transformed by such models, correlates more closely with regional case rate positivity as published by the Office for National Statistics (ONS) Coronavirus (COVID-19) Infection Survey. The modelled output is more robust and is therefore capable of better complementing traditional surveillance than untransformed data or a simple moving average, providing additional confidence and utility for public health decision making.</p> <p>La détection et la surveillance du SARS-CoV-2 dans les eaux usées ont été développées et réalisées à un rythme sans précédent, mais l'interprétation des mesures de concentrations en ARN viral, et de leurs variations spatio-temporelles, pose question. En particulier, l'importante proportion de mesures en deçà du seuil de quantification, généralement pendant les périodes non endémiques, constitue un défi pour l'analyse de ces séries temporelles. Inspirés par un travail de recherche ayant produit un lisseur de Kalman adapté pour estimer les concentrations réelles en ARN de SARS-CoV-2 dans les eaux usées à partir de ce type de données, nous proposons un nouveau modèle linéaire dynamique avec censure à gauche. Une validation croisée de ces lisseurs, ainsi que d'un simple lissage par moyenne glissante, sur des données provenant de 286 stations d'épuration couvrant l'Angleterre, valide de façon complète l'approche proposée. Le modèle présenté est plus parcimonieux, offre un cadre de modélisation plus flexible et nécessite un temps de calcul réduit par rapport au Lisseur de Kalman équivalent. Les données issues des eaux usées ainsi lissées sont en outre plus fortement corrélées avec le taux d'incidence régional produit par le bureau des statistiques nationales (ONS) Coronavirus Infection Survey. Elles se montrent plus robustes que les données brutes, ou lissées par simple moyenne glissante, et donc plus à même de compléter la surveillance traditionnelle, renforçant ainsi la confiance en l'épidémiologie fondée sur les eaux usées et son utilité pour la prise de décisions de santé publique.</p></abstract>

Statistical Methods for Handling Nondetected Results in Food Chemical Monitoring Data to Improve Food Risk Assessments

Chemical risk assessment is important for risk management, and estimates of chemical exposure must be as accurate as possible. Chemical concentrations in food below the limit of detection are known as nondetects and result in leftcensored data. During statistical analysis, the method used for handling values below the limit of detection is important. Many risk assessors employ widely used substitution methods to treat left-censored data, as recommended by international organizations. The National Institute of Food and Drug Safety Evaluation of South Korea also recommend these methods, which are currently used for chemical exposure assessments. However, these methods have statistical limitations, and international organizations recommend more advanced alternative statistical approaches. In this study, we assessed the validity of currently used statistical methods for handling nondetects. The best method was determined based on a simulation study. In three case studies, we compared the various methods based on the root mean squared error. The data for all case studies were from the same source, to avoid heterogeneity. Across various sample sizes and nondetection rates, the mean and 95th percentile values for all treatment methods were similar. However, “log normal maximum likelihood estimation” method was not suitable for estimating the mean. Risk assessors should consider statistical processing of monitoring data to reduce uncertainty. Currently used substitution methods are effective and easy to apply to large data sets with nondetection rates < 80%. However, advanced statistical methods are required in some circumstances, and national guidelines are needed regarding their use in risk assessments.

Optimal method for determining the intraclass correlation coefficients of urinary biomarkers such as dialkylphosphates from imputed data

Urinary biomarkers are commonly used in epidemiological studies as surrogates or indicators of exposure to chemical substances. Evaluating the reliability of a biomarker is highly important because use of an unreliable marker may lead to misclassification and attenuation bias, resulting in flawed interpretations and conclusions. Although intraclass correlation coefficient (ICC) is regarded as a typical index of test reliability, methods for determining the ICCs of urinary biomarkers have not been standardised, and different methods have been used. This study evaluated different imputation methods for left-censored data, i.e., four imputation or one substitution methods, before calculating ICCs, and at the same time mathematically assessed the impact of the left-censoring proportion on the estimated ICCs. Biomarkers of exposure to organophosphate pesticides, i.e., dialkylphosphates, were used as an example. The Gibbs sampler-based left-censored missing value imputation approach had the best performance for imputation of values below reporting limits, with lower values on Kolmogorov–Smirnov test statistics than other imputation/substitution methods, i.e., a univariate distribution fitting approach, multiple imputation by chained equation, a bootstrap expectation–maximisation algorithm approach, and a single value substitution. In all imputation methods, however, ICCs decreased as censoring rates increased. We propose a method to estimate true ICCs based on mathematical estimation.

Parametric Regression Model Based on Reversed Hazard Rate: An Application to left censored heavy tailed HIV Viral Load Data

Parametric Regression Model Based on Reversed Hazard Rate: An Application to left censored heavy tailed HIV Viral Load Data

Addressing Missing Data in GC × GC Metabolomics: Identifying Missingness Type and Evaluating the Impact of Imputation Methods on Experimental Replication.

Missing data is a significant issue in metabolomics that is often neglected when conducting data preprocessing, particularly when it comes to imputation. This can have serious implications for downstream statistical analyses and lead to misleading or uninterpretable inferences. In this study, we aim to identify the primary types of missingness that affect untargeted metabolomics data and compare strategies for imputation using two real-world comprehensive two-dimensional gas chromatography (GC × GC) data sets. We also present these goals in the context of experimental replication whereby imputation is conducted in a within-replicate-based fashion─the first description and evaluation of this strategy─and introduce an R package MetabImpute to carry out these analyses. Our results conclude that, in these two GC × GC data sets, missingness was most likely of the missing at-random (MAR) and missing not-at-random (MNAR) types as opposed to missing completely at-random (MCAR). Gibbs sampler imputation and Random Forest gave the best results when imputing MAR and MNAR compared against single-value imputation (zero, minimum, mean, median, and half-minimum) and other more sophisticated approaches (Bayesian principal component analysis and quantile regression imputation for left-censored data). When samples are replicated, within-replicate imputation approaches led to an increase in the reproducibility of peak quantification compared to imputation that ignores replication, suggesting that imputing with respect to replication may preserve potentially important features in downstream analyses for biomarker discovery.

남편의 지원이 기혼여성의 노동시장 참여에 미치는 영향

This paper analyzed the effect of husband's support on the labor supply of married women by using data from the KLIPS 17th (2014) survey and additional survey. In the analysis of labor supply, the employment status and working hours model were used, and then the factors of husband's participation in housework sharing were examined. The probit estimation was applied to the employment status model, and the tobit estimation method was applied to the working hours and husband's household share rate model because women's working hours and men's housework hours had the characteristics of left-censored data. As a result of the estimation, it was found that the couple's perception of the progressive gender role and the sharing of domestic labor among men had a positive effect on married women's participation in the labor market. On the other hand, the stronger men's perception of the fair sharing of household labor, the higher the participation rate of household labor. In order to increase Korean women's participation in the labor market, it seems necessary to improve men's perception of gender roles and to allow men to participate in housework sharing. In addition, it seems necessary to expand national childcare so that women can participate in the labor market without worrying about their children.

County-Level Spatiotemporal Patterns of New HIV Diagnoses and Pre-exposure Prophylaxis (PrEP) Use in Mississippi, 2014–2018: A Bayesian Analysis of Publicly Accessible Censored Data

In the South region of the United States, HIV is disproportionately high and levels of pre-exposure prophylaxis (PrEP) use, which is highly effective in reducing the risk of acquiring HIV, are among the lowest across the country. Simultaneously examining the geographical distributions of both new HIV diagnoses and PrEP use as well as how they evolve over time at the county level is valuable for developing locally tailored intervention programs to target areas most in need of help. There is scant research on this topic using publicly accessible data sets, however, partly because of statistical challenges in modeling censored spatiotemporal data. This study fills this gap by applying a Bayesian spatiotemporal model to analyze interval-censored new HIV diagnoses and left-censored PrEP user data sets in Mississippi at the county level between 2014 and 2018. Suppressed values were modeled with Poisson distributions restricted to ranges where the possible values lie within. A simulation study indicates that the proposed model performs well in estimating censored values and regression coefficients as well as detecting hot spots. At the state level, new HIV diagnoses had a stable trend and PrEP use sharply increased during the study period. DeSoto and Hinds counties warrant special attention because their trends in new HIV diagnoses departed from the state-level trend. We demonstrate that publicly accessible, censored new HIV diagnosis and PrEP user data could be analyzed in ways that yield robust results, which can help health departments and other stakeholders more confidently identify areas that should be prioritized for aggressive HIV prevention.

Mixture discrete reversed hazard rate and its main properties

The reversed hazard rate, defined as the ratio of the density to the distribution function, has recently drawn great interest in reliability and survival analysis due to its usefulness in the analysis of left-censored data. This paper develops the mixture model of discrete reversed hazard rates and studies its main properties. Similar to the continuous case, results of the bending property for the reversed hazard rate in discrete case are studied. In addition, mixtures of the mean inactivity time, which are closely related to the reversed hazard rate, are discussed and the corresponding results of its bending property are also investigated. Preservation of aging properties under both mixtures of reversed hazard rate and mean inactivity time is discussed as well. As a special case, the proportional discrete reversed hazard rate model is presented and the bending properties for this case are also discussed. Finally, stochastic comparisons of the mixture discrete reversed hazard rates of two mixed populations are addressed.

Failure tolerance of the human lumbar spine in dynamic combined compression and flexion loading

Vehicle safety systems have substantially decreased motor vehicle crash-related injuries and fatalities, but injuries to the lumbar spine still have been reported. Experimental and computational analyses of upright and, particularly, reclined occupants in frontal crashes have shown that the lumbar spine can be subjected to simultaneous and out-of-phase combined axial compression and flexion loading. Lumbar spine failure tolerance in combined compression-flexion has not been widely explored in the literature. Therefore, the goal of this study was to measure the failure tolerance of the lumbar spine in combined compression and flexion. Forty lumbar spine segments with three vertebrae (one unconstrained) and two intervertebral discs (both unconstrained) were pre-loaded with axial compression (2200N, 3300N, or 4500N) and then subjected to rotation-controlled dynamic flexion bending until failure. Clinically relevant middle vertebra fractures were observed in twenty-one of the specimens, including compression and burst fractures. The remaining nineteen specimens experienced failure at the potting-grip interface. Failure tolerance varied within the sample and were categorized by the appropriate data censoring, with clinically relevant middle vertebrae fractures characterized as uncensored or left-censored and potting-grip fractures characterized as right-censored. Average failure force and moment were 3290N (range: 1580N to 5042N) and 51Nm (range: 0Nm to 156 Nm) for uncensored data, 3686N (range: 3145N to 4112N) and 0Nm for left-censored data, and 3470N (range: 2138N to 5062N) and 101Nm (range: 27Nm to 182Nm) for right-censored data. These data can be used to develop and improve injury prediction tools for lumbar spine fractures and further research in future safety systems.

Serum cobalamin in children with moderate acute malnutrition in Burkina Faso: Secondary analysis of a randomized trial.

Among children with moderate acute malnutrition (MAM) the level of serum cobalamin (SC) and effect of food supplements are unknown. We aimed to assess prevalence and correlates of low SC in children with MAM, associations with hemoglobin and development, and effects of food supplements on SC. A randomized 2 × 2 × 3 factorial trial was conducted in Burkina Faso. Children aged 6 to 23 months with MAM received 500 kcal/d as lipid-based nutrient supplement (LNS) or corn-soy blend (CSB), containing dehulled soy (DS) or soy isolate (SI) and 0%, 20%, or 50% of total protein from milk for 3 months. Randomization resulted in baseline equivalence between intervention groups. Data on hemoglobin and development were available at baseline. SC was available at baseline and after 3 and 6 months. SC was available from 1,192 (74.1%) of 1,609 children at baseline. The mean (±SD) age was 12.6 (±5.0) months, and 54% were females. Low mid-upper arm circumference (MUAC; <125 mm) was found in 80.4% (958) of the children and low weight-for-length z-score (WLZ; <-2) in 70.6% (841). Stunting was seen in 38.2% (456). Only 5.9% were not breastfed. Median (IQR) SC was 188 (137; 259) pmol/L. Two-thirds had SC ≤222 pmol/L, which was associated with lower hemoglobin. After age and sex adjustments, very low SC (<112 pmol/L) was associated with 0.21 (95% CI: 0.01; 0.41, p = 0.04) and 0.24 (95% CI: 0.06; 0.42, p = 0.01) z-score lower fine and gross motor development, respectively. SC data were available from 1,330 (85.9%) of 1,548 children followed up after 3 months and 398 (26.5%) of the 1,503 children after 6 months. Based on tobit regression, accounting for left censored data, and adjustments for correlates of missing data, the mean (95% CI) increments in SC from baseline to the 3- and 6-month follow-up were 72 (65; 79, p < 0.001) and 26 (16; 37, p < 0.001) pmol/L, respectively. The changes were similar among the 310 children with SC data at all 3 time points. Yet, the increase was 39 (20; 57, p < 0.001) pmol/L larger in children given LNS compared to CSB if based on SI (interaction, p < 0.001). No effect of milk was found. Four children died, and no child developed an allergic reaction to supplements. The main limitation of this study was that only SC was available as a marker of status and was missing from a quarter of the children. Low SC is prevalent among children with MAM and may contribute to impaired erythropoiesis and child development. The SC increase during supplementation was inadequate. The bioavailability and adequacy of cobalamin in food supplements should be reconsidered. ISRCTN Registry ISRCTN42569496.

Monitoring Alternaria toxins in Italian food to support upcoming regulation

ABSTRACT The collection of occurrence data on Alternaria toxins in food and feed across the European countries is required since 2012 by the European Commission, endorsing the relevant scientific opinion by the EFSA CONTAM Panel. Within this framework, occurrence data for Alternaria toxins (Alternariol, Alternariol monomethyl ether, Tenuazonic acid, Tentoxin, and Altenuene) in 97 samples of cereal foods, tomato products, and sunflower seeds have been provided as requested by the Italian national monitoring programme (years 2017–2020). To this purpose, an LC-MS/MS method was set up and validated, obtaining fit for purpose sensitivity, recoveries (70–120%), repeatability (≤20%) and within laboratory reproducibility (≤26%). Occurrence data showed that oilseeds were the most contaminated food group with levels of Tenuazonic acid up to 16752 µg/kg and Tentoxin up to 570 µg/kg, whereas for the other mycotoxin/commodities combinations, the percentage of left censored data (below the limit of quantification) ranged from 74 to 100%.

Undertaking a New Regulatory Challenge: Monitoring of Ergot Alkaloids in Italian Food Commodities.

The present manuscript reports on monitoring data of 12 ergot alkaloids (EAs) in cereal and cereal-derived products, collected in Italy over the period 2017–2020, for official control purposes under the edge of the Commission Recommendation 2012/154/EU on the monitoring of the presence of EAs in feed and food. To these purposes, an LC-MS/MS method was set up and applied, after in-house verification of its analytical performance. Besides satisfactory recoveries and precision, the method’s quantification limits proved suitable to assess the compliance of cereals and cereal-based foods with the recently issued EU maximum permitted levels (Commission Regulation 2021/1399/EU). The validity of the generated data was also evaluated through the adoption of four proficiency tests, from which acceptable z-score values (−2 ≤ z ≤ 2) were obtained. The method was then applied to analyse a total of 67 samples, collected in Italy over the period 2017–2020. The samples consisted of 18 cereal grains, 16 flours (14 of wheat and 2 of spelt) and 31 other types of cereals derivatives (including 9 for infants). Overall, the EAs analysis returned a high percentage of left-censored data (>86%). Among the positive samples, the highest contamination levels, up to 94.2 µg/kg, were found for ergocristine (12% incidence), followed by ergocristinine (7% incidence) with levels of up to 48.3 µg/kg.

Bayesian analysis of left-censored data using Weibull mixture model

Bayesian analysis of left-censored data using Weibull mixture model

Impact of the Look-Back Period on Identifying Recurrent Myocardial Infarctions in the Danish National Patient Registry.

PurposeStudies examining myocardial infarction (MI) often seek to include only incident MIs by excluding recurrent MIs. When based on historical data, identification of previous MI depends on the length of the look-back period. However, international registries often cover a short time period, consequently containing left-censored data, making it impossible to determine if a first MI in a period is truly an incident MI. We evaluated whether the proportion of MIs identified as recurrent MIs depends on the look-back period, and how including recurrent MIs in a planned incident MI cohort impacts survival estimates.Patients and MethodsWe used the Danish National Patient Registry, covering all Danish hospitals since 1977 to identify first MIs during 2010–2016 (index events). The hospital registry history preceding the index event was then searched for previous MIs. We plotted the proportion of index events identified as recurrent MIs as a function of the look-back period. Moreover, we calculated 5-year all-cause mortality and confidence intervals (CIs) using the 1-Kaplan–Meier method for five cohorts based on the index events and defined by look-back periods of 0, 5, 10, 20, and up to 39 years.ResultsAmong 63,885 index events, 3.4% were identified as recurrent MIs with 5 years of look-back, 7.9% with 10 years, 14% with 24 years, and 15% with up to 39 years. All-cause mortality risk was 36% (95% CI: 36–37%) with 0 years of look-back, 35% (95% CI: 35–36%) with 5 years, 35% (95% CI: 35–36%) with 10 years, 34% (95% CI: 34–35%) with 20 years, and 34% (95% CI: 33–34%) with up to 39 years.ConclusionMost recurrent MIs were identified with a look-back period of 24 years. Including recurrent MIs in a planned incident MI cohort, due to shorter look-back periods, overestimated the mortality risk.

Using Maximum Simulated Likelihood Methods to Overcome Left Censoring: Dynamic Event History Models of Heart Attack Risk in New Zealand

Abstract This paper describes how the risk of experiencing heart attacks varies across gender and ethnicity in New Zealand. We estimate dynamic hazard models using administrative data. We deal with left-censored data using recently developed maximum simulated likelihood methods. The models allow risk to vary with age, previous heart attack history and unobserved individual heterogeneity. We find that the risk of subsequent events is far higher than the risk of the first event, particularly high within 1 year after an event, and that unobserved heterogeneity is important. Generally, male Maoris have the highest risk, followed by female Maoris, then ethnically European males, while ethnically European females have the lowest risk.

Influence of living in the same home on biomonitored levels of consumer product chemicals.

Individuals living in the same home may share exposures from direct contact with sources or indirectly through contamination of the home environment. We investigated the influence of sharing a home on urine levels of ten phenolic chemicals present in some consumer products. We used data from Silent Spring Institute's Detox Me Action Kit (DMAK), a crowdsourced biomonitoring program in the US. Of the 726 DMAK participants, 185 lived in the same home with one or more other DMAK participants (n = 137 pairs, up to six participants in a home). The concentration distributions included values below the detection limit so we used statistical methods that account for left-censored data, including non-parametric correlation estimation and hierarchical Bayesian regression models. Concentrations were significantly positively correlated between pair-members sharing a home for nine of the ten chemicals. Concentrations of 2,5-dichlorophenol were the most strongly correlated between pair-members (tau = 0.46), followed by benzophenone-3 (tau = 0.31) and bisphenol A (tau = 0.21). The relative contribution of personal product use reported product use of other household members (up to 5 others), and the residual contribution from a shared household, including exposures not asked about, varied by chemical. Paraben concentrations were largely influenced by personal behaviors whereas dichlorophenol and bisphenol concentrations were largely influenced by shared home exposures not related to reported behaviors. Measuring the influence of personal and household practices on biomonitoring exposures helps pinpoint major sources of exposure and highlights chemical-specific intervention strategies to reduce them.

Nonparametric Multiple Imputation of Left Censored Event Times in Analysis of Follow-up Data

In this paper, we consider analysis of follow-up data where each event time is either right censored, observed, left censored or left truncated. In the case of left censoring, the covariates measured at baseline are considered as missing. The work is motivated by data from the MORGAM Project, which explores the association between cardiovascular diseases and their classic and genetic risk factors. We propose a nonparametric multiple imputation (NPMI) approach where the left censored event times and the missing covariates are imputed in hot deck manner. The left truncation due to deaths prior to baseline is compensated by Lexis diagram imputation introduced in the paper. After imputation, the standard estimation methods for right censored survival data can be directly applied. The performance of the proposed imputation approach is studied with simulated and real world data. The results suggest that the NPMI is a flexible and reliable approach to the analysis of left and right censored data.