Comparing Performance between Models Assessing the Additive Effects of Exposure Mixtures

Abstract

Background/Aim: Multiple methods have been proposed to assess the relationship between mixtures of environmental exposures and health outcomes. However, few simulation experiments have been performed to compare statistical properties. Methods: We simulated data to compare the performance of additive mixture models in inferring associations with continuous outcomes. We tested weighted quantile sum (WQS) regressions run using either a bootstrap or random subset algorithm to determine mixture weights. Each of these was tested without splitting the data into training and validation datasets, splitting the data once, or repeatedly splitting the data. In addition, we tested a novel WQS regression permutation test designed to maintain statistical power with nominal false positive rate (FPR) by not requiring splitting the data. We also tested quantile g-computation regression and novel Bayesian formulations of WQS and quantile g-computation regressions. Simulations modeled a positive or null association between the outcome and a mixture with individual components having contributions in the same direction and varying weights. We also applied these methods to assess the association between prenatal phthalate exposure and early childhood full scale intelligence quotient (FSIQ) in the CANDLE cohort. Results: The random subset WQS regressions without splitting the data into training and validation datasets coupled with the permutation test had the best balance of accuracy, specificity, and sensitivity for mixture coefficients and weights (Power=0.86-0.92, FPR=0.05), compared to splitting the data once (Power=0.57-0.69, FPR=0.03-0.06), multiple times (Power=0.88-0.94, FPR=0.07-0.11), or not at all (Power=0.93-0.97, FPR=0.05-0.18). Quantile g-computation performed similarly though with increased weight estimate error. Associations with FSIQ varied substantially across analytic methods. Conclusions: WQS methods perform best when paired with a permutation test and when using a random subset algorithm, even with relatively few mixture components. Variation in model results with the real data highlights the importance of evaluating the relative performance of these models.