Evaluating the effect of individualized treatment rules using observational data

Abstract

Individualized treatment rules (ITR) recommend different treatments to individuals based on their observed characteristics. Approaches range from the use of conventional risk prediction models in a counterfactual framework to causal machine learning methods. Randomized controlled trials comparing the use of ITRs to usual care are rare. Objective: we wished to develop a method to emulate such trials from observational data. In the ITR arm, clinicians would not always use the ITR or follow its recommendations. We, therefore, introduce a stochastic component for ITRs’ implementation. Setup: Using Rubin's causal model, we rely on the assumptions of consistency, ignorability and overlap. Estimand of interest: Average Implementation Effect (AIE). The AIE represents the effect the stochastic implementation of a deterministic ITR would have on a given population. Modeling stochastic implementation functions: We propose to consider: - A cognitive bias scenario, i.e., the ITR is implemented more often when the ITR's recommendation is similar to usual care. - A confidence level scenario, i.e., the ITR is implemented only when the (1-α)% confidence interval provided along ITR's recommendation does not cross the critical value. Inference: we developed a procedure to numerically study how the AIE would vary under different stochastic implementation scenarios. Application: we used the MIMIC-III electronic health record to evaluate the population-level impact on 60-day mortality of a new ITR that recommends initiating dialysis only in specific patients based on a combination of six biomarkers. Inclusion criteria included admission to an intensive care unit with acute kidney injury preceded by either mechanical ventilation or vasopressor infusion. From the 53,423 individuals contained in the MIMIC-III database, we included 3,748 adult patients. Results are depicted in the Figure: Panels depict the AIE as a function of the proportion of patients implementing the new ITR under 1/ the cognitive bias scenario (Panel A) and 2/ the confidence level scenario (Panel B). More negative values of the AIE indicate greater benefit from ITR implementation. Ninety-five percent confidence intervals are from the bootstrap. We have developed a new method for observational data that allows emulating randomized trials that compare the stochastic implementation of a new ITR to usual care. Our formal setup delineates the assumption needed for our methodology while the example we provide demonstrates applicability in practice. Causal inference; Precision medicine; Heterogenous treatment effects; Machine learning; External validation Les auteurs n'ont pas précisé leurs éventuels liens d'intérêts.