Optimal Individualized Treatment Rules Research Articles

Robust Covariate-Balancing Method in Learning Optimal Individualized Treatment Regimes

Summary One of the most important problems in precision medicine is to find the optimal individualized treatment rule, which is designed to recommend treatment decisions and maximize overall clinical benefit to patients based on their individual characteristics. Typically, the expected clinical outcome is required to be estimated first, in which an outcome regression model or a propensity score model usually needs to be assumed for most of the existing statistical methods. However, if either model assumption is invalid, the estimated treatment regime is not reliable. In this article, we first define a contrast value function, which is the basis of the study for individualized treatment regimes. Then we construct a hybrid estimator of the contrast value function, by combining two types of estimation methods. We further propose a robust covariate-balancing estimator of the contrast value function by combining the inverse probability weighted method and matching method, which is based on the covariate balancing propensity score proposed by Imai and Ratkovic (2014). Theoretical results show that the proposed estimator is doubly robust, that is, it is consistent if either the propensity score model or the matching is correct. Based on a large number of simulation studies, we demonstrate that the proposed estimator outperforms existing methods. Lastly, the proposed method is illustrated through analysis of the SUPPORT study.

Individualized treatment rule characterization via a value function surrogate.

Precision medicine is a promising framework for generating evidence to improve health and health care. Yet, a gap persists between the ever-growing number of statistical precision medicine strategies for evidence generation and implementation in real-world clinical settings, and the strategies for closing this gap will likely be context-dependent. In this paper, we consider the specific context of partial compliance to wound management among patients with peripheral artery disease. Using a Gaussian process surrogate for the value function, we show the feasibility of using Bayesian optimization to learn optimal individualized treatment rules. Further, we expand beyond the common precision medicine task of learning an optimal individualized treatment rule to the characterization of classes of individualized treatment rules and show how those findings can be translated into clinical contexts.

Robust sample weighting to facilitate individualized treatment rule learning for a target population

Summary Learning individualized treatment rules is an important topic in precision medicine. Current literature mainly focuses on deriving individualized treatment rules from a single source population. We consider the observational data setting when the source population differs from a target population of interest. Compared with causal generalization for the average treatment effect that is a scalar quantity, individualized treatment rule generalization poses new challenges due to the need to model and generalize the rules based on a prespecified class of functions that may not contain the unrestricted true optimal individualized treatment rule. The aim of this paper is to develop a weighting framework to mitigate the impact of such misspecification, and thus facilitate the generalizability of optimal individualized treatment rules from a source population to a target population. Our method seeks covariate balance over a nonparametric function class characterized by a reproducing kernel Hilbert space and can improve many individualized treatment rule learning methods that rely on weights. We show that the proposed method encompasses importance weights and overlap weights as two extreme cases, allowing for a better bias-variance trade-off in between. Numerical examples demonstrate that the use of our weighting method can greatly improve individualized treatment rule estimation for the target population compared with other weighting methods.

Individualized treatment rules under stochastic treatment cost constraints.

Estimation and evaluation of individualized treatment rules have been studied extensively, but real-world treatment resource constraints have received limited attention in existing methods. We investigate a setting in which treatment is intervened upon based on covariates to optimize the mean counterfactual outcome under treatment cost constraints when the treatment cost is random. In a particularly interesting special case, an instrumental variable corresponding to encouragement to treatment is intervened upon with constraints on the proportion receiving treatment. For such settings, we first develop a method to estimate optimal individualized treatment rules. We further construct an asymptotically efficient plug-in estimator of the corresponding average treatment effect relative to a given reference rule.

Model selection for survival individualized treatment rules using the jackknife estimator

BackgroundPrecision medicine is an emerging field that involves the selection of treatments based on patients’ individual prognostic data. It is formalized through the identification of individualized treatment rules (ITRs) that maximize a clinical outcome. When the type of outcome is time-to-event, the correct handling of censoring is crucial for estimating reliable optimal ITRs.MethodsWe propose a jackknife estimator of the value function to allow for right-censored data for a binary treatment. The jackknife estimator or leave-one-out-cross-validation approach can be used to estimate the value function and select optimal ITRs using existing machine learning methods. We address the issue of censoring in survival data by introducing an inverse probability of censoring weighted (IPCW) adjustment in the expression of the jackknife estimator of the value function. In this paper, we estimate the optimal ITR by using random survival forest (RSF) and Cox proportional hazards model (COX). We use a Z-test to compare the optimal ITRs learned by RSF and COX with the zero-order model (or one-size-fits-all). Through simulation studies, we investigate the asymptotic properties and the performance of our proposed estimator under different censoring rates. We illustrate our proposed method on a phase III clinical trial of non-small cell lung cancer data.ResultsOur simulations show that COX outperforms RSF for small sample sizes. As sample sizes increase, the performance of RSF improves, in particular when the expected log failure time is not linear in the covariates. The estimator is fairly normally distributed across different combinations of simulation scenarios and censoring rates. When applied to a non-small-cell lung cancer data set, our method determines the zero-order model (ZOM) as the best performing model. This finding highlights the possibility that tailoring may not be needed for this cancer data set.ConclusionThe jackknife approach for estimating the value function in the presence of right-censored data shows satisfactory performance when there is small to moderate censoring. Winsorizing the upper and lower percentiles of the estimated survival weights for computing the IPCWs stabilizes the estimator.

An efficient approach for optimizing the cost-effective individualized treatment rule using conditional random forest.

Evidence from observational studies has become increasingly important for supporting healthcare policy making via cost-effectiveness analyses. Similar as in comparative effectiveness studies, health economic evaluations that consider subject-level heterogeneity produce individualized treatment rules that are often more cost-effective than one-size-fits-all treatment. Thus, it is of great interest to develop statistical tools for learning such a cost-effective individualized treatment rule under the causal inference framework that allows proper handling of potential confounding and can be applied to both trials and observational studies. In this paper, we use the concept of net-monetary-benefit to assess the trade-off between health benefits and related costs. We estimate cost-effective individualized treatment rule as a function of patients' characteristics that, when implemented, optimizes the allocation of limited healthcare resources by maximizing health gains while minimizing treatment-related costs. We employ the conditional random forest approach and identify the optimal cost-effective individualized treatment rule using net-monetary-benefit-based classification algorithms, where two partitioned estimators are proposed for the subject-specific weights to effectively incorporate information from censored individuals. We conduct simulation studies to evaluate the performance of our proposals. We apply our top-performing algorithm to the NIH-funded Systolic Blood Pressure Intervention Trial to illustrate the cost-effectiveness gains of assigning customized intensive blood pressure therapy.

Multiple Domain and Multiple Kernel Outcome-Weighted Learning for Estimating Individualized Treatment Regimes

Individualized treatment rules (ITRs) recommend treatments that are tailored specifically according to each patient’s own characteristics. It can be challenging to estimate optimal ITRs when there are many features, especially when these features have arisen from multiple data domains (e.g., demographics, clinical measurements, neuroimaging modalities). Considering data from complementary domains and using multiple similarity measures to capture the potential complex relationship between features and treatment can potentially improve the accuracy of assigning treatments. Outcome weighted learning (OWL) methods that are based on support vector machines using a predetermined single kernel function have previously been developed to estimate optimal ITRs. In this article, we propose an approach to estimate optimal ITRs by exploiting multiple kernel functions to describe the similarity of features between subjects both within and across data domains within the OWL framework, as opposed to preselecting a single kernel function to be used for all features for all domains. Our method takes into account the heterogeneity of each data domain and combines multiple data domains optimally. Our learning process estimates optimal ITRs and also identifies the data domains that are most important for determining ITRs. This approach can thus be used to prioritize the collection of data from multiple domains, potentially reducing cost without sacrificing accuracy. The comparative advantage of our method is demonstrated by simulation studies and by an application to a randomized clinical trial for major depressive disorder that collected features from multiple data domains. Supplementary materials for this article are available online.

Self-matched learning to construct treatment decision rulesfrom electronic health records.

Electronic health records (EHRs) collected from large-scale health systems provide rich subject-specific information on a broad patient population at a lower cost compared to randomized controlled trials. Thus, EHRs may serve as a complementary resource to provide real-world data to construct individualized treatment rules (ITRs) and achieve precision medicine. However, in the absence of randomization, inferring treatment rules from EHR data may suffer from unmeasured confounding. In this article, we propose a self-matched learning method inspired by the self-controlled case series (SCCS) design to mitigate this challenge. We alleviate unmeasured time-invariant confounding between patients by matching different periods of treatments within the same patient (self-controlled matching) to infer the optimal ITRs. The proposed method constructs a within-subject matched value function for optimizing ITRs and bears similarity to the SCCS design. We examine assumptions that ensure Fisher consistency, and show that our method requires weaker assumptions on unmeasured confounding than alternative methods. Through extensive simulation studies, we demonstrate that self-matched learning has comparable performance to other existing methods when there are no unmeasured confounders, but performs markedly better when unobserved time-invariant confounders are present, which is often the case for EHRs. Sensitivity analyses show that the proposed method is robust under different scenarios. Finally, we apply self-matched learning to estimate the optimal ITRs from type 2 diabetes patient EHRs, which shows our estimated decision rules lead to greater advantages in reducing patients' diabetes-related complications.

Privacy-preserving estimation of an optimal individualized treatment rule: a case study in maximizing time to severe depression-related outcomes.

Estimating individualized treatment rules-particularly in the context of right-censored outcomes-is challenging because the treatment effect heterogeneity of interest is often small, thus difficult to detect. While this motivates the use of very large datasets such as those from multiple health systems or centres, data privacy may be of concern with participating data centres reluctant to share individual-level data. In this case study on the treatment of depression, we demonstrate an application of distributed regression for privacy protection used in combination with dynamic weighted survival modelling (DWSurv) to estimate an optimal individualized treatment rule whilst obscuring individual-level data. In simulations, we demonstrate the flexibility of this approach to address local treatment practices that may affect confounding, and show that DWSurv retains its double robustness even when performed through a (weighted) distributed regression approach. The work is motivated by, and illustrated with, an analysis of treatment for unipolar depression using the United Kingdom's Clinical Practice Research Datalink.

Ranking tailoring variables for constructing individualized treatment rules: an application to schizophrenia.

As with many chronic conditions, matching patients with schizophrenia to the best treatment options is difficult. Selecting antipsychotic medication is especially challenging because many of the medications can have burdensome side effects. Adjusting or tailoring medications based on patients' characteristics could improve symptoms. However, it is often not known which patient characteristics are most helpful for informing treatment selection. In this paper, we address the challenge of identifying and ranking important variables for tailoring treatment decisions. We consider a value-search approach implemented through dynamic marginal structural models to estimate an optimal individualized treatment rule. We apply our methodology to the Clinical Antipsychotics Trial of Intervention and Effectiveness (CATIE) study for schizophrenia, to evaluate if some tailoring variables have greater potential than others for selecting treatments for patients with schizophrenia (Stroup et al., 2003).

Fairness-Oriented Learning for Optimal Individualized Treatment Rules

ABSTRACT There has recently been a surge on the methodological development for optimal individualized treatment rule (ITR) estimation. The standard methods in the literature are designed to maximize the potential average performance (assuming larger outcomes are desirable). A notable drawback of the standard approach, due to heterogeneity in treatment response, is that the estimated optimal ITR may be suboptimal or even detrimental to certain disadvantaged subpopulations. Motivated by the importance of incorporating an appropriate fairness constraint in optimal decision making (e.g., assign treatment with protection to those with shorter survival time, or assign a job training program with protection to those with lower wages), we propose a new framework that aims to estimate an optimal ITR to maximize the average value with the guarantee that its tail performance exceeds a prespecified threshold. The optimal fairness-oriented ITR corresponds to a solution of a nonconvex optimization problem. To handle the computational challenge, we develop a new efficient first-order algorithm. We establish theoretical guarantees for the proposed estimator. Furthermore, we extend the proposed method to dynamic optimal ITRs. The advantages of the proposed approach over existing methods are demonstrated via extensive numerical studies and real data analysis.

Estimating individualized treatment rules for treatments with hierarchical structure

Precision medicine is an increasingly important area of research. Due to the heterogeneity of individual characteristics, patients may respond differently to treatments. One of the most important goals for precision medicine is to develop individualized treatment rules (ITRs) involving patients’ characteristics directly. As an interesting topic in clinical research, many statistical methods have been developed in recent years to find optimal ITRs. For binary treatments, outcome weighted learning (OWL) was proposed to find a decision function of patient characteristics maximizing the expected clinical outcome. Treatments with hierarchical structure are commonly seen in practice. In hierarchical scenarios, how to estimate ITRs is still unclear. We propose a new framework named hierarchical outcome-weighted angle-based learning (HOAL) to estimate ITRs for treatments with hierarchical structure. Statistical properties including Fisher consistency and convergence rates of the proposed method are presented. Simulations and an application to a type 2 diabetes study under linear and nonlinear learning show the highly competitive performance of our proposed procedure in both numerical accuracy and computational efficiency.

Efficient Learning of Optimal Individualized Treatment Rules for Heteroscedastic or Misspecified Treatment-Free Effect Models

AbstractRecent development in data-driven decision science has seen great advances in individualized decision making. Given data with individual covariates, treatment assignments and outcomes, researchers can search for the optimal individualized treatment rule (ITR) that maximizes the expected outcome. Existing methods typically require initial estimation of some nuisance models. The double robustness property that can protect from misspecification of either the treatment-free effect or the propensity score has been widely advocated. However, when model misspecification exists, a doubly robust estimate can be consistent but may suffer from downgraded efficiency. Other than potential misspecified nuisance models, most existing methods do not account for the potential problem when the variance of outcome is heterogeneous among covariates and treatment. We observe that such heteroscedasticity can greatly affect the estimation efficiency of the optimal ITR. In this paper, we demonstrate that the consequences of misspecified treatment-free effect and heteroscedasticity can be unified as a covariate-treatment dependent variance of residuals. To improve efficiency of the estimated ITR, we propose an Efficient Learning (E-Learning) framework for finding an optimal ITR in the multi-armed treatment setting. We show that the proposed E-Learning is optimal among a regular class of semiparametric estimates that can allow treatment-free effect misspecification. In our simulation study, E-Learning demonstrates its effectiveness if one of or both misspecified treatment-free effect and heteroscedasticity exist. Our analysis of a type 2 diabetes mellitus (T2DM) observational study also suggests the improved efficiency of E-Learning.

Robust index of confidence weighted learning for optimal individualized treatment rule estimation

Determination of optimal individual treatment rules (ITR) is a rapidly growing area in precision medicine; various parametric and non‐parametric methods have been proposed. Existing methods, however, focus on the mean outcome and thus are sensitive to outliers, skewed and heavy‐tailed outcome distributions. In this paper, we propose an optimal ITR estimation framework using a weighted classifier with robust weights based on measures of similarity. Compared to previous methods in the literature, this two‐stage nonparametric model is novel and enjoys several advantages. First, due to its non‐parametric nature, it is more flexible than regression‐based parametric and semi‐parametric models. Second, the similarity‐based confidence index is essentially a weighted sum of indicator functions depending on the sign of pairwise outcome differences; therefore, it is robust to outliers, skewed and heavy‐tailed outcome distributions. The performance of the proposed approach is demonstrated via simulation studies and an analysis of data from a randomized clinical trial for depression.

Estimation of Optimal Individualized Treatment Rules Using a Covariate-Specific Treatment Effect Curve With High-Dimensional Covariates

With a large number of baseline covariates, we propose a new semiparametric modeling strategy for heterogeneous treatment effect estimation and individualized treatment selection, which are two major goals in personalized medicine. We achieve the first goal through estimating a covariate-specific treatment effect (CSTE) curve modeled as an unknown function of a weighted linear combination of all baseline covariates. The weight or the coefficient for each covariate is estimated by fitting a sparse semiparametric logistic single-index coefficient model. The CSTE curve is estimated by a spline-backfitted kernel procedure, which enables us to further construct a simultaneous confidence band (SCB) for the CSTE curve under a desired confidence level. Based on the SCB, we find the subgroups of patients that benefit from each treatment, so that we can make individualized treatment selection. The innovations of the proposed method are 3-fold. First, the proposed method can quantify variability associated with the estimated optimal individualized treatment rule with high-dimensional covariates. Second, the proposed method is very flexible to depict both local and global associations between the treatment and baseline covariates in the presence of high-dimensional covariates, and thus it enjoys flexibility while achieving dimensionality reduction. Third, the SCB achieves the nominal confidence level asymptotically, and it provides a uniform inferential tool in making individualized treatment decisions. Supplementary materials for this article are available online.

Estimating the optimal individualized treatment rule from a cost-effectiveness perspective.

Optimal individualized treatment rules (ITRs) provide customized treatment recommendations based on subject characteristics to maximize clinical benefit in accordance with the objectives in precision medicine. As a result, there is growing interest in developing statistical tools for estimating optimal ITRs in evidence-based research. In health economic perspectives, policy makers consider the tradeoff between health gains and incremental costs of interventions to set priorities and allocate resources. However, most work on ITRs has focused on maximizing the effectiveness of treatment without considering costs. In this paper, we jointly consider the impact of effectiveness and cost on treatment decisions and define ITRs under a composite-outcome setting, so that we identify the most cost-effective ITR that accounts for individual-level heterogeneity through direct optimization. In particular, we propose a decision-tree-based statistical learning algorithm that uses a net-monetary-benefit-based reward to provide nonparametric estimations of the optimal ITR. We provide several approaches to estimating the reward underlying the ITR as a function of subject characteristics. We present the strengths and weaknesses of each approach and provide practical guidelines by comparing their performance in simulation studies. We illustrate the top-performing approach from our simulations by evaluating the projected 15-year personalized cost-effectiveness of the intensive blood pressure control of the Systolic Blood Pressure Intervention Trial (SPRINT)study.

High-Dimensional Precision Medicine From Patient-Derived Xenografts

The complexity of human cancer often results in significant heterogeneity in response to treatment. Precision medicine offers the potential to improve patient outcomes by leveraging this heterogeneity. Individualized treatment rules (ITRs) formalize precision medicine as maps from the patient covariate space into the space of allowable treatments. The optimal ITR is that which maximizes the mean of a clinical outcome in a population of interest. Patient-derived xenograft (PDX) studies permit the evaluation of multiple treatments within a single tumor, and thus are ideally suited for estimating optimal ITRs. PDX data are characterized by correlated outcomes, a high-dimensional feature space, and a large number of treatments. Here we explore machine learning methods for estimating optimal ITRs from PDX data. We analyze data from a large PDX study to identify biomarkers that are informative for developing personalized treatment recommendations in multiple cancers. We estimate optimal ITRs using regression-based (Q-learning) and direct-search methods (outcome weighted learning). Finally, we implement a superlearner approach to combine multiple estimated ITRs and show that the resulting ITR performs better than any of the input ITRs, mitigating uncertainty regarding user choice. Our results indicate that PDX data are a valuable resource for developing individualized treatment strategies in oncology. Supplementary materials for this article are available online.

Learning Individualized Treatment Rules for Multiple-Domain Latent Outcomes

For many mental disorders, latent mental status from multiple-domain psychological or clinical symptoms may perform as a better characterization of the underlying disorder status than a simple summary score of the symptoms, and they may also serve as more reliable and representative features to differentiate treatment responses. Therefore, to address the complexity and heterogeneity of treatment responses for mental disorders, we provide a new paradigm for learning optimal individualized treatment rules (ITRs) by modeling patients’ latent mental status. We first learn the multi-domain latent states at baseline from the observed symptoms under a restricted Boltzmann machine (RBM) model, which encodes patients’ heterogeneous symptoms using an economical number of latent variables and yet remains flexible. We then optimize a value function defined by the latent states after treatment by exploiting a transformation of the observed symptoms based on the RBM without modeling the relationship between the latent mental states before and after treatment. The optimal treatment rules are derived using a weighted large margin classifier. We derive the convergence rate of the proposed estimator under the latent models. Simulation studies are conducted to test the performance of the proposed method. Finally, we apply the developed method to real world studies and we demonstrate the utility and advantage of our method in tailoring treatments for patients with major depression, and identify patient subgroups informative for treatment recommendations. Supplementary materials for this article are available online.

Learning Optimal Distributionally Robust Individualized Treatment Rules

Recent development in the data-driven decision science has seen great advances in individualized decision making. Given data with individual covariates, treatment assignments and outcomes, policy makers best individualized treatment rule (ITR) that maximizes the expected outcome, known as the value function. Many existing methods assume that the training and testing distributions are the same. However, the estimated optimal ITR may have poor generalizability when the training and testing distributions are not identical. In this article, we consider the problem of finding an optimal ITR from a restricted ITR class where there are some unknown covariate changes between the training and testing distributions. We propose a novel distributionally robust ITR (DR-ITR) framework that maximizes the worst-case value function across the values under a set of underlying distributions that are “close” to the training distribution. The resulting DR-ITR can guarantee the performance among all such distributions reasonably well. We further propose a calibrating procedure that tunes the DR-ITR adaptively to a small amount of calibration data from a target population. In this way, the calibrated DR-ITR can be shown to enjoy better generalizability than the standard ITR based on our numerical studies. Supplementary materials for this article are available online.

Improved Doubly Robust Estimation in Learning Optimal Individualized Treatment Rules

Individualized treatment rules (ITRs) recommend treatment according to patient characteristics. There is a growing interest in developing novel and efficient statistical methods in constructing ITRs. We propose an improved doubly robust estimator of the optimal ITRs. The proposed estimator is based on a direct optimization of an augmented inverse-probability weighted estimator of the expected clinical outcome over a class of ITRs. The method enjoys two key properties. First, it is doubly robust, meaning that the proposed estimator is consistent when either the propensity score or the outcome model is correct. Second, it achieves the smallest variance among the class of doubly robust estimators when the propensity score model is correctly specified, regardless of the specification of the outcome model. Simulation studies show that the estimated ITRs obtained from our method yield better results than those obtained from current popular methods. Data from the Sequenced Treatment Alternatives to Relieve Depression study is analyzed as an illustrative example. Supplementary materials for this article are available online.