Q-Learning in Dynamic Treatment Regimes With Misclassified Binary Outcome.

Abstract

The study of precision medicine involves dynamic treatment regimes (DTRs), which are sequences of treatment decision rules recommended based on patient-level information. The primary goal of the DTR study is to identify an optimal DTR, a sequence of treatment decision rules that optimizes the clinical outcome across multiple decision points. Statistical methods have been developed in recent years to estimate an optimal DTR, including Q-learning, a regression-based method in the DTR literature. Although there are many studies concerning Q-learning, little attention has been paid in the presence of noisy data, such as misclassified outcomes. In this article, we investigate the effect of outcome misclassification on identifying optimal DTRs using Q-learning and propose a correction method to accommodate the misclassification effect on DTR. Simulation studies are conducted to demonstrate the satisfactory performance of the proposed method. We illustrate the proposed method using two examples from the National Health and Nutrition Examination Survey Data I Epidemiologic Follow-up Study and the Population Assessment of Tobacco and Health Study.