Reinforcement Learning Model for Managing Noninvasive Ventilation Switching Policy.

Abstract

Noninvasive ventilation (NIV) has been recognized as a first-line treatment for respiratory failure in patients with chronic obstructive pulmonary disease (COPD) and hypercapnia respiratory failure, which can reduce mortality and burden of intubation. However, during the long-term NIV process, failure to respond to NIV may cause overtreatment or delayed intubation, which is associated with increased mortality or costs. Optimal strategies for switching regime in the course of NIV treatment remain to be explored.For the goal of reducing 28-day mortality of the patients undergoing NIV, Double Dueling Deep Q Network (D3QN) of offline-reinforcement learning algorithm was adopted to develop an optimal regime model for making treatment decisions of discontinuing ventilation, continuing NIV, or intubation. The model was trained and tested using the data from Multi-Parameter Intelligent Monitoring in Intensive Care III (MIMIC-III) and evaluated by the practical strategies. Furthermore, the applicability of the model in majority disease subgroups (Catalogued by International Classification of Diseases, ICD) was investigated. Compared with physician's strategies, the proposed model achieved a higher expected return score (4.25 vs. 2.68) and its recommended treatments reduced the expected mortality from 27.82% to 25.44% in all NIV cases. In particular, for these patients finally received intubation in practice, if the model also supported the regime, it would warn of switching to intubation 13.36 hours earlier than clinicians (8.64 vs. 22 hours after the NIV treatment), granting a 21.7% reduction in estimated mortality. In addition, the model was applicable across various disease groups with distinguished achievement in dealing with respiratory disorders. The proposed model is promising to dynamically provide personalized optimal NIV switching regime for patients undergoing NIV with the potential of improving treatment outcomes.