Application of the Integrated Deep Reinforcement Learning Solution in Automatic IMRT Planning for Gastric Cancer

Abstract

The intensity-modulated radiation therapy (IMRT) treatment planning optimization process is usually a manual, tedious and time-consuming task. An unsupervised integrated automatic IMRT planning solution was discussed and implemented to simulate the manual operation during the whole planning process. Based on knowledge-based planning (KBP) and deep reinforcement learning (DRL) schemes, a novel integrated solution combining the multi-objectives optimization policy network (MOPN) and three-dimensional dose prediction module (3D-DPM) was proposed. The MOPN was trained to learn how to adjust multiple optimization objectives in commercial Eclipse treatment planning system (TPS). The 3D-DPM was developed to generate the patient-specific initial optimization objectives to reduce the overall exploration space during MOPN training. The Eclipse 15.6 TPS Scripting Application Programming Interface (ESAPI) was used to realize the automatic interaction between models and TPS. In this study, 100 previously treated gastric cancer cases were selected from the clinical database, 70 cases were used for the training of dose prediction model and MOPN, the remaining 30 cases for evaluating the feasibility and effectiveness of automatic treatment planning solution. For all tested cases, the complete automatic treatment plan for a new case was generated based on the integrated solution, with about 6 min. Compared with the MOPN initial plans, the actual dose of spinal cord, liver, kidney-left, kidney-right, and intestine in the MOPN final plans reduced 42.4%, 25.2%, 52.4%, 37.8%, 11.8% respectively. The dose result of OARs in the MOPN final plans was similar to those in the clinical plans. In addition, a new case was able to complete optimization with about 15 adjustment steps by the trained MOPN. We successfully developed an integrated solution for automatic treatment planning. It first includes the dose prediction module for obtaining patient-specific initial optimization objectives. We demonstrated that the trained MOPN can mimic the operations of the physicians to adjust multiple objectives and obtain a high-quality plan in a shorter time. This integrated solution contributes to improving the efficiency of the overall planning workflow and reducing the variation of plan quality in clinical practice. Although improvement is warranted, the proposed integrated solution is a promising practical and effective approach for automatic planning in commercial TPS.