Multimodal imaging-based material mass density estimation for proton therapy using supervised deep learning.

Abstract

Mapping CT number to material property dominates the proton range uncertainty. This work aims to develop a physics-constrained deep learning-based multimodal imaging (PDMI) framework to integrate physics, deep learning, MRI, and advanced dual-energy CT (DECT) to derive accurate patient mass density maps. Seven tissue substitute MRI phantoms were used for validation including adipose, brain, muscle, liver, skin, spongiosa, 45% hydroxyapatite (HA) bone. MRI images were acquired using T1 weighted Dixon and T2 weighted short tau inversion recovery sequences. Training inputs are from MRI and twin-beam dual-energy images acquired at 120 kVp with gold/tin filters. The feasibility investigation included an empirical model and four residual networks (ResNet) derived from different training inputs and strategies by PDMI framework. PRN-MR-DE and RN-MR-DE denote ResNet (RN) trained with and without a physics constraint (P) using MRI (MR) and DECT (DE) images. PRN-DE stands for RN trained with a physics constraint using only DE images. A retrospective study using institutional patient data was also conducted to investigate the feasibility of the proposed framework. For the tissue surrogate study, PRN-MR-DE, PRN-DE, and RN-MR-DE result in mean mass density errors: -0.72%/2.62%/-3.58% for adipose; -0.03%/-0.61%/-0.18% for muscle; -0.58%/-1.36%/-4.86% for 45% HA bone. The retrospective patient study indicated that PRN-MR-DE predicted the densities of soft tissue and bone within expected intervals based on the literature survey, while PRN-DE generated large density deviations. The proposed PDMI framework can generate accurate mass density maps using MRI and DECT images. The supervised learning can further enhance model efficacy, making PRN-MR-DE outperform RN-MR-DE. The patient investigation also shows that the framework can potentially improve proton range uncertainty with accurate patient mass density maps. PDMI framework is proposed for the first time to inform deep learning models by physics insights and leverage the information from MRI to derive accurate mass density maps.