A deep learning approach to transform two orthogonal X-ray images to volumetric images for image-guided proton therapy

Abstract

Due to potential patient anatomy variation during the treatment, proton therapy requires image-guidance systems to minimize treatment uncertainty and ensure conformal doses are precisely delivered to tumor target. This study proposes a feature matching network to generate volumetric images through two orthogonal 2D X-ray images to detect target location during the treatment. The method leverages the kV image system of proton treatment machines to simultaneously acquire two orthogonal 2D images to consistently predict 3D real-time patient anatomy without biasing by motion. The proposed feature mapping network includes four components: a n° kV feature extractor, a n°+90° kV feature extractor, a feature re-alignment, and a depth estimation. The two feature extractors are constructed by two independent convolutional neural networks (CNN) to capture the details of 2D image structures. The re-aliment component ensures that the two orthogonal 2D images are aligned in the same coordinate system. The last part is built by a fully convolutional network (FCN) to inversely transform 2D information to volumetric images. 10 patient cases with 4D CT (100 3D CT images) were used to demonstrate the proposed method. Each 4DCT includes 10 respiratory phases, and the 2D orthogonal images are generated by forward projecting the 3D CT from each phase to planes. Leave-phase-out experiments were performed to ensure that each phase amount of the 10 phases will be used for testing once. Among all 10 patients, the mean absolute error, peak signal-to-noise ratio, and structural similarity index are 83.29 ± 2.83 HU, 16.86 ± 0.16 dB, and 0.83 ± 0.01. The results indicate that the feature matching network can successfully reconstruct the 3D tumor location using 2D projections, which can support treatment decision-making. The proposed method potentially can benefit proton FLASH therapy by offering real-time target localization and patient dose verification to ensure treatment quality.