Single X-ray Projection Research Articles

RT-SRTS: Angle-agnostic real-time simultaneous 3D reconstruction and tumor segmentation from single X-ray projection

Radiotherapy is one of the primary treatment methods for tumors, but the organ movement caused by respiration limits its accuracy. Recently, 3D imaging from a single X-ray projection has received extensive attention as a promising approach to address this issue. However, current methods can only reconstruct 3D images without directly locating the tumor and are only validated for fixed-angle imaging, which fails to fully meet the requirements of motion control in radiotherapy. In this study, a novel imaging method RT-SRTS is proposed which integrates 3D imaging and tumor segmentation into one network based on multi-task learning (MTL) and achieves real-time simultaneous 3D reconstruction and tumor segmentation from a single X-ray projection at any angle. Furthermore, the attention enhanced calibrator (AEC) and uncertain-region elaboration (URE) modules have been proposed to aid feature extraction and improve segmentation accuracy. The proposed method was evaluated on fifteen patient cases and compared with three state-of-the-art methods. It not only delivers superior 3D reconstruction but also demonstrates commendable tumor segmentation results. Simultaneous reconstruction and segmentation can be completed in approximately 70 ms, significantly faster than the required time threshold for real-time tumor tracking. The efficacies of both AEC and URE have also been validated in ablation studies. The code of work is available at https://github.com/ZywooSimple/RT-SRTS.

ACnerf: enhancement of neural radiance field by alignment and correction of pose to reconstruct new views from a single x-ray* *This work was supported in part by the Science and Technology Commission of Shanghai Municipality under Grant 20DZ2254400 and 20DZ2261200; in part by Shanghai Municipal Science and Technology Major Project under Grant ZD2021CY001.

Objective. Computed tomography (CT) is widely used in medical research and clinical diagnosis. However, acquiring CT data requires patients to be exposed to considerable ionizing radiance, leading to physical harm. Recent studies have considered using neural radiance field (NERF) techniques to infer the full-view CT projections from single-view x-ray projection, thus aiding physician judgment and reducing Radiance hazards. This paper enhances this technique in two directions: (1) accurate generalization capabilities for control models. (2) Consider different ranges of viewpoints. Approach. Building upon generative radiance fields (GRAF), we propose a method called ACnerf to enhance the generalization of the NERF through alignment and pose correction. ACnerf aligns with a reference single x-ray by utilizing a combination of positional encoding with Gaussian random noise (latent code) obtained from GRAF training. This approach avoids compromising the 3D structure caused by altering the generator. During inference, a pose judgment network is employed to correct the pose and optimize the rendered viewpoint. Additionally, when generating a narrow range of views, ACnerf employs frequency-domain regularization to fine-tune the generator and achieve precise projections. Main results. The proposed ACnerf method surpasses the state-of-the-art NERF technique in terms of rendering quality for knee and chest data with varying contrasts. It achieved an average improvement of 2.496 dB in PSNR and 41% in LPIPS for 0°–360° projections. Additionally, for −15° to 15° projections, ACnerf achieved an average improvement of 0.691 dB in PSNR and 25.8% in LPIPS. Significance. With adjustments in alignment, inference, and rendering range, our experiments and evaluations on knee and chest data of different contrasts show that ACnerf effectively reduces artifacts and aberrations in the new view. ACnerf’s ability to recover more accurate 3D structures from single x-rays has excellent potential for reducing damage from ionising radiation in clinical diagnostics.

Volumetric tumor tracking from a single cone-beam X-ray projection image enabled by deep learning

Radiotherapy serves as a pivotal treatment modality for malignant tumors. However, the accuracy of radiotherapy is significantly compromised due to respiratory-induced fluctuations in the size, shape, and position of the tumor. To address this challenge, we introduce a deep learning-anchored, volumetric tumor tracking methodology that employs single-angle X-ray projection images. This process involves aligning the intraoperative two-dimensional (2D) X-ray images with the pre-treatment three-dimensional (3D) planning Computed Tomography (CT) scans, enabling the extraction of the 3D tumor position and segmentation. Prior to therapy, a bespoke patient-specific tumor tracking model is formulated, leveraging a hybrid data augmentation, style correction, and registration network to create a mapping from single-angle 2D X-ray images to the corresponding 3D tumors. During the treatment phase, real-time X-ray images are fed into the trained model, producing the respective 3D tumor positioning. Rigorous validation conducted on actual patient lung data and lung phantoms attests to the high localization precision of our method at lowered radiation doses, thus heralding promising strides towards enhancing the precision of radiotherapy.

Patient-specific three-dimensional image reconstruction from a single X-ray projection using a convolutional neural network for on-line radiotherapy applications.

Background and purpose: Radiotherapy is commonly chosen to treat thoracic and abdominal cancers. However, irradiating mobile tumors accurately is extremely complex due to the organs' breathing-related movements. Different methods have been studied and developed to treat mobile tumors properly. The combination of X-ray projection acquisition and implanted markers is used to locate the tumor in two dimensions (2D) but does not provide three-dimensional (3D) information. The aim of this work is to reconstruct a high-quality 3D computed tomography (3D-CT) image based on a single X-ray projection to locate the tumor in 3D without the need for implanted markers. Materials and Methods: Nine patients treated for a lung or liver cancer in radiotherapy were studied. For each patient, a data augmentation tool was used to create 500 new 3D-CT images from the planning four-dimensional computed tomography (4D-CT). For each 3D-CT, the corresponding digitally reconstructed radiograph was generated, and the 500 2D images were input into a convolutional neural network that then learned to reconstruct the 3D-CT. The dice score coefficient, normalized root mean squared error and difference between the ground-truth and the predicted 3D-CT images were computed and used as metrics. Results: Metrics' averages across all patients were 85.5% and 96.2% for the gross target volume, 0.04 and 0.45 Hounsfield unit (HU), respectively. Conclusions: The proposed method allows reconstruction of a 3D-CT image from a single digitally reconstructed radiograph that could be used in real-time for better tumor localization and improved treatment of mobile tumors without the need for implanted markers.

Real-time liver tumor localization via combined surface imaging and a single x-ray projection

Objective. Real-time imaging, a building block of real-time adaptive radiotherapy, provides instantaneous knowledge of anatomical motion to drive delivery adaptation to improve patient safety and treatment efficacy. The temporal constraint of real-time imaging (<500 milliseconds) significantly limits the imaging signals that can be acquired, rendering volumetric imaging and 3D tumor localization extremely challenging. Real-time liver imaging is particularly difficult, compounded by the low soft tissue contrast within the liver. We proposed a deep learning (DL)-based framework (Surf-X-Bio), to track 3D liver tumor motion in real-time from combined optical surface image and a single on-board x-ray projection. Approach. Surf-X-Bio performs mesh-based deformable registration to track/localize liver tumors volumetrically via three steps. First, a DL model was built to estimate liver boundary motion from an optical surface image, using learnt motion correlations between the respiratory-induced external body surface and liver boundary. Second, the residual liver boundary motion estimation error was further corrected by a graph neural network-based DL model, using information extracted from a single x-ray projection. Finally, a biomechanical modeling-driven DL model was applied to solve the intra-liver motion for tumor localization, using the liver boundary motion derived via prior steps. Main results. Surf-X-Bio demonstrated higher accuracy and better robustness in tumor localization, as compared to surface-image-only and x-ray-only models. By Surf-X-Bio, the mean (±s.d.) 95-percentile Hausdorff distance of the liver boundary from the ‘ground-truth’ decreased from 9.8 (±4.5) (before motion estimation) to 2.4 (±1.6) mm. The mean (±s.d.) center-of-mass localization error of the liver tumors decreased from 8.3 (±4.8) to 1.9 (±1.6) mm. Significance. Surf-X-Bio can accurately track liver tumors from combined surface imaging and x-ray imaging. The fast computational speed (<250 milliseconds per inference) allows it to be applied clinically for real-time motion management and adaptive radiotherapy.

Keep Attention to the Mapping:Application of AI for Geometric X-Ray CT Scan Calibration

This paper presents a method for mapping markers from CT calibration bodies in X-ray projections. In twin robotic CT systems, determining projection geometry from projection views is essential for reconstruction. Many algorithms, such as direct linear transformation (DLT), use a calibration body for this purpose. This consists of individual markers that must be individually assigned in a single X-ray projection. In the described approach, a transformation model is used to map the coordinates of the 2D marker centers to the correct index. This allows the use of freely designed calibration bodies, where the arrangement of the markers is not limited to geometric arrangements.

CT-based data generation for foreign object detection on a single X-ray projection

Although X-ray imaging is used routinely in industry for high-throughput product quality control, its capability to detect internal defects has strong limitations. The main challenge stems from the superposition of multiple object features within a single X-ray view. Deep Convolutional neural networks can be trained by annotated datasets of X-ray images to detect foreign objects in real-time. However, this approach depends heavily on the availability of a large amount of data, strongly hampering the viability of industrial use with high variability between batches of products. We present a computationally efficient, CT-based approach for creating artificial single-view X-ray data based on just a few physically CT-scanned objects. By algorithmically modifying the CT-volume, a large variety of training examples is obtained. Our results show that applying the generative model to a single CT-scanned object results in image analysis accuracy that would otherwise be achieved with scans of tens of real-world samples. Our methodology leads to a strong reduction in training data needed, improved coverage of the combinations of base and foreign objects, and extensive generalizability to additional features. Once trained on just a single CT-scanned object, the resulting deep neural network can detect foreign objects in real-time with high accuracy.

Real-time liver tumor localization via a single x-ray projection using deep graph neural network-assisted biomechanical modeling

Objective. Real-time imaging is highly desirable in image-guided radiotherapy, as it provides instantaneous knowledge of patients’ anatomy and motion during treatments and enables online treatment adaptation to achieve the highest tumor targeting accuracy. Due to extremely limited acquisition time, only one or few x-ray projections can be acquired for real-time imaging, which poses a substantial challenge to localize the tumor from the scarce projections. For liver radiotherapy, such a challenge is further exacerbated by the diminished contrast between the tumor and the surrounding normal liver tissues. Here, we propose a framework combining graph neural network-based deep learning and biomechanical modeling to track liver tumor in real-time from a single onboard x-ray projection. Approach. Liver tumor tracking is achieved in two steps. First, a deep learning network is developed to predict the liver surface deformation using image features learned from the x-ray projection. Second, the intra-liver deformation is estimated through biomechanical modeling, using the liver surface deformation as the boundary condition to solve tumor motion by finite element analysis. The accuracy of the proposed framework was evaluated using a dataset of 10 patients with liver cancer. Main results. The results show accurate liver surface registration from the graph neural network-based deep learning model, which translates into accurate, fiducial-less liver tumor localization after biomechanical modeling (<1.2 (±1.2) mm average localization error). Significance. The method demonstrates its potentiality towards intra-treatment and real-time 3D liver tumor monitoring and localization. It could be applied to facilitate 4D dose accumulation, multi-leaf collimator tracking and real-time plan adaptation. The method can be adapted to other anatomical sites as well.

MO-0215 3DCT reconstruction from single X-ray projection using Deep Learning

MO-0215 3DCT reconstruction from single X-ray projection using Deep Learning

Biomedical image restoration using machine learning GPU acceleration approach for precision improvement

Focusing the image in a single x-ray projection, an algorithm proposed for actual time volumetric image rebuilt and 3-dimensional location of the lump. Using the Principal Component Analysis (PCA) initialize the parameterized Deformation Vector Fields (DVF) of pulmonary movement. By adjusting the PCA coefficients, applied the DVF applied to optimize the reference image so that, the simulated projection of the rebuilt image matches the observed projection. The digital phantom &amp; patient information was used to evaluate the technique. The phantom has an average relative image rebuilt error of 7.5 percent and a 3-dimensional location of the lump inaccuracy of 0.9 mm, correspondingly. The patient's location of the lump inaccuracy is less than 2 mm. On a GPU NVIDIA C1060, recreating a single volumetric image from every projection takes about 0.2 &amp; 0.3 secs for patient and phantom. From a single image, clinical relevance could lead to reliable 3-dimensional lump tracking.

A patient-independent CT intensity matching method using conditional generative adversarial networks (cGAN) for single x-ray projection-based tumor localization

A convolutional neural network (CNN)-based tumor localization method with a single x-ray projection was previously developed by us. One finding is that the discrepancy in the discrepancy in the intensity between a digitally reconstructed radiograph (DRR) of a three-dimensional computed tomography (3D-CT) and the measured x-ray projection has an impact on the performance. To address this issue, a patient-dependent intensity matching process for 3D-CT was performed using 3D-cone-beam computed tomography (3D-CBCT) from the same patient, which was sometimes inefficient and could adversely affect the clinical implementation of the framework. To circumvent this, in this work, we propose and validate a patient-independent intensity matching method based on a conditional generative adversarial network (cGAN). A 3D cGAN was trained to approximate the mapping from 3D-CT to 3D-CBCT from previous patient data. By applying the trained network to a new patient, a synthetic 3D-CBCT could be generated without the need to perform an actual CBCT scan on that patient. The DRR of the synthetic 3D-CBCT was subsequently utilized in our CNN-based tumor localization scheme. The method was tested using data from 12 patients with the same imaging parameters. The resulting 3D-CBCT and DRR were compared with real ones to demonstrate the efficacy of the proposed method. The tumor localization errors were also analyzed. The difference between the synthetic and real 3D-CBCT had a median value of no more than 10 HU for all patients. The relative error between the DRR and the measured x-ray projection was less than 4.8% ± 2.0% for all patients. For the three patients with a visible tumor in the x-ray projections, the average tumor localization errors were below 1.7 and 0.9 mm in the superior-inferior and lateral directions, resepectively. A patient-independent CT intensity matching method was developed, based on which accurate tumor localization was achieved. It does not require an actual CBCT scan to be performed before treatment for each patient, therefore making it more efficient in the clinical workflow.

Material decomposition from a single x-ray projection via single-grid phase contrast imaging.

This study describes a new approach for material decomposition in x-ray imaging, utilizing phase contrast both to increase sensitivity to weakly attenuating samples and to act as a complementary measurement to attenuation, therefore allowing two overlaid materials to be separated. The measurements are captured using the single-exposure, single-grid x-ray phase contrast imaging technique, with a novel correction that aims to remove propagation-based phase effects seen at sharp edges in the attenuation image. The use of a single-exposure technique means that images can be collected in a high-speed sequence. Results are shown for both a known two-material sample and for a biological specimen.

Real-time tumor localization with single x-ray projection at arbitrary gantry angles using a convolutional neural network (CNN)

For tumor tracking therapy, precise knowledge of tumor position in real-time is very important. A technique using single x-ray projection based on a convolutional neural network (CNN) was recently developed which can achieve accurate tumor localization in real-time. However, this method was only validated at fixed gantry angles. In this study, an improved technique is developed to handle arbitrary gantry angles for rotational radiotherapy.To evaluate the highly complex relationship between x-ray projections at arbitrary angles and tumor motion, a special CNN was proposed. In this network, a binary region of interest (ROI) mask was applied on every extracted feature map. This avoids the overfitting problem due to gantry rotation by directing the network to neglect those irrelevant pixels whose intensity variation had nothing to do with breathing motion. In addition, an angle-dependent fully connection layer (ADFCL) was utilized to recover the mapping from extracted feature maps to tumor motion, which would vary with the gantry angles. The method was tested with images from 15 realistic patients and compared with a variant network of VGG, developed by Oxford University’s Visual Geometry Group.The tumors were clearly visible on x-ray projections for five patients only. The average tumor localization error was under 1.8 mm and 1.0 mm in superior-inferior and lateral directions. For the other ten patients whose tumors were not clearly visible in the x-ray projection, a feature point localization error was computed to evaluate the proposed method, the mean value of which was no more than 1.5 mm and 1.0 mm in both directions for all patients.A tumor localization method for single x-ray projection at arbitrary angles based on a novel CNN was developed and validated in this study for real-time operation. This greatly expanded the applicability of the tumor localization framework to the rotation therapy.

Simultaneous reconstruction of multiple stiff wires from a single X-ray projection for endovascular aortic repair

Endovascular repair of aortic aneurysms (EVAR) can be supported by fusing pre- and intraoperative data to allow for improved navigation and to reduce the amount of contrast agent needed during the intervention. However, stiff wires and delivery devices can deform the vasculature severely, which reduces the accuracy of the fusion. Knowledge about the 3D position of the inserted instruments can help to transfer these deformations to the preoperative information. We propose a method to simultaneously reconstruct the stiff wires in both iliac arteries based on only a single monoplane acquisition, thereby avoiding interference with the clinical workflow. In the available X-ray projection, the 2D course of the wire is extracted. Then, a virtual second view of each wire orthogonal to the real projection is estimated using the preoperative vessel anatomy from a computed tomography angiography as prior information. Based on the real and virtual 2D wire courses, the wires can then be reconstructed in 3D using epipolar geometry. We achieve a mean modified Hausdorff distance of 4.2mm between the estimated 3D position and the true wire course for the contralateral side and 4.5mm for the ipsilateral side. The accuracy and speed of the proposed method allow for use in an intraoperative setting of deformation correction for EVAR.

Convolutional Neural Network (CNN) Based Three Dimensional Tumor Localization Using Single X-Ray Projection

Accurate localization of lung tumor in real time based on a single X-ray projection is of great interest to the tumor-tracking radiotherapy but is very challenging. In this paper, a convolutional neural network (CNN)-based tumor localization method was proposed to address this problem with the aid of principal component analysis-based motion modeling. A CNN regression model was trained before treatment to recover the ill-conditioned nonlinear mapping from the single X-ray projection to the tumor motion. Novel intensity correction and data augmentation techniques were adopted to improve the model's robustness to the scatter and noise in the X-ray projection image. During treatment, the volumetric image and tumor position could be obtained by applying the CNN model on the acquired X-ray projection. This method was validated and compared with the other state-of-the-art methods on three real patient data. It was found that the proposed method could achieve real-time tumor localization with much higher accuracy (<;1 mm) and robustness.

Time-resolved 2D concentration maps in flowing suspensions using X-ray

In this paper, we introduce a new technique based on X-ray radiography with high temporal (O(0.1 s)) and spatial (O(10 μm)) resolutions to study fast suspension flows regardless of optical access. We benefit from the axial symmetry of our flow configuration, a wide gap Couette setup, to extract a 3D solid volume fraction field from a single X-ray projection image. We propose a mathematical algorithm based on the inversion of Abel transform in conjunction with H1 regularization and data denoising to measure the solid volume fraction field in suspensions in a fraction of a second. We show that the results are in excellent agreement with those obtained from micro Computed Tomography (CT scan) in one hour. This significant reduction in the data acquisition time opens a new avenue in the field of suspensions. As a proof of concept, we study the kinetics of shear-induced migration for suspensions of particles in both Newtonian and yield stress suspending fluids. The latter experiments include two different conditions: With and without a plug region. In both cases, we are able to capture in detail the kinetics of migration. In the presence of a plug region, we manage to accurately describe the particle accumulation at the interface between the sheared and the static regions. Remarkably, even in the absence of sedimentation, the concentration profiles show a complex 2D structure, with no z-invariant region, which illustrates the strong impact of top and bottom boundary effects on migration. We also show the importance of boundary effects on the shear induced migration of particles in a Newtonian suspending fluid. This further shows the necessity of developing techniques that give access to the full spatial concentration field, as the one we present here.

Theranostics with radiation-induced ultrasound emission (TRUE)

Two novel ultrasound imaging techniques with imaging contrast mechanisms are in the works: X-ray-induced acoustic computed tomography (XACT), and nanoscale photoacoustic tomography (nPAT). XACT has incredible potential in: (1) biomedical imaging, through which a 3D image can be generated using only a single X-ray projection, and (2) radiation dosimetry. nPAT as a new alternative of super-resolution microscopy can break through the optical diffraction limit and is capable of exploring sub-cellular structures without reliance on fluorescence labeling. We expect these new imaging techniques to find widespread applications in both pre-clinical and clinical biomedical research.

Evaluation of low-dose limits in 3D-2D rigid registration for surgical guidance

An algorithm for intensity-based 3D-2D registration of CT and C-arm fluoroscopy is evaluated for use in surgical guidance, specifically considering the low-dose limits of the fluoroscopic x-ray projections. The registration method is based on a framework using the covariance matrix adaptation evolution strategy (CMA-ES) to identify the 3D patient pose that maximizes the gradient information similarity metric. Registration performance was evaluated in an anthropomorphic head phantom emulating intracranial neurosurgery, using target registration error (TRE) to characterize accuracy and robustness in terms of 95% confidence upper bound in comparison to that of an infrared surgical tracking system. Three clinical scenarios were considered: (1) single-view image + guidance, wherein a single x-ray projection is used for visualization and 3D-2D guidance; (2) dual-view image + guidance, wherein one projection is acquired for visualization, combined with a second (lower-dose) projection acquired at a different C-arm angle for 3D-2D guidance; and (3) dual-view guidance, wherein both projections are acquired at low dose for the purpose of 3D-2D guidance alone (not visualization). In each case, registration accuracy was evaluated as a function of the entrance surface dose associated with the projection view(s). Results indicate that images acquired at a dose as low as 4 μGy (approximately one-tenth the dose of a typical fluoroscopic frame) were sufficient to provide TRE comparable or superior to that of conventional surgical tracking, allowing 3D-2D guidance at a level of dose that is at most 10% greater than conventional fluoroscopy (scenario #2) and potentially reducing the dose to approximately 20% of the level in a conventional fluoroscopically guided procedure (scenario #3).

3D tumor localization through real-time volumetric x-ray imaging for lung cancer radiotherapy

To evaluate an algorithm for real-time 3D tumor localization from a single x-ray projection image for lung cancer radiotherapy. Recently, we have developed an algorithm for reconstructing volumetric images and extracting 3D tumor motion information from a single x-ray projection [Li et al., Med. Phys. 37, 2822-2826 (2010)]. We have demonstrated its feasibility using a digital respiratory phantom with regular breathing patterns. In this work, we present a detailed description and a comprehensive evaluation of the improved algorithm. The algorithm was improved by incorporating respiratory motion prediction. The accuracy and efficiency of using this algorithm for 3D tumor localization were then evaluated on (1) a digital respiratory phantom, (2) a physical respiratory phantom, and (3) five lung cancer patients. These evaluation cases include both regular and irregular breathing patterns that are different from the training dataset. For the digital respiratory phantom with regular and irregular breathing, the average 3D tumor localization error is less than 1 mm which does not seem to be affected by amplitude change, period change, or baseline shift. On an NVIDIA Tesla C1060 graphic processing unit (GPU) card, the average computation time for 3D tumor localization from each projection ranges between 0.19 and 0.26 s, for both regular and irregular breathing, which is about a 10% improvement over previously reported results. For the physical respiratory phantom, an average tumor localization error below 1 mm was achieved with an average computation time of 0.13 and 0.16 s on the same graphic processing unit (GPU) card, for regular and irregular breathing, respectively. For the five lung cancer patients, the average tumor localization error is below 2 mm in both the axial and tangential directions. The average computation time on the same GPU card ranges between 0.26 and 0.34 s. Through a comprehensive evaluation of our algorithm, we have established its accuracy in 3D tumor localization to be on the order of 1 mm on average and 2 mm at 95 percentile for both digital and physical phantoms, and within 2 mm on average and 4 mm at 95 percentile for lung cancer patients. The results also indicate that the accuracy is not affected by the breathing pattern, be it regular or irregular. High computational efficiency can be achieved on GPU, requiring 0.1-0.3 s for each x-ray projection.

Real-time 3D Tumor Localization and Volumetric Image Reconstruction using a Single X-ray Projection Image for Lung Cancer Radiotherapy

Real-time 3D Tumor Localization and Volumetric Image Reconstruction using a Single X-ray Projection Image for Lung Cancer Radiotherapy