Eye-view Imaging Research Articles

Mining High-Quality Pseudoinstance Soft Labels for Weakly Supervised Object Detection in Remote Sensing Images

Weakly supervised object detection in remote sensing image (RSI) is still a challenge because of the lack of instance-level labels, and many existing methods have two problems. Firstly, most of the existing methods usually mine the pseudo ground truth (PGT) instances solely relying on proposal class scores (PCS). Actually, the reliability of PCS is not enough because of the bird’s eye view imaging and large-scale chaotic background of RSIs, and the instances with high PCS incline to cover the discriminative region rather than the whole object. Secondly, the existing methods assign a one-hot label to each instance, and the label of PGT instance is copied to its neighbor instances, which induces the misclassification problem to some extent. Actually, the probability that the neighbor instances contain the object with the same category is smaller than the PGT instance. For the first problem, the proposal quality score (PQS) is proposed for mining high-quality PGT instances, which contains PCS and dual-context projection score (DCPS). The DCPS is calculated through semantic segmentation, and is employed to measure the completeness that each proposal covers an object. For the second problem, a pseudo soft label assignment (PSLA) strategy is proposed to assign more precise soft label for each instance, where the soft label is determined by the spatial distance between each instance and its nearest PGT instance. The ablation study validates the effectiveness of the PQS and PSLA. The comprehensive comparisons with other WSOD methods on three popular benchmarks show the excellent performance of our method.

Beam's-Eye-View Imaging of Liver SBRT With a Novel Multi-Layer Imager

Beams-eye view (BEV) imaging during radiation therapy can be used for real-time localization of the treatment target. However, clinical implementation is limited by the poor performance of current electronic portal imaging devices (EPIDs). A novel multilayer imager (MLI), consisting of four stacked conventional flat-panel imagers, was recently shown to have 5.7 times the detective quantum efficiency and nearly 2 times the contrast-to-noise ratio of a conventional EPID. In the current study, we compare the MLI against a single layer imager for the application of fiducial tracking during liver SBRT procedures. The hypothesis is that the MLI will provide more accurate and efficient fiducial tracking.The prototype MLI was installed in the standard MV imager housing on a clinical LINAC, replacing the conventional EPID. The MV imager arm was extended during volumetric modulated arc therapy (VMAT) SBRT treatments for passive data acquisition. Six patients with liver metastases receiving SBRT in 3-5 fractions (900-1200 cGy/fx) were studied. Data was acquired for two fractions of each treatment course, one fraction with the MLI using all 4 layers and one with the MLI using the top layer only. Automatic fiducial localization was performed using a modification of a previously published algorithm. Truth data was derived using external surrogate respiratory traces combined with manual tracking. Results for 4- and 1-layer modes were compared against truth data to determine tracking accuracy and efficiency. Tracking and noise improvements were assessed for any correlation.Between 2771 to 6505 image frames were acquired for each patient, with roughly half of each corresponding to acquisition in 1-layer mode and the other half 4-layer mode. Tracking accuracy with the 1-layer imager returned a root mean square error (RMSE) of 2.1 mm compared to 4-layer RMSE of 1.5 mm, a statistically significant (P < 0.001) improvement of 0.6 mm. Successful tracking efficiency with 4 layers improved to 82.8% versus 58.4% for the 1-layer mode, a relative improvement of 41.8% (P < 0.001). There was a strong correlation (r = 0.913) between the increase in successfully tracked frames and reduction in noise when using the 4-layer mode.Noise reduction and increased MV photon detection efficiency achieved by utilizing a multi-layer MV imager results in improved fiducial tracking for liver SBRT treatments. Future BEV clinical applications may be improved by pursuing similar noise reduction and photon detection enhancement strategies.

Clinical translation of a new flat-panel detector for beam’s-eye-view imaging

Electronic portal imaging devices (EPIDs) lend themselves to beams-eye view clinical applications, such as tumor tracking, but are limited by low contrast and detective quantum efficiency (DQE). We characterize a novel EPID prototype consisting of multiple layers and investigate its suitability for use under clinical conditions. A prototype multi-layer imager (MLI) was constructed utilizing four conventional EPID layers, each consisting of a copper plate, a Gd2O2S:Tb phosphor scintillator, and an amorphous silicon flat panel array detector. We measured the detector’s response to a 6 MV photon beam with regards to modulation transfer function, noise power spectrum, DQE, contrast-to-noise ratio (CNR), signal-to-noise ratio (SNR), and the linearity of the detector’s response to dose. Additionally, we compared MLI performance to the single top layer of the MLI and the standard Varian AS-1200 detector. Pre-clinical imaging was done on an anthropomorphic phantom, and the detector’s CNR, SNR and spatial resolution were assessed in a clinical environment. Images obtained from spine and liver patient treatment deliveries were analyzed to verify CNR and SNR improvements. The MLI has a DQE(0) of 9.7%, about 5.7 times the reference AS-1200 detector. Improved noise performance largely drives the increase. CNR and SNR of clinical images improved three-fold compared to reference. A novel MLI was characterized and prepared for clinical translation. The MLI substantially improved DQE and CNR performance while maintaining the same resolution. Pre-clinical tests on an anthropomorphic phantom demonstrated improved performance as predicted theoretically. Preliminary patient data were analyzed, confirming improved CNR and SNR. Clinical applications are anticipated to include more accurate soft tissue tracking.

Investigation of planar image quality for a novel 2.5MV diamond target beam from a radiotherapy linear accelerator.

Background and purposeA commercial 2.5 MV beam has been clinically available for beam’s-eye-view imaging in radiotherapy, offering improved contrast-to-noise ratio (CNR) compared to therapeutic beams, due to the softer spectrum. Previous research suggested that imaging performance could be improved using a low-Z diamond target to reduce the self-absorption of diagnostic energy photons. The aim of this study was to 1) investigate the feasibility of two 2.5 MV diamond target beamline configurations and 2) characterize the dosimetry and planar image quality of these novel low-Z beams.Materials and methodsThe commercial 2.5 MV beam was modified by replacing the copper target with sintered diamond. Two beamlines were investigated: a carousel-mounted diamond target beamline and a ‘conventional’ beamline, with the diamond target in the target arm. Planar image quality was assessed in terms of spatial resolution and CNR.ResultsDue to image artifacts, image quality could not be assessed for the carousel-mounted low-Z target beam. The ‘conventional’ 2.5 MV low-Z beam quality was softer by 2.7% compared to the commercial imaging beam, resulting in improved CNR by factors of up to 1.3 and 1.7 in thin and thick phantoms, respectively. In regard to spatial resolution, the ‘conventional’ 2.5 MV low-Z beam slightly outperformed the commercial imaging beam.ConclusionWith a simple modification to the 2.5 MV commercial beamline, we produced an improved energy spectrum for imaging. This 2.5 MV diamond target beam proved to be an advantageous alternative to the commercial target configuration, offering both superior resolution and CNR.

High quality proton portal imaging using deep learning for proton radiation therapy: a phantom study

Purpose; For shoot-through proton treatments, like FLASH radiotherapy, there will be protons exiting the patient which can be used for proton portal imaging (PPI), revealing valuable information for the validation of tumor location in the beam’s-eye-view at native gantry angles. However, PPI has poor inherent contrast and spatial resolution. To deal with this issue, we propose a deep-learning-based method to use kV digitally reconstructed radiographs (DRR) to improve PPI image quality. Method; We used a residual generative adversarial network (GAN) framework to learn the nonlinear mapping between PPIs and DRRs. Residual blocks were used to force the model to focus on the structural differences between DRR and PPI. To assess the accuracy of our method, we used 149 images for training and 30 images for testing. PPIs were acquired using a double-scattered proton beam. The DRRs acquired from CT acted as learning targets in the training process and were used to evaluate results from the proposed method using a six-fold cross-validation scheme. Results; Qualitatively, the corrected PPIs showed enhanced spatial resolution and captured fine details present in the DRRs that are missed in the PPIs. The quantitative results for corrected PPIs show average normalized mean error (NME), normalized mean absolute error (NMAE), peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) index of −0.1%, 0.3%, 39.14 dB, and 0.987, respectively. Conclusion; The results indicate the proposed method can generate high quality corrected PPIs and this work shows the potential to use a deep-learning model to make PPI available in proton radiotherapy. This will allow for beam’s-eye-view (BEV) imaging with the particle used for treatment, leading to a valuable alternative to orthogonal x-rays or cone-beam CT for patient position verification.

Beam's-eye-view imaging during non-coplanar lung SBRT.

Beam's-eye-view (BEV) imaging with an electronic portal imaging device (EPID) can be performed during lung stereotactic body radiation therapy (SBRT) to monitor the tumor location in real-time. Image quality for each patient and treatment field depends on several factors including the patient anatomy and the gantry and couch angles. The authors investigated the angular dependence of automatic tumor localization during non-coplanar lung SBRT delivery. All images were acquired at a frame rate of 12 Hz with an amorphous silicon EPID. A previously validated markerless lung tumor localization algorithm was employed with manual localization as the reference. From ten SBRT patients, 12 987 image frames of 123 image sequences acquired at 48 different gantry-couch rotations were analyzed. δ was defined by the position difference of the automatic and manual localization. Regardless of the couch angle, the best tracking performance was found in image sequences with a gantry angle within 20° of 250° (δ = 1.40 mm). Image sequences acquired with gantry angles of 150°, 210°, and 350° also led to good tracking performances with δ = 1.77-2.00 mm. Overall, the couch angle was not correlated with the tracking results. Among all the gantry-couch combinations, image sequences acquired at (θ = 30°, ϕ = 330°), (θ = 210°, ϕ = 10°), and (θ = 250°, ϕ = 30°) led to the best tracking results with δ = 1.19-1.82 mm. The worst performing combinations were (θ = 90° and 230°, ϕ = 10°) and (θ = 270°, ϕ = 30°) with δ > 3.5 mm. However, 35% (17/48) of the gantry-couch rotations demonstrated substantial variability in tracking performances between patients. For example, the field angle (θ = 70°, ϕ = 10°) was acquired for five patients. While the tracking errors were ≤1.98 mm for three patients, poor performance was found for the other two patients with δ ≥ 2.18 mm, leading to average tracking error of 2.70 mm. Only one image sequence was acquired for all other gantry-couch rotations (δ = 1.18-10.29 mm). Non-coplanar beams with gantry-couch rotation of (θ = 30°, ϕ = 330°), (θ = 210°, ϕ = 10°), and (θ = 250°, ϕ = 30°) have the highest accuracy for BEV lung tumor localization. Additionally, gantry angles of 150°, 210°, 250°, and 350° also offer good tracking performance. The beam geometries (θ = 90° and 230°, ϕ = 10°) and (θ = 270°, ϕ = 30°) are associated with substantial automatic localization errors. Overall, lung tumor visibility and tracking performance were patient dependent for a substantial number of the gantry-couch angle combinations studied.

A piecewise-focused high DQE detector for MV imaging.

Electronic portal imagers (EPIDs) with high detective quantum efficiencies (DQEs) are sought to facilitate the use of the megavoltage (MV) radiotherapy treatment beam for image guidance. Potential advantages include high quality (treatment) beam's eye view imaging, and improved cone-beam computed tomography (CBCT) generating images with more accurate electron density maps with immunity to metal artifacts. One approach to increasing detector sensitivity is to couple a thick pixelated scintillator array to an active matrix flat panel imager (AMFPI) incorporating amorphous silicon thin film electronics. Cadmium tungstate (CWO) has many desirable scintillation properties including good light output, a high index of refraction, high optical transparency, and reasonable cost. However, due to the 0 1 0 cleave plane inherent in its crystalline structure, the difficulty of cutting and polishing CWO has, in part, limited its study relative to other scintillators such as cesium iodide and bismuth germanate (BGO). The goal of this work was to build and test a focused large-area pixelated "strip" CWO detector. A 361 × 52 mm scintillator assembly that contained a total of 28 072 pixels was constructed. The assembly comprised seven subarrays, each 15 mm thick. Six of the subarrays were fabricated from CWO with a pixel pitch of 0.784 mm, while one array was constructed from BGO for comparison. Focusing was achieved by coupling the arrays to the Varian AS1000 AMFPI through a piecewise linear arc-shaped fiber optic plate. Simulation and experimental studies of modulation transfer function (MTF) and DQE were undertaken using a 6 MV beam, and comparisons were made between the performance of the pixelated strip assembly and the most common EPID configuration comprising a 1 mm-thick copper build-up plate attached to a 133 mg/cm(2) gadolinium oxysulfide scintillator screen (Cu-GOS). Projection radiographs and CBCT images of phantoms were acquired. The work also introduces the use of a lightweight edge phantom to generate MTF measurements at MV energies and shows its functional equivalence to the more cumbersome slit-based method. Measured and simulated DQE(0)'s of the pixelated CWO detector were 22% and 26%, respectively. The average measured and simulated ratios of CWO DQE(f) to Cu-GOS DQE(f) across the frequency range of 0.0-0.62 mm(-1) were 23 and 29, respectively. 2D and 3D imaging studies confirmed the large dose efficiency improvement and that focus was maintained across the field of view. In the CWO CBCT images, the measured spatial resolution was 7 lp/cm. The contrast-to-noise ratio was dramatically improved reflecting a 22 × sensitivity increase relative to Cu-GOS. The CWO scintillator material showed significantly higher stability and light yield than the BGO material. An efficient piecewise-focused pixelated strip scintillator for MV imaging is described that offers more than a 20-fold dose efficiency improvement over Cu-GOS.

Deformable motion reconstruction for scanned proton beam therapy using on-line x-ray imaging

Organ motion is a major problem for any dynamic radiotherapy delivery technique, and is particularly so for spot scanned proton therapy. On the other hand, the use of narrow, magnetically deflected proton pencil beams is potentially an ideal delivery technique for tracking tumour motion on-line. At PSI, our new Gantry is equipped with a Beams Eye View (BEV) imaging system which will be able to acquire 2D x-ray images in fluoroscopy mode during treatment delivery. However, besides precisely tracking motion from BEVs, it is also essential to obtain information on the 3D motion vector throughout the whole region of interest, and any sparsely acquired surrogate motion is generally not sufficient to describe the deformable behaviour of the whole volume in three dimensions. In this study, we propose a method by which 3D deformable motions can be estimated from surrogate motions obtained using this monoscopic imaging system. The method assumes that example motions over a number of breathing cycles can be acquired before treatment for each patient using 4DMRI. In this study, for each of 11 different subjects, 100 continuous breathing cycles have been extracted from extended 4DMRI studies in the liver and then subject specific motion models have been built using principle component analysis (PCA). To simulate treatment conditions, a different set of 30 continuous breathing cycles from the same subjects have then been used to generate a set of simulated 4DCT data sets (so-called 4DCT(MRI) data sets), from which time-resolved digitally reconstructed radiographs (DRRs) were calculated using the BEV geometry for three treatment fields respectively. From these DRRs, surrogate motions from fiducial markers or the diaphragm have been used as a predictor to estimate 3D motions in the liver region for each subject. The prediction results have been directly compared to the ‘ground truth’ motions extracted from the same 30 breath cycles of the originating 4DMRI data set. Averaged over all 11 subjects, and for three field directions, for 99% of predicted positions, median (max) error magnitudes of better than 2.63(5.67) mm can be achieved when fiducial markers was chosen as predictor. Furthermore, three single fields, 4D dose calculations have been performed as a verification tool to evaluate the prediction performance of such a model in the context of scanned proton beam therapy. These show a high similarity between plans considering either PCA predicted motion or ground truth motion, where absolute dose differences of more than 5% (Vdosediff = 5%) occur for the worst field scenarios in only 3.61% (median) or 15.13% (max) of dose calculation points in the irradiated volume. The magnitude of these dose differences were insignificantly dependent on whether surrogate motions were tracked by monoscopic or stereoscopic imaging systems, or whether fiducial markers or diaphragm were chosen as surrogate. This study has demonstrated that on-line deformable motion reconstruction from sparse surrogate motions is feasible, even when using only a monoscopic imaging system. In addition, it has also been found that diaphragm motion can be considered as a good predictor for respiratory deformable liver motion prediction, implying that fiducial markers might not be compulsory if used in conjunction with a patient specific PCA based model.

Convergent Radiotherapy: Adaptive Treatment Planning with Delivered Dose Information

Recent studies have shown the value of beam's-eye-view imaging during radiotherapy to monitor the tumor location. This information can further be used to calculate the delivered dose, even in the presence of motion. Convergent Radiotherapy is the replanning of a radiotherapy treatment, in-between fractions, with the cumulative delivered dose included in the re-optimization. The goal is for the delivered dose to converge with the prescription. Although this is distinct from Adaptive Radiotherapy (ART) in that the measured delivered dose is used for the re-optimization, principles of Adaptive Radiotherapy could also be incorporated in Convergent Radiotherapy.

A novel hardware design for image and dose guided radiotherapy

Conventional radiation therapy assumes that possible movements of organs during therapy can either be neglected or taken into account in the planning phase. In this presentation, we describe a novel imaging hardware set-up to overcome this limitation. The imaging hardware was designed and integrated into a standard medical accelerator to provide a tomographic and a beams eye view imaging which provides a radiographic localisation of bone and soft-tissue targets. Furthermore this novel imaging hardware provides also a system for verifying of the delivered dose by means of entrance dosimetry. For imaging purposes a kV X-ray tube (POWERPHOS, SIEMENS) was integrated with a SIEMENS Primus linear accelerator. The kV X-ray tube was mounted opposite to the source of the therapy beam. As a detector a flat-panel device RID1640 (Perkin Elmer Optoelectronics, Germany) was mounted on the side of the therapy beam in front of the patient. The amorphous silicon flat-panel imager provides 1024 × 1024 pixels with 400 mm pitch and an active area of 40.96cm × 40.96cm. While the flat-panel is irradiated with the kV x-ray from its front side, the detector is irradiated from the back side with the MV therapy beam. This imaging set-up provides on one hand a tomographic cone beam imaging system which allows monitoring the patient’s anatomy in treatment position just before the start of the treatment fraction. On the other hand it allows acquiring an image of a volume of interest of the patient e.g. the treatment volume in fluoroscopic mode during therapy beam application utilising the pulse recess of the Linac beam. Because both the therapy beam and the diagnostic x-ray beam are mounted on the same beam axis the fluoroscopic images can be used to match these fluoroscopic views with images of the shape of the treatment. These images are acquired nearly at the same time just with the same flat-panel detector by an interlaced read out of the detector between MV and kV X-ray beam. For CT image reconstruction we use a standard 3D algorithm (Feldkamp, Kress). The flat-panel imager is used as a 2D beam monitor to record the intensity modulated fluence distribution of incoming photons of the therapy beam. Different sample IMRT plans based on a solid water phantom and an anthropomorphic head and body phantom were generated using the Virtuos/KonRad inverse planning system (DKFZ / SIEMENS, Heidelberg, Germany). The IMRT beams from these pre-calculated treatment plans were delivered and recorded with the flat-panel imager by acquiring sequences of images. The treatment set-up of the phantom was verified by the imaging system. Compared to the dose of a conventional CT first results of the tomographic cone beam images show with a similar dose a contrast resolution of about 1% for objects larger than 3mm and a spatial resolution of about 1mm for high contrast objects. A very good agreement could be seen between planned and measured entrance fluence distributions for each IMRT beam. The error was calculated to be less than ±2%. An error of less then ±2mm was seen for isodose lines down to the 50% line. For the lower isodose lines or regions with steep dose gradients the error somewhat increases. Different dose measurements in a water tank indicate that the therapy beam quality is not significant affected by the flat-panel. Only the absolute dose is reduced by 10% because of the additional attenuation. We have integrated an imaging system with a medical linear accelerator for kV tomographic and beam’s eye view imaging which provides a radiographic target localisation and for delivered therapy beam verification. Preliminary results show that image acquisition in tomographic and radiographic mode gives high quality images, sufficient for patient set-up and for target localisation before and during therapy beam application. A method of in vivo dose reconstruction based on measurements of entrance beams was developed and tested by means of comparisons with sample IMRT plans. Generally good agreement was seen between planning, reconstruction and direct measurement of the three-dimensional dose distribution

A novel hardware design for image and dose guided radiotherapy

Conventional radiation therapy assumes that possible movements of organs during therapy can either be neglected or taken into account in the planning phase. In this presentation, we describe a novel imaging hardware set-up to overcome this limitation. The imaging hardware was designed and integrated into a standard medical accelerator to provide a tomographic and a beams eye view imaging which provides a radiographic localisation of bone and soft-tissue targets. Furthermore this novel imaging hardware provides also a system for verifying of the delivered dose by means of entrance dosimetry.