Ultrasound-guided Radiation Therapy Research Articles

Long-short diffeomorphism memory network for weakly-supervised ultrasound landmark tracking

Ultrasound is a promising medical imaging modality benefiting from low-cost and real-time acquisition. Accurate tracking of an anatomical landmark has been of high interest for various clinical workflows such as minimally invasive surgery and ultrasound-guided radiation therapy. However, tracking an anatomical landmark accurately in ultrasound video is very challenging, due to landmark deformation, visual ambiguity and partial observation. In this paper, we propose a long-short diffeomorphism memory network (LSDM), which is a multi-task framework with an auxiliary learnable deformation prior to supporting accurate landmark tracking. Specifically, we design a novel diffeomorphic representation, which contains both long and short temporal information stored in separate memory banks for delineating motion margins and reducing cumulative errors. We further propose an expectation maximization memory alignment (EMMA) algorithm to iteratively optimize both the long and short deformation memory, updating the memory queue for mitigating local anatomical ambiguity. The proposed multi-task system can be trained in a weakly-supervised manner, which only requires few landmark annotations for tracking and zero annotation for deformation learning. We conduct extensive experiments on both public and private ultrasound landmark tracking datasets. Experimental results show that LSDM can achieve better or competitive landmark tracking performance with a strong generalization capability across different scanner types and different ultrasound modalities, compared with other state-of-the-art methods.

Strategy for automatic ultrasound (US) probe positioning in robot-assisted ultrasound guided radiation therapy

Objective. As part of image-guided radiotherapy, ultrasound-guided radiotherapy is currently already in use and under investigation for robot assisted systems Ipsen 2021. It promises a real-time tumor localization during irradiation (intrafractional) without extra dose. The ultrasound probe is held and guided by a robot. However, there is a lack of basic safety mechanisms and interaction strategies to enable a safe clinical procedure. In this study we investigate potential positioning strategies with safety mechanisms for a safe robot-human-interaction. Approach. A compact setup of ultrasound device, lightweight robot, tracking camera, force sensor and control computer were integrated in a software application to represent a potential USgRT setup. For the realization of a clinical procedure, positioning strategies for the ultrasound head with the help of the robot were developed, implemented, and tested. In addition, basic safety mechanisms for the robot have been implemented, using the integrated force sensor, and have been tested by intentional collisions. Main results. Various positioning methods from manual guidance to completely automated procedures were tested. Robot-guided methods achieved higher positioning accuracy and were faster in execution compared to conventional hand-guided methods. The developed safety mechanisms worked as intended and the detected collision force were below 20 N. Significance. The study demonstrates the feasibility of a new approach for safe robotic ultrasound imaging, with a focus on abdominal usage (liver, prostate, kidney). The safety measures applied here can be extended to other human-robot interactions and present the basic for further studies in medical applications.

AI-based optimization for US-guided radiation therapy of the prostate

ObjectivesFast volumetric ultrasound presents an interesting modality for continuous and real-time intra-fractional target tracking in radiation therapy of lesions in the abdomen. However, the placement of the ultrasound probe close to the target structures leads to blocking some beam directions.MethodsTo handle the combinatorial complexity of searching for the ultrasound-robot pose and the subset of optimal treatment beams, we combine CNN-based candidate beam selection with simulated annealing for setup optimization of the ultrasound robot, and linear optimization for treatment plan optimization into an AI-based approach. For 50 prostate cases previously treated with the CyberKnife, we study setup and treatment plan optimization when including robotic ultrasound guidance.ResultsThe CNN-based search substantially outperforms previous randomized heuristics, increasing coverage from 93.66 to 97.20% on average. Moreover, in some cases the total MU was also reduced, particularly for smaller target volumes. Results after AI-based optimization are similar for treatment plans with and without beam blocking due to ultrasound guidance.ConclusionsAI-based optimization allows for fast and effective search for configurations for robotic ultrasound-guided radiation therapy. The negative impact of the ultrasound robot on the plan quality can successfully be mitigated resulting only in minor differences.

Development of a robot-assisted ultrasound-guided radiation therapy (USgRT).

Radiation treatment is improved by the use of image-guided workflows. This work pursues the approach of using ultrasound (US) as a real-time imaging modality. The primary focus of this study is to develop and test a breathing and motion control for a robotic-guided US transducer. All control functions of the robot and the US image processing were then integrated into one software platform enabling US-guided radiation therapy. The robot (KUKA LBR iiwa 7 R800) and the US image processing workflows were integrated into the Medical Interaction Toolkit (MITK) (Nolden et al. in Int J Comput Assist Radiol Surg 8(4):607-620, 2013). The positions of the US probe were tracked with an optical tracking system. As a main function of robot positioning control, a highly sensitive breathing and motion compensation method was developed using KUKA's robotic application programming interface. The resulting autonomous robot motions were tested by the use of defined breathing patterns with two volunteers. Furthermore, a filter pipeline for 3D US image processing with MITK was developed. Thus, image registration of US images and previously acquired planning image data was enabled. The implemented breathing and motion compensation feature was successful with the addition of the remote rotating, translating capability of the US probe. Desired force applied to the US probe, and thus to the patient, is stable and enables a continuous US imaging. The developed filter pipeline for image processing facilitates registration and display of planning data and US image data in one graphical user interface. A stable and robust method for motion compensation for robot-assisted US imaging was developed and tested successfully. This is a first step toward the safe use of autonomous robot motions in interaction with patients. Furthermore, main software components were integrated into a single platform and used with the purpose of ultrasound-guided radiation therapy.

Optimizing robot motion for robotic ultrasound-guided radiation therapy

An important aspect of robotic radiation therapy is active compensation of target motion. Recently, ultrasound has been proposed to obtain real-time volumetric images of abdominal organ motion. One approach to realize flexible probe placement throughout the treatment fraction is based on a robotic arm holding the ultrasound probe. However, the probe and the robot holding it may obstruct some of the beams with a potentially adverse effect on the plan quality. This can be mitigated by using a kinematically redundant robot, which allows maintaining a steady pose of the ultrasound probe while moving its elbow in order to minimize beam blocking. Ultimately, the motion of both the beam source carrying and the ultrasound probe holding robot contributes to the overall treatment time, i.e. beam delivery and robot motion. We propose an approach to optimize the motion and coordination of both robots based on a generalized traveling salesman problem. Furthermore, we study an application of the model to a prostate treatment scenario. Because the underlying optimization problem is hard, we compare results from a state-of-the-art heuristic solver and an approximation scheme with low computational effort. Our results show that integration of the robot holding the ultrasound probe is feasible with acceptable overhead in overall treatment time. For clinically realistic velocities of the robots, the overhead is less than 4% which is a small cost for the added benefit of continuous, volumetric, and non-ionizing tracking of organ motion over periodic x-ray-based tracking.

Ultrasound prostate segmentation based on multidirectional deeply supervised V-Net.

Transrectal ultrasound (TRUS) is a versatile and real-time imaging modality that is commonly used in image-guided prostate cancer interventions (e.g., biopsy and brachytherapy). Accurate segmentation of the prostate is key to biopsy needle placement, brachytherapy treatment planning, and motion management. Manual segmentation during these interventions is time-consuming and subject to inter- and intraobserver variation. To address these drawbacks, we aimed to develop a deep learning-based method which integrates deep supervision into a three-dimensional (3D) patch-based V-Net for prostate segmentation. We developed a multidirectional deep-learning-based method to automatically segment the prostate for ultrasound-guided radiation therapy. A 3D supervision mechanism is integrated into the V-Net stages to deal with the optimization difficulties when training a deep network with limited training data. We combine a binary cross-entropy (BCE) loss and a batch-based Dice loss into the stage-wise hybrid loss function for a deep supervision training. During the segmentation stage, the patches are extracted from the newly acquired ultrasound image as the input of the well-trained network and the well-trained network adaptively labels the prostate tissue. The final segmented prostate volume is reconstructed using patch fusion and further refined through a contour refinement processing. Forty-four patients' TRUS images were used to test our segmentation method. Our segmentation results were compared with the manually segmented contours (ground truth). The mean prostate volume Dice similarity coefficient (DSC), Hausdorff distance (HD), mean surface distance (MSD), and residual mean surface distance (RMSD) were 0.92±0.03, 3.94±1.55, 0.60±0.23, and 0.90±0.38mm, respectively. We developed a novel deeply supervised deep learning-based approach with reliable contour refinement to automatically segment the TRUS prostate, demonstrated its clinical feasibility, and validated its accuracy compared to manual segmentation. The proposed technique could be a useful tool for diagnostic and therapeutic applications in prostate cancer.

Feasibility of Image Registration for Ultrasound-Guided Prostate Radiotherapy Based on Similarity Measurement by a Convolutional Neural Network.

Purpose:Registration of 3-dimensional ultrasound images poses a challenge for ultrasound-guided radiation therapy of the prostate since ultrasound image content changes significantly with anatomic motion and ultrasound probe position. The purpose of this work is to investigate the feasibility of using a pretrained deep convolutional neural network for similarity measurement in image registration of 3-dimensional transperineal ultrasound prostate images.Methods:We propose convolutional neural network-based registration that maximizes a similarity score between 2 identical in size 3-dimensional regions of interest: one encompassing the prostate within a simulation (reference) 3-dimensional ultrasound image and another that sweeps different spatial locations around the expected prostate position within a pretreatment 3-dimensional ultrasound image. The similarity score is calculated by (1) extracting pairs of corresponding 2-dimensional slices (patches) from the regions of interest, (2) providing these pairs as an input to a pretrained convolutional neural network which assigns a similarity score to each pair, and (3) calculating an overall similarity by summing all pairwise scores. The convolutional neural network method was evaluated against ground truth registrations determined by matching implanted fiducial markers visualized in a pretreatment orthogonal pair of x-ray images. The convolutional neural network method was further compared to manual registration and a standard commonly used intensity-based automatic registration approach based on advanced normalized correlation.Results:For 83 image pairs from 5 patients, convolutional neural network registration errors were smaller than 5 mm in 81% of the cases. In comparison, manual registration errors were smaller than 5 mm in 61% of the cases and advanced normalized correlation registration errors were smaller than 5 mm only in 25% of the cases.Conclusion:Convolutional neural network evaluation against manual registration and an advanced normalized correlation -based registration demonstrated better accuracy and reliability of the convolutional neural network. This suggests that with training on a large data set of transperineal ultrasound prostate images, the convolutional neural network method has potential for robust ultrasound-to-ultrasound registration.

Evaluation of Optical Localization in the Visible Band for Ultrasound Guidance in Radiotherapy Using A Robotic Arm

In many medical fields, spatial localization of devices is of paramount importance to e.g. deliver treatments correctly or to perform accurate diagnostic evaluations. Among the different technologies available, optical localization in the visible band of the spectrum offers many advantages: minimum interference with other instruments, flexibility in position of the devices used and extra features such as patient/operator surface tracking or vital signs monitoring. Its application in the domain of radiotherapy is novel, and potentially very beneficial. In this work we introduced a workflow to test the reliability of one of these systems, developed in house, to localize an ultrasound transducer for ultrasound guided radiation therapy (USgRT). Accuracy, precision, latency, sensitive volume and sensitivity to light intensity were evaluated. The workflow we proposed allowed us to establish that, while all the clinically relevant parameters of the system are acceptable, accuracy in locating the transducer could be as poor as 7-8 mm, therefore not acceptable for the intended purpose of the system.

In vivo reproducibility of robotic probe placement for a novel ultrasound-guided radiation therapy system.

Ultrasound can provide real-time image guidance of radiation therapy, but the probe-induced tissue deformations cause local deviations from the treatment plan. If placed during treatment planning, the probe causes streak artifacts in required computed tomography (CT) images. To overcome these challenges, we propose robot-assisted placement of an ultrasound probe, followed by replacement with a geometrically identical, CT-compatible model probe. In vivo reproducibility was investigated by implanting a canine prostate, liver, and pancreas with three 2.38-mm spherical markers in each organ. The real probe was placed to visualize the markers and subsequently replaced with the model probe. Each probe was automatically removed and returned to the same position or force. Under position control, the median three-dimensional reproducibility of marker positions was 0.6 to 0.7mm, 0.3 to 0.6mm, and 1.1 to 1.6mm in the prostate, liver, and pancreas, respectively. Reproducibility was worse under force control. Probe substitution errors were smallest for the prostate (0.2 to 0.6mm) and larger for the liver and pancreas (4.1 to 6.3mm), where force control generally produced larger errors than position control. Results indicate that position control is better than force control for this application, and the robotic approach has potential, particularly for relatively constrained organs and reproducibility errors that are smaller than established treatment margins.

SU‐E‐J‐114: Towards Integrated CT and Ultrasound Guided Radiation Therapy Using A Robotic Arm with Virtual Springs

Purpose:Currently there is an urgent need in Radiation Therapy for noninvasive and nonionizing soft tissue target guidance such as localization before treatment and continuous monitoring during treatment. Ultrasound is a portable, low cost option that can be easily integrated with the LINAC room. We are developing a cooperatively controlled robot arm that has high intrafraction reproducibility with repositioning of the ultrasound probe. In this study, we introduce virtual springs (VS) to assist with interfraction probe repositioning and we compare the soft tissue deformation introduced by VS to the deformation that would exist without them.Methods:Three metal markers were surgically implanted in the kidney of one dog. The dog was anesthetized and immobilized supine in an alpha cradle. The reference ultrasound probe position and force to ideally visualize the kidney was defined by an experienced ultrasonographer using the Clarity ultrasound system and robot sensor. For each interfraction study, the dog was removed from the cradle and re‐setup based on CBCT with bony anatomy alignment to mimic regular patient setup. The ultrasound probe was automatically returned to the reference position using the robot. To accommodate the soft tissue anatomy changes between each setup the operator used the VS feature to adjust the probe and obtain an ultrasound image that matched the reference image. CBCT images were acquired and each interfraction marker location was compared with the first interfractionResult. Results:Analysis of the marker positions revealed that the kidney was displaced by 18.8 ± 6.4 mm without VS and 19.9 ± 10.5 mm with VS. No statistically significant differences were found between two procedures.Conclusion:The VS feature is necessary to obtain matching ultrasound images, and they do not introduce further changes to the tissue deformation. Future work will focus on automatic VS based on ultrasound feedback.Supported in part by: NCI R01 CA161613; Elekta Sponsored Research

WE-G-BRF-09: Force- and Image-Adaptive Strategies for Robotised Placement of 4D Ultrasound Probes

Purpose: To allow continuous acquisition of high quality 4D ultrasound images for non-invasive live tracking of tumours for IGRT, image- and force-adaptive strategies for robotised placement of 4D ultrasound probes are developed and evaluated. Methods: The developed robotised ultrasound system is based on a 6-axes industrial robot (adept Viper s850) carrying a 4D ultrasound transducer with a mounted force-torque sensor. The force-adaptive placement strategies include probe position control using artificial potential fields and contact pressure regulation by a PD controller strategy. The basis for live target tracking is a continuous minimum contact pressure to ensure good image quality and high patient comfort. This contact pressure can be significantly disturbed by respiratory movements and has to be compensated. All measurements were performed on human subjects under realistic conditions. When performing cardiac ultrasound, rib- and lung shadows are a common source of interference and can disrupt the tracking. To ensure continuous tracking, these artefacts had to be detected to automatically realign the probe. The detection is realised by multiple algorithms based on entropy calculations as well as a determination of the image quality. Results: Through active contact pressure regulation it was possible to reduce the variance of the contact pressure by 89.79% despite respiratory motion of the chest. The results regarding the image processing clearly demonstrate the feasibility to detect image artefacts like rib shadows in real-time. Conclusion: In all cases, it was possible to stabilise the image quality by active contact pressure control and automatically detected image artefacts. This fact enables the possibility to compensate for such interferences by realigning the probe and thus continuously optimising the ultrasound images. This is a huge step towards fully automated transducer positioning and opens the possibility for stable target tracking in ultrasoundguided radiation therapy requiring contact pressure of 5–10 N. This work was supported by the Graduate School for Computing in Medicine and Life Sciences funded by Germany's Excellence Initiative [DFG GSC 235/1].