Improved Motion Tracking Research Articles

Using sensor devices in combination with computer vision algorithms to improve motion tracking systems

The article investigates the integration of sensor devices with computer vision algorithms to improve motion tracking systems. The article shows the feasibility of using different types of computer vision algorithms, each of which plays a special role in improving the performance of motion tracking systems. This makes it possible to create motion tracking systems that provide high accuracy, speed, and reliability. Moreover, such systems can work in different lighting conditions and with different types of objects. Computer vision algorithms can detect motion in video images regardless of lighting conditions, and motion sensors can help detect and track objects even in limited visual conditions. A comparative analysis of sensor devices and computer vision algorithms, their type of operation and feasibility of application was performed. We also developed our own face tracking algorithm written in the Python programming language.

Patient specific prior cross attention for kV decomposition in paraspinal motion tracking.

X-ray image quality is critical for accurate intrafraction motion tracking in radiation therapy. This study aims to develop a deep-learning algorithm to improve kV image contrast by decomposing the image into bony and soft tissue components. In particular, we designed a priori attention mechanism in the neural network framework for optimal decomposition. We show that a patient-specific prior cross-attention (PCAT) mechanism can boost the performance of kV image decomposition. We demonstrate its use in paraspinal SBRT motion tracking with online kV imaging. Online 2D kV projections were acquired during paraspinal SBRT for patient motion monitoring. The patient-specific prior images were generated by randomly shifting and rotating spine-only DRR created from the setup CBCT, simulating potential motions. The latent features of the prior images were incorporated into the PCAT using multi-head cross attention. The neural network aimed to learn to selectively amplify the transmission of the projection image features that correlate with features of the priori. The PCAT network structure consisted of (1) a dual-branch generator that separates the spine and soft tissue component of the kV projection image and (2) a dual-function discriminator (DFD) that provides the realness score of the predicted images. For supervision, we used a loss combining mean absolute error loss, discriminator loss, perceptual loss, total variation, and mean squared error loss for soft tissues. The proposed PCAT approach was benchmarked against previous work using the ResNet generative adversarial network (ResNetGAN) without prior information. The trained PCAT had improved performance in effectively retaining and preserving the spine structure and texture information while suppressing the soft tissues from the kV projection images. The decomposed spine-only x-ray images had the submillimeter matching accuracy at all beam angles. The decomposed spine-only x-ray significantly reduced the maximum errors to 0.44mm(<2 pixels) in comparison to 0.92mm (∼4 pixels) of ResNetGAN. The PCAT decomposed spine images also had higher PSNR and SSIM (p-value<0.001). The PCAT selectively learned the important latent features by incorporating the patient-specific prior knowledge into the deep learning algorithm, significantly improving the robustness of the kV projection image decomposition, and leading to improved motion tracking accuracy in paraspinal SBRT.

Developing control systems to improve motion tracking of electro-hydraulic systems subjected to external load

Developing control systems to improve motion tracking of electro-hydraulic systems subjected to external load

Fusion of video and inertial sensing data via dynamic optimization of a biomechanical model

Inertial sensing and computer vision are promising alternatives to traditional optical motion tracking, but until now these data sources have been explored either in isolation or fused via unconstrained optimization, which may not take full advantage of their complementary strengths. By adding physiological plausibility and dynamical robustness to a proposed solution, biomechanical modeling may enable better fusion than unconstrained optimization. To test this hypothesis, we fused video and inertial sensing data via dynamic optimization with a nine degree-of-freedom model and investigated when this approach outperforms video-only, inertial-sensing-only, and unconstrained-fusion methods. We used both experimental and synthetic data that mimicked different ranges of video and inertial measurement unit (IMU) data noise. Fusion with a dynamically constrained model significantly improved estimation of lower-extremity kinematics over the video-only approach and estimation of joint centers over the IMU-only approach. It consistently outperformed single-modality approaches across different noise profiles. When the quality of video data was high and that of inertial data was low, dynamically constrained fusion improved estimation of joint kinematics and joint centers over unconstrained fusion, while unconstrained fusion was advantageous in the opposite scenario. These findings indicate that complementary modalities and techniques can improve motion tracking by clinically meaningful margins and that data quality and computational complexity must be considered when selecting the most appropriate method for a particular application.

Co-attention spatial transformer network for unsupervised motion tracking and cardiac strain analysis in 3D echocardiography.

Myocardial ischemia/infarction causes wall-motion abnormalities in the left ventricle. Therefore, reliable motion estimation and strain analysis using 3D+time echocardiography for localization and characterization of myocardial injury is valuable for early detection and targeted interventions. Previous unsupervised cardiac motion tracking methods rely on heavily-weighted regularization functions to smooth out the noisy displacement fields in echocardiography. In this work, we present a Co-Attention Spatial Transformer Network (STN) for improved motion tracking and strain analysis in 3D echocardiography. Co-Attention STN aims to extract inter-frame dependent features between frames to improve the motion tracking in otherwise noisy 3D echocardiography images. We also propose a novel temporal constraint to further regularize the motion field to produce smooth and realistic cardiac displacement paths over time without prior assumptions on cardiac motion. Our experimental results on both synthetic and in vivo 3D echocardiography datasets demonstrate that our Co-Attention STN provides superior performance compared to existing methods. Strain analysis from Co-Attention STNs also correspond well with the matched SPECT perfusion maps, demonstrating the clinical utility for using 3D echocardiography for infarct localization.

A novel tissue mechanics-based method for improved motion tracking in quasi-static ultrasound elastography.

Most cancers are associated with biological and structural changes that lead to tissue stiffening. Therefore, imaging tissue stiffness using quasi-static ultrasound elastography (USE) can potentially be effective in cancer diagnosis. USE techniques developed for stiffness image reconstruction use noisy displacement data to obtain the stiffness images. In this study, we propose a technique to substantially improve the accuracy of the displacement data computed through ultrasound tissue motion tracking techniques, especially in the lateral direction. The proposed technique uses mathematical constraints derived from fundamental tissue mechanics principles to regularize displacement and strain fields obtained using Global Ultrasound Elastography (GLUE) and Second-Order Ultrasound Elastography (SOUL) methods. The principles include a novel technique to enforce (1) tissue incompressibility using 3D Boussinesq model and (2) deformation compatibility using the compatibility differential equation. The technique was validated thoroughly using metrics pertaining to Signal-to-Noise-Ratio (SNR), Contrast-to-Noise-Ratio (CNR) and Normalized Cross Correlation (NCC) for four tissue-mimicking phantom models and two clinical breast ultrasound elastography cases. The results show substantial improvement in the displacement and strain images generated using the proposed technique. The tissue-mimicking phantom study results indicate that the proposed method is superior in improving image quality compared to the GLUE and SOUL techniques as it shows an average axial strain SNR and CNR improvement of 44% and 63%, and lateral strain SNR and CNR improvement of 130% and 435%, respectively. The results of the phantom study also indicate higher accuracy of displacement images obtained using the proposed technique, including improvement ranges of 7-84% and 26-140% for axial and lateral displacement images, respectively. For the clinical cases, the results indicate average improvement of 48% and 64% in SNR and CNR, respectively, in the axial strain images, and average improvement of 40% and 41% in SNR and CNR, respectively, in the lateral strain images. The proposed method is very effective in producing improved estimate of tissue displacement and strain images, especially with the lateral displacement and strain where the improvement is highly remarkable. While the method shows promise for clinical applications, further investigation is necessary for rigorous assessment of the method's performance in the clinic.

Cardiac Strain Imaging Using Multi-Perspective Ultrafast Ultrasound

The heart is a complex organ with a high level of deformation occurring in different directions. Ultrasound imaging has proven to be a valuable tool to quantify these deformations. However, strain is not always measured accurately in vital parts of the heart when only using a single probe, even at a high frame rate, because of the anisotropic spatial resolution and contrast. Therefore, a multi-perspective ultrafast ultrasound (US) strain imaging method was developed, aiming at investigating improvements in strain while operating two probes at different relative angles. In an ex-vivo experiment of a beating porcine heart, parasternal short axis views of the left ventricle (LV) were acquired by two phased array probes with different relative angles (30°–75°) at a frame rate of 170 frames per second (FPS). A fully automatic registration algorithm was developed to register the image datasets for all cases. Next, radio frequency (RF) based strain imaging was performed. Axial displacements were compounded based on the unit axial vectors of the dual probes to improve motion tracking and strain estimation. After performing multi-perspective strain imaging, compounded radial and circumferential strain both improve compared to single probe strain imaging. While increasing the inter-probe angle from 30° to 75°, the mean tracking error (ME), mean drift error (MDE) and strain variability (SV) decreased, and signal-to-noise ratio (SNRe) increased for both strain components. For the largest angle (75°), large reductions in ME (−42%), MDE (−50%) and SV (−48%) were observed. The SNRe increased by 253 and 39% for radial and circumferential strain, respectively, and strain curves revealed less noise for each region. In summary, a multi-perspective ultrafast US strain imaging method was introduced to improve cardiac strain estimation in an ex-vivo beating porcine heart setup.

Decompose kV projection using neural network for improved motion tracking in paraspinal SBRT.

On-treatment kV images have been used in tracking patient motion. One challenge of markerless motion tracking in paraspinal SBRT is the reduced contrast when the X-ray beam needs to pass through a large portion of the patient's body, for example, from the lateral direction. Besides, due to the spine's overlapping with the surrounding moving organs in the X-ray images, auto-registration could lead to potential errors. This work aims to automatically extract the spine component from the conventional 2D X-ray images, to achieve more robust and more accurate motion management. A ResNet generative adversarial network (ResNetGAN) consisting of one generator and one discriminator was developed to learn the mapping between 2D kV image and the reference spine digitally reconstructed radiograph (DRR). A tailored multi-channel multi-domain loss function was used to improve the quality of the decomposed spine image. The trained model took a 2D kV image as input and learned to generate the spine component of the X-ray image. The training dataset included 1347 2D kV thoracic and lumbar region X-ray images from 20 randomly selected patients, and the corresponding matched reference spine DRR. Another 226 2D kV images from the remaining four patients were used for evaluation. The resulted decomposed spine images and the original X-ray images were registered to the reference spine DRRs, to compare the spine tracking accuracy. The decomposed spine image had the mean peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) of 60.08 and 0.99, respectively, indicating the model retained and enhanced the spine structure information in the original 2D X-ray image. The decomposed spine image matching with the reference spine DRR had submillimeter accuracy (in mm) with a mean error of 0.13, 0.12, and a maximum of 0.58, 0.49 in the - and -directions (in the imager coordinates), respectively. The accuracy improvement is robust in all lateral and anteroposterior X-ray beam angles. We developed a deep learning-based approach to remove soft tissues in the kV image, leading to more accurate spine tracking in paraspinal SBRT.

Multiperspective Ultrasound Strain Imaging of the Abdominal Aorta.

Current decision-making for clinical intervention of abdominal aortic aneurysms (AAAs) is based on the maximum diameter of the aortic wall, but this does not provide patient-specific information on rupture risk. Ultrasound (US) imaging can assess both geometry and deformation of the aortic wall. However, low lateral contrast and resolution are currently limiting the precision of both geometry and local strain estimates. To tackle these drawbacks, a multiperspective scanning mode was developed on a dual transducer US system to perform strain imaging at high frame rates. Experimental imaging was performed on porcine aortas embedded in a phantom of the abdomen, pressurized in a mock circulation loop. US images were acquired with three acquisition schemes: Multiperspective ultrafast imaging, single perspective ultrafast imaging, and conventional line-by-line scanning. Image registration was performed by automatic detection of the transducer surfaces. Multiperspective images and axial displacements were compounded for improved segmentation and tracking of the aortic wall, respectively. Performance was compared in terms of image quality, motion tracking, and strain estimation. Multiperspective compound displacement estimation reduced the mean motion tracking error over one cardiac cycle by a factor 10 compared to conventional scanning. Resolution increased in radial and circumferential strain images, and circumferential signal-to-noise ratio (SNRe) increased by 10 dB. Radial SNRe is high in wall regions moving towards the transducer. In other regions, radial strain estimates remain cumbersome for the frequency used. In conclusion, multiperspective US imaging was demonstrated to improve motion tracking and circumferential strain estimation of porcine aortas in an experimental set-up.

Improved respiratory self-navigation for 3D radial acquisitions through the use of a pencil-beam 2D-T2 -prep for free-breathing, whole-heart coronary MRA.

In respiratory self-navigation (SN), signal from static structures, such as the chest wall, may complicate motion detection or introduce post-correction artefacts. Suppressing signal from superfluous tissues may therefore improve image quality. We thus test the hypothesis that SN whole-heart coronary magnetic resonance angiography (MRA) will benefit from an outer-volume suppressing 2D-T2 -Prep and present both phantom and in vivo results. A 2D-T2 -Prep and a conventional T2 -Prep were used prior to a free-breathing 3D-radial SN sequence. Both techniques were compared by imaging a home-built moving cardiac phantom and by performing coronary MRA in nine healthy volunteers. Reconstructions were performed using both a reference-based and a reference-independent approach to motion tracking, along with several coil combinations. Signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) were compared, along with vessel sharpness (VS). In phantoms, using the 2D-T2 -Prep increased SNR by 16% to 53% and mean VS by 8%; improved motion tracking precision was also achieved. In volunteers, SNR increased by an average of 29% to 33% in the blood pool and by 15% to 25% in the myocardium, depending on the choice of reconstruction coils and algorithm, and VS increased by 34%. A 2D-T2 -Prep significantly improves image quality in both phantoms and volunteers when performing SN coronary MRA. Magn Reson Med 79:1293-1303, 2018. © 2017 International Society for Magnetic Resonance in Medicine.

Designing an optimized model-free controller for improved motion tracking by rugged electrohydraulic system

Electrohydraulic actuation systems with proportional valves and industry-grade cylinders have widespread use due to their low cost and ease of maintenance. Achieving good tracking of output position of the piston in such a system by a proper controller design is the objective of this study. Due to various parameter uncertainties and complex nonlinearities like piston friction, valve deadband and flow characteristics, the variation of oil property with working temperature and air entrapment volume, the implementation of a model-based control becomes quite difficult for such systems. A notable contribution of this work is to couple a model-free fuzzy controller with a feedforward controller with all the parameters estimated by a real-coded genetic algorithm. Also, the optimizer has been used for identifying an appropriate fuzzy structure that is easy to implement in real time. A comprehensive experimental study has revealed the best fuzzy structure to be achieved through a Gaussian fuzzification of linear combination of the position and velocity errors as the input and singleton membership function for the output voltage actuating the proportional valve. Excellent responses with good disturbance rejection capability and energy efficiency have been achieved. These responses have emerged superior to those achieved in a similar system with different controller structures and in systems with much costlier components.

SU‐E‐J‐142: Performance Study of Automatic Image‐Segmentation Algorithms in Motion Tracking Via MR‐IGRT

Purpose:Evaluate commonly used segmentation algorithms on a commercially available real‐time MR image guided radiotherapy (MR‐IGRT) system (ViewRay), compare the strengths and weaknesses of each method, with the purpose of improving motion tracking for more accurate radiotherapy.Methods:MR motion images of bladder, kidney, duodenum, and liver tumor were acquired for three patients using a commercial on‐board MR imaging system and an imaging protocol used during MR‐IGRT. A series of 40 frames were selected for each case to cover at least 3 respiratory cycles. Thresholding, Canny edge detection, fuzzy k‐means (FKM), k‐harmonic means (KHM), and reaction‐diffusion level set evolution (RD‐LSE), along with the ViewRay treatment planning and delivery system (TPDS) were included in the comparisons. To evaluate the segmentation results, an expert manual contouring of the organs or tumor from a physician was used as a ground‐truth. Metrics value of sensitivity, specificity, Jaccard similarity, and Dice coefficient were computed for comparison.Results:In the segmentation of single image frame, all methods successfully segmented the bladder and kidney, but only FKM, KHM and TPDS were able to segment the liver tumor and the duodenum. For segmenting motion image series, the TPDS method had the highest sensitivity, Jarccard, and Dice coefficients in segmenting bladder and kidney, while FKM and KHM had a slightly higher specificity. A similar pattern was observed when segmenting the liver tumor and the duodenum. The Canny method is not suitable for consistently segmenting motion frames in an automated process, while thresholding and RD‐LSE cannot consistently segment a liver tumor and the duodenum.Conclusion:The study compared six different segmentation methods and showed the effectiveness of the ViewRay TPDS algorithm in segmenting motion images during MR‐IGRT. Future studies include a selection of conformal segmentation methods based on image/organ‐specific information, different filtering methods and their influences on the segmentation results.Parag Parikh receives research grant from ViewRay.Sasa Mutic has consulting and research agreements with ViewRay.Yanle Hu receives travel reimbursement from ViewRay.Iwan Kawrakow and James Dempsey are ViewRay employees.

Fuzzy logic versus a PID controller for position control of a muscle-like actuated arm

This study compares three different control algorithms for a muscle-like actuated arm developed to replicate motion in two degrees-of-freedom (df): elbow flexion/extension (f/e) and forearm pronation/supination (p/s). Electromyogram (EMG) is employed to help determine the control signal used to actuate the muscle cylinders. Three different types of control strategies were attempted. The first algorithm used fuzzy logic with EMG signals and position error as control inputs (Fuzzy Controller). The second algorithm incorporated moment arm information into the existing fuzzy logic controller (Fuzzy-MA Controller). The third algorithm was a conventional Proportional-Integral-Derivative (PID) controller, which operated solely on position and integration error (PID Controller). Overall, moment arm scaling aided the fuzzy logic control algorithm by improving movement accuracy as determined by relative error and correlation. The PID controller resulted in the most accurate movement tracking after fine tuning the control gains. This study implies that moment arm scaling is an effective tool for improving motion tracking accuracy of the fuzzy controller in the mechanical arm. The study also implies that PID controller can be used as a substitute for the fuzzy based controller once the desired motion is prescribed.

Improving motion tracking of servomotor-driven closed-loop mechanisms using mass-redistribution

This paper proposes a new design methodology for the improvement of motion tracking performance of single-freedom closed-loop mechanisms (actuated by controlled servomotors) under the operation conditions of robot manipulators. In particular, this design methodology suggests a mass redistribution scheme (prior to designing controllers), which follows the principle of a full shaking force balancing scheme, for the modification of an existing mechanism, with the aim to obtain a “simple” system dynamic model and thus to facilitate controller design. By taking a four-bar mechanism as example, the paper will show that motion-tracking performance of the mechanism is significantly improved by simply applying a conventional proportional-plus-derivative control algorithm. Simulation studies on cases of velocity tracking and trajectory tracking verify the effectiveness of the proposed methodology.