Motion Tracking Techniques Research Articles

Simultaneous recording of lateral excursions using motion tracking technique

Simultaneous recording of lateral excursions using motion tracking technique

Human-exoskeleton cooperation for reducing the musculoskeletal load of manual handling tasks in orchid farms

Work-related musculoskeletal diseases of the lower back and shoulders are frequently caused by lifting, carrying and trunk bending. Therefore, a hybrid exoskeleton (HExo) that provides external torque to the upper extremities and back may reduce discomfort in the lower back and shoulder tendinopathies. In this study, we designed and fabricated HExo, a novel upper-limb and lower-back combined exoskeleton, and evaluated its impact in reducing the musculoskeletal load on the shoulder and back during agricultural work in orchid farms. The human motion tracking technique was utilised to collect the joint angles of orchid farmers during their work, and the joint angle data were subsequently used for the design of HExo. Electromyography (EMG) and centre of pressure (CoP) measurements were taken for 5 participants performing farm tasks and 16 participants performing laboratory-standardised tasks both with and without the HExo. The amplitude probability density function and Wilcoxon signed rank test were utilised in the data analysis to verify the advantage in wearing HExo. The use of HExo resulted in a reduction in erector spinae, latissimus dorsi and anterior deltoid muscle activation of up to 36.92 % (p = 0.05), 31.43 % (p = 0.017), and 57.59 % (p = 0.012), respectively. Furthermore, its use did not affect the ability of participants to maintain balance. Hence, HExo could be a promising for reducing solution musculoskeletal load.

Adaptive zone based active noise control for a moving target

This study focuses on enhancing active noise control (ANC) in room enclosures, specifically targeting zones of quiet (ZoQ) around aircraft passenger seats. By integrating virtual sensing and motion tracking techniques, we aim to dynamically adapt the ZoQ to moving targets. Key to our approach is the strategic placement of actuators and sensors, forming the core of the ANC system. Our methodology includes virtual sensing for ANC analysis, ZoQ optimization for varied applications, and in-depth case studies. We introduce an innovative combination of a speaker gimbal system, a vision system, and custom software for precise motion tracking, significantly improving ZoQ localization. The findings offer insights into maintaining effective ZoQs for multiple input multiple output (MIMO) local ANC configurations, laying the groundwork for adaptive ZoQ control about sound sources and desired cancellation locations. This research marks a significant step towards more effective and adaptable noise cancellation in enclosed spaces.

On the soft tissue ultrasound elastography using FEM based inversion approach.

Elastography is a medical imaging modality that enables visualization of tissue stiffness. It involves quasi-static or harmonic mechanical stimulation of the tissue to generate a displacement field which is used as input in an inversion algorithm to reconstruct tissue elastic modulus. This paper considers quasi-static stimulation and presents a novel inversion technique for elastic modulus reconstruction. The technique follows an inverse finite element framework. Reconstructed elastic modulus maps produced in this technique do not depend on the initial guess, while it is computationally less involved than iterative reconstruction approaches. The method was first evaluated using simulated data (in-silico) where modulus reconstruction's sensitivity to displacement noise and elastic modulus was assessed. To demonstrate the method's performance, displacement fields of two tissue mimicking phantoms determined using three different motion tracking techniques were used as input to the developed elastography method to reconstruct the distribution of relative elastic modulus of the inclusion to background tissue. In the next stage, the relative elastic modulus of three clinical cases pertaining to liver cancer patient were determined. The obtained results demonstrate reasonably high elastic modulus reconstruction accuracy in comparison with similar direct methods. Also it is associated with reduced computational cost in comparison with iterative techniques, which suffer from convergence and uniqueness issues, following the same formulation concept. Moreover, in comparison with other methods which need initial guess, the presented method does not require initial guess while it is easy to understand and implement.

Complete mouth rehabilitation, using jaw motion tracking and double scan technique in a patient with osteoarthrosis: a case report

Complete mouth rehabilitation, using jaw motion tracking and double scan technique in a patient with osteoarthrosis: a case report

Comparative Study of Ultrasound Tissue Motion Tracking Techniques for Effective Breast Ultrasound Elastography

Breast cancer is the most common and deadly cancer in women, where early detection is of the utmost importance as survival rates decrease with the advancement of the disease. Most available methods of breast cancer screening and evaluation lack the ability to effectively differentiate between benign and malignant lesions without a biopsy. Ultrasound elastography (USE) is a cost-effective method that can potentially provide an initial malignancy assessment at the bedside. One of the challenges, however, is the uncertainty of tissue displacement data when performing USE due to out-of-plane movement of the tissue during mechanical stimulation, in addition to the computational efficiency necessary for real-time image reconstruction. This work presents a comparison of four different theoretically sound displacement estimators for their ability in tissue Young’s modulus reconstruction level with an emphasis on quality-to-runtime ratio to determine which estimators are most suitable for real-time USE systems. The methods are known in literature as AM2D, GLUE, OVERWIND, and SOUL methods. The effectiveness of each method was assessed as a stand-alone method or in combination with a strain field enhancement technique known as STREAL, which was recently developed using tissue mechanics-based regularization. The study was performed using radiofrequency US data pertaining to in silico and tissue mimicking phantoms in addition to clinical data. This data was used to generate tissue displacement fields employed to generate axial and lateral strain images before Young’s modulus images were reconstructed. The study indicates that the AM2D displacement estimator, which is an older and computationally less involved method, along with a tissue-mechanics-based image processing algorithm, performs very well, with high CNR, SNR, and preservation of tumor heterogeneity obtained at both strain and stiffness image levels, while its computation run-time is much lower compared to other estimation methods. As such, it can be recommended for incorporation in real-time USE systems.

Neural signals-based respiratory motion tracking: a proof-of-concept study

Objective. Respiratory motion tracking techniques can provide optimal treatment accuracy for thoracoabdominal radiotherapy and robotic surgery. However, conventional imaging-based respiratory motion tracking techniques are time-lagged owing to the system latency of medical linear accelerators and surgical robots. This study aims to investigate the precursor time of respiratory-related neural signals and analyze the potential of neural signals-based respiratory motion tracking. Approach. The neural signals and respiratory motion from eighteen healthy volunteers were acquired simultaneously using a 256-channel scalp electroencephalography (EEG) system. The neural signals were preprocessed using the MNE python package to extract respiratory-related EEG neural signals. Cross-correlation analysis was performed to assess the precursor time and cross-correlation coefficient between respiratory-related EEG neural signals and respiratory motion. Main results. Respiratory-related neural signals that precede the emergence of respiratory motion are detectable via non-invasive EEG. On average, the precursor time of respiratory-related EEG neural signals was 0.68 s. The representative cross-correlation coefficients between EEG neural signals and respiratory motion of the eighteen healthy subjects varied from 0.22 to 0.87. Significance. Our findings suggest that neural signals have the potential to compensate for the system latency of medical linear accelerators and surgical robots. This indicates that neural signals-based respiratory motion tracking is a potential promising solution to respiratory motion and could be useful in thoracoabdominal radiotherapy and robotic surgery.

Robust retrospective motion correction of head motion using navigator-based and markerless motion tracking techniques.

This study investigated the artifacts arising from different types of head motion in brain MR images and how well these artifacts can be compensated using retrospective correction based on two different motion-tracking techniques. MPRAGE images were acquired using a 3 T MR scanner on a cohort of nine healthy participants. Subjects moved their head to generate circular motion (4 or 6 cycles/min), stepwise motion (small and large) and "simulated realistic" motion (nodding and slow diagonal motion), based on visual instructions. One MPRAGE scan without deliberate motion was always acquired as a "no motion" reference. Three dimensional fat-navigator (FatNavs) and a Tracoline markerless device (TracInnovations) were used to obtain motion estimates and images were separately reconstructed retrospectively from the raw data based on these different motion estimates. Image quality was recovered from both motion tracking techniques in our stepwise and slow diagonal motion scenarios in almost all cases, with the apparent visual image quality comparable to the no-motion case. FatNav-based motion correction was further improved in the case of stepwise motion using a skull masking procedure to exclude non-rigid motion of the neck from the co-registration step. In the case of circular motion, both methods struggled to correct for all motion artifacts. High image quality could be recovered in cases of stepwise and slow diagonal motion using both motion estimation techniques. The circular motion scenario led to more severe image artifacts that could not be fully compensated by the retrospective motion correction techniques used.

Markerless motion tracking with simultaneous MV and kV imaging in spine SBRT treatment—a feasibility study

Objective. Motion tracking with simultaneous MV-kV imaging has distinct advantages over single kV systems. This research is a feasibility study of utilizing this technique for spine stereotactic body radiotherapy (SBRT) through phantom and patient studies. Approach. A clinical spine SBRT plan was developed using 6xFFF beams and nine sliding-window IMRT fields. The plan was delivered to a chest phantom on a linear accelerator. Simultaneous MV-kV image pairs were acquired during beam delivery. KV images were triggered at predefined intervals, and synthetic MV images showing enlarged MLC apertures were created by combining multiple raw MV frames with corrections for scattering and intensity variation. Digitally reconstructed radiograph (DRR) templates were generated using high-resolution CBCT reconstructions (isotropic voxel size (0.243 mm)3) as the reference for 2D–2D matching. 3D shifts were calculated from triangulation of kV-to-DRR and MV-to-DRR registrations. To evaluate tracking accuracy, detected shifts were compared to known phantom shifts as introduced before treatment. The patient study included a T-spine patient and an L-spine patient. Patient datasets were retrospectively analyzed to demonstrate the performance in clinical settings. Main results. The treatment plan was delivered to the phantom in five scenarios: no shift, 2 mm shift in one of the longitudinal, lateral and vertical directions, and 2 mm shift in all the three directions. The calculated 3D shifts agreed well with the actual couch shifts, and overall, the uncertainty of 3D detection is estimated to be 0.3 mm. The patient study revealed that with clinical patient image quality, the calculated 3D motion agreed with the post-treatment cone beam CT. It is feasible to automate both kV-to-DRR and MV-to-DRR registrations using a mutual information-based method, and the difference from manual registration is generally less than 0.3 mm. Significance. The MV-kV imaging-based markerless motion tracking technique was validated through a feasibility study. It is a step forward toward effective motion tracking and accurate delivery for spinal SBRT.

The assessment of injury risk in the healthcare sector via integrating motion tracking techniques with digital human modelling ergonomic tools

The assessment of injury risk in the healthcare sector via integrating motion tracking techniques with digital human modelling ergonomic tools

The assessment of injury risk in the healthcare sector via integrating motion tracking techniques with digital human modelling ergonomic tools

The assessment of injury risk in the healthcare sector via integrating motion tracking techniques with digital human modelling ergonomic tools

A novel tissue mechanics-based method for improved motion tracking in quasi-static ultrasound elastography.

Most cancers are associated with biological and structural changes that lead to tissue stiffening. Therefore, imaging tissue stiffness using quasi-static ultrasound elastography (USE) can potentially be effective in cancer diagnosis. USE techniques developed for stiffness image reconstruction use noisy displacement data to obtain the stiffness images. In this study, we propose a technique to substantially improve the accuracy of the displacement data computed through ultrasound tissue motion tracking techniques, especially in the lateral direction. The proposed technique uses mathematical constraints derived from fundamental tissue mechanics principles to regularize displacement and strain fields obtained using Global Ultrasound Elastography (GLUE) and Second-Order Ultrasound Elastography (SOUL) methods. The principles include a novel technique to enforce (1) tissue incompressibility using 3D Boussinesq model and (2) deformation compatibility using the compatibility differential equation. The technique was validated thoroughly using metrics pertaining to Signal-to-Noise-Ratio (SNR), Contrast-to-Noise-Ratio (CNR) and Normalized Cross Correlation (NCC) for four tissue-mimicking phantom models and two clinical breast ultrasound elastography cases. The results show substantial improvement in the displacement and strain images generated using the proposed technique. The tissue-mimicking phantom study results indicate that the proposed method is superior in improving image quality compared to the GLUE and SOUL techniques as it shows an average axial strain SNR and CNR improvement of 44% and 63%, and lateral strain SNR and CNR improvement of 130% and 435%, respectively. The results of the phantom study also indicate higher accuracy of displacement images obtained using the proposed technique, including improvement ranges of 7-84% and 26-140% for axial and lateral displacement images, respectively. For the clinical cases, the results indicate average improvement of 48% and 64% in SNR and CNR, respectively, in the axial strain images, and average improvement of 40% and 41% in SNR and CNR, respectively, in the lateral strain images. The proposed method is very effective in producing improved estimate of tissue displacement and strain images, especially with the lateral displacement and strain where the improvement is highly remarkable. While the method shows promise for clinical applications, further investigation is necessary for rigorous assessment of the method's performance in the clinic.

Moving beyond response times with accessible measures of manual dynamics

Button-press measures of response time (RT) and accuracy have long served a central role in psychological research. However, RT and accuracy provide limited insight into how cognitive processes unfold over time. To address this limitation, researchers have used hand-tracking techniques to investigate how cognitive processes unfold over the course of a response, are modulated by recent experience, and function across the lifespan. Despite the efficacy of these techniques for investigating a wide range of psychological phenomena, widespread adoption of hand-tracking techniques within the field is hindered by a range of factors, including equipment costs and the use of specialized software. Here, we demonstrate that the behavioral dynamics previously observed with specialized motion-tracking equipment in an Eriksen flanker task can be captured with an affordable, portable, and easy-to-assemble response box. Six-to-eight-year-olds and adults (N = 90) completed a computerized version of the flanker task by pressing and holding a central button until a stimulus array appeared. Participants then responded by releasing the central button and reaching to press one of two response buttons. This method allowed RT to be separated into initiation time (when the central button was released) and movement time (time elapsed between initiation and completion of the response). Consistent with previous research using motion-tracking techniques, initiation times and movement times revealed distinct patterns of effects across trials and between age groups, indicating that the method used in the current study presents a simple solution for researchers from across the psychological and brain sciences looking to move beyond RTs.

Abstract 9916: Prognostic Value of Simultaneous Analysis With Myocardial Flow Reserve and Right Ventricular Strain by Hybrid 13 N-ammonia Positron Emission Tomography/Magnetic Resonance Imaging in Coronary Artery Disease

Introduction: Myocardial flow reserve (MFR) derived from 13 N-ammonia positron emission tomography (PET) is used to predict adverse cardiac events in the patients with coronary artery disease (CAD). Right ventricular (RV) strain measured by magnetic resonance imaging (MRI) is used to evaluate RV function. This study aimed to evaluate the prognostic value of combined MFR and RV strain measured by hybrid 13 N-ammonia PET/MRI in patients with CAD. Methods: Sixty-one patients who underwent 13 N-ammonia PET/MRI were enrolled. MFR was calculated from dynamic acquisition of 13 N-ammonia PET under vasodilator stress with intravenous injection of adenosine. RV global longitudinal strain (GLS) was measured by wall motion tracking techniques in cine-mode MRI. The end points were defined as a composite of all-cause death, myocardial infarction, sustained ventricular arrhythmia, hospitalization due to decompensated heart failure, and revascularization. Results: At a follow-up of 2.8 ± 1.9 years, 21 events occurred. Kaplan-Meier analysis showed that the event-free rate was significantly lower in the group with MFR < 1.80 than that with MFR ≥ 1.80 (P < 0.001, Figure a). Additionally, the event-free rate was significantly lower in the group with RVGLS > –18.22% than that with RVGLS ≤ –18.22% (P = 0.025, Figure b). After dividing the patients into four groups by the median MFR and the median RVGLS, the event-free rate was lowest in the combined group of MFR < 1.80 and RVGLS > -18.22% than any other groups (P < 0.001, Figure c). In the Cox proportional hazard analysis, MFR and RVGLS were independent predictors of cardiac adverse events in the patients with CAD. Conclusion: The simultaneous assessment of MFR and RV strain by 13 N-ammonia PET/MRI revealed the feasibility of precise risk stratification for cardiac events in patients with CAD.

Left-right asymmetry and attractor-like dynamics of dog’s tail wagging during dog-human interactions

SummaryTail wagging plays an important role in social interactions, e.g., dogs show asymmetrical tail wagging in response to different social stimuli. However, the effects of social cues on tail wagging and the intrinsic organization of wagging behavior remain largely unknown. Here, we developed a platform using a deep-learning-based motion-tracking technique to extract and analyze the movement trajectory of a dog’s tail tip during dog-human interactions. Individual dogs exhibited unique and stable wagging characteristics. We further found that tail wagging developed asymmetry toward the right side over three days of dog-human interactions, suggesting that it is a time-sensitive indicator of social familiarity. In addition, wagging appeared to follow an attractor-like dynamic process consisting of stable states and unstable, transitional states. Together, these results revealed sophisticated characteristics and organization of a dog’s tail-wagging behavior during interactions with humans, providing a useful paradigm for studying dogs’ social behaviors and the underlying neural mechanisms.

A study on diameter-dependent support selection of the tendrils of Cayratia japonica

Organisms make decisions when they perceive cues of varying intensities. In case of climbing plants, the diameter of supports in contact (tree or stem) is an important cue for their growth as plants that coil around a support with large diameter are unable to maintain tensional forces required for continued attachment to the support. The negative association between the diameter and the climbing success has been reported since Darwin published his study on climbing plants. However, it is not known if a climbing plant makes a decision to avoid a support with larger diameter. Here, we tested this possibility by observing the coiling response of tendrils of Cayratia japonica to supports with different diameters. The coiling success of the tendrils was affected by the diameter of the support and the tendril lengths. We propose a decision tree to describe the different phases of the coiling response and demonstrated that the tendrils change their coiling shape depending on the support diameter and the tendril length. To understand the behavioural rules regulating the phase pattern, we constructed a simple model with two assumptions on the tendril movement, (1) when the tendrils receive a contact stimulus, they begin to coil from around the contact point and (2) there is a minimum coiling angle at which the tendrils coil up, once the tendril starts coiling. Image analysis and 3D motion tracking technique revealed that the movement of the tendrils were consistent with the two assumptions of the model. The results suggested that the tendrils flexibly changed the coiling shapes depending on the support diameter and simple behavioural rules could regulate this diameter-dependent response.

Fusion Siamese network with drift correction for target tracking in ultrasound sequences

Motion tracking techniques can revise the bias arising from respiration-caused motion in radiation therapy. Tracking key structures accurately and at a real-time speed is necessary for effective motion tracking. In this work, we propose a fusion Siamese network with drift correction for target tracking in ultrasound sequences. Specifically, the network fuses four response maps generated by the cross-correlation between convolution layers at different resolutions to reduce up-sampling error. A correction strategy combining local structural similarity and target trajectory is proposed to revise the target drift predicted by the network. Moreover, a coarse-to-fine strategy is proposed to train the network with a limited number of annotated images, in which an augmented dataset is generated by corner points to learn network features with high generalizability. The proposed method is evaluated on the basis of the public dataset of the MICCAI 2015 Challenge on Liver UltraSound Tracking (CLUST) and our ultrasound image dataset, which is provided by the Chinese People’s Liberation Army General Hospital (CPLAGH). A tracking error of 0.80 ± 1.16 mm is observed for 85 targets across 39 ultrasound sequences in the CLUST dataset. A tracking error of 0.61 ± 0.36 mm is observed for 20 targets across 10 ultrasound sequences in the CPLAGH dataset. The effectiveness of the proposed fusion and correction strategies is verified via two ablation experiments. Overall, the experimental results demonstrate the effectiveness of the proposed fusion Siamese network with drift correction and reveal its potential in clinical practice.

Real-Time Motion Tracking and Correction During Tomotherapy for Prostate Cancer: Impact of Fiducial Migration

A real-time motion tracking technique has been recently implemented in our clinic to manage intrafraction prostate motion (IPM) during helical tomotherapy for prostate cancer. It involves detecting motion of multiple fiducials implanted in prostate based on real-time x-ray radiographs acquired at various beam angles and correcting the motion using dynamic jaw and MLC. The purpose of this study was to study interfractional fiducial migration (IFM) and its impact on intrafraction motion tracking.Daily MVCT and radiographs acquired during tomotherapy with real-time motion tracking for 5 prostate cancer patients were analyzed to quantify IFM and IPM. All patients were treated with either 70 Gy in 28 or 75.6 Gy in 42 fractions based on plans generated on planning CTs acquired 14-70 days after the implantation of 4 fiducials for each patient. Time intervals between the first treatment and the planning CT acquisition ranged 4-16 days. The IPM was measured based on the maximum target position difference between the radiographs. Daily migration of each fiducial was obtained by (1) rigidly registering the daily MVCT with the planning CT based on prostate matching, and (2) calculating coordinate differences between the fiducial positions on the daily and plan images. To investigate the impact of fiducial migration on motion tracking that uses fiducial positions on the planning CT as baseline, we compared the maximum fiducial pair distance change (RigidBody, a tracking parameter) calculated from the planning CT to radiographs (default) to the value calculated from daily MVCT to radiographs. The daily target offsets (another tracking parameter) determined between the planning CT and radiographs were assessed if the fiducial migrations were removed.The average daily IPM was 3.2 ± 2.9 (up to 5.8 ± 4.0 for a patient) mm. The average IFM was 2.0 ± 1.1 mm (ranging 1.5-2.5 mm among the patients) with median and interquartile range (IQR) values of 1.8 and 1.2 mm, respectively. The fiducials migrated differently in the longitudinal (std/IQR = 1.8/2.2 mm), vertical (1.2/1.6 mm), and lateral (0.7/0.8 mm) directions. The daily fiducial COM deviated from its plan position by 1.6 ± 0.8 mm (ranging 1.3-2.0 mm among the patients) with median/IQR = 1.5/1.0 mm. The overall average of RigidBody reduced from 2.6 ± 0.7 mm (default) to 1.3 ± 0.4 mm if from daily MVCT. The average daily target offsets decreased from 3.2 ± 1.5 to 2.2 ± 1.0 mm if the fiducial migrations were removed.The fiducial-based real-time prostate motion tracking during tomotherapy is advantageous, even in the presence of the measurable IFMs. Our results indicate that, to reduce/eliminate the effect of the fiducial migration on motion tracking stability and accuracy, the fiducial positions on the daily MVCT, instead of the planning CT in the current setting, should be used as the baseline.G. Chen: North Central Chapter of AAPM, Medical College of Wisconsin. A. Tai: None. E.A. Omari: None. M. Bedi: None. C.A. Lawton: None. A. Li: Research Grant; Elekta AB, Accuray Inc, Siemens Healthineers, Manteia Med. Honoraria; Elekta AB, Accuray Inc. Patent/License Fees/Copyright; Manteia Med.

Real-Time Motion Tracking Based on Orthogonal Cine MRI During MR-Guided Radiation Therapy for Prostate Cancer

In contrast to radiographic imaging with implanted radio-opaque fiducials, the high soft tissue contrast and noninvasive nature of MRI are advantageous for real-time motion tracking of prostate motion. The purpose of this study is to investigate the feasibility of tracking the intrafraction prostate motion using an intensity-based template registration technique between real-time 2D cine and 3D MRI acquired during MR-guided Adaptative Radiation Therapy (MRgART).The motion tracking algorithm includes two steps: (1) creating a template by rigidly registering a set of cine images with a 3D reference MRI, and (2) tracking the target motion by registering incoming (i.e., real-time) cine with the template with a known relationship to the 3D volume. The daily 3D T2 MRI and 2D T2/T1 bFFE real-time cine MRI acquired on a 1.5T MR-Linac during MRgART for 6 prostate cancer patients were selected to test the algorithm limits. The cine images were acquired with a temporal resolution of 0.2 s for a total duration of 9-11 minutes interleaved between coronal and sagittal orientations. A 2D template image set was generated in the coronal and sagittal planes using the cine data acquired during the first 20 s registered to the 3D volume using in-plane rigid-body image registration based on a region of interest defined by the prostate contour on 3D T2 MRI. Motion tracking of the prostate was performed by registering the rest of the 2D cine series with the corresponding template. Manually delineated contours on sampled cine frames were used as the ground truth. Tracking accuracy was defined as the standard deviation of the error (SDE) between the center of mass (CM) of the ground-truth and tracked motions in 1 min. Prostate drift was evaluated by tracking the CM shift with the algorithm every 8-9 s for 9-11 min.The prostate tracking algorithm was qualitatively validated by visual assessment of 6 prostate cases. Overall, the tracking was found robust to implant-induced image artefacts in two cases, large change in bladder size in one case, and body motion in one other case. The tracking algorithm was also validated quantitatively on 2 cases for which ground truth contours were available and reliable. Mean SDE for the two cases was 0.45, 0.32, and 0.31 mm in superior-inferior, left-right, and anterior-posterior directions, respectively. A continuous motion drift of the prostate was observed over a 600 s period for one of these two cases and was accurately tracked by the technique.We have demonstrated the feasibility to track prostate motion based on real-time orthogonal cine MRI without implanting fiducials. With further investigation on more patients and following careful workflow, the real-time motion tracking technique may be implemented in MRgART to manage intrafraction motion.H. Jassar: None. A. Tai: None. X. Chen: None. T. Keiper: None. E.S. Paulson: Research Grant; Siemens Healthineers, Elekta Instruments AB.F. Lathuiliere: None. S. Bériault: None. D. Cooper: None. F. Hebert: None. L. Savard: None. S. Cloake: None. W.A. Hall: Research Grant; Elekta, National Cancer Institute, American Cancer Society. C.A. Lawton: None. A. Li: Research Grant; Elekta AB, Accuray Inc, Siemens Healthineers, Manteia Med. Honoraria; Elekta AB, Accuray Inc. Patent/License Fees/Copyright; Manteia Med.

Transfer learning from an artificial radiograph-landmark dataset for registration of the anatomic skull model to dual fluoroscopic X-ray images

Registration of 3D anatomic structures to their 2D dual fluoroscopic X-ray images is a widely used motion tracking technique. However, deep learning implementation is often impeded by a paucity of medical images and ground truths. In this study, we proposed a transfer learning strategy for 3D-to-2D registration using deep neural networks trained from an artificial dataset. Digitally reconstructed radiographs (DRRs) and radiographic skull landmarks were automatically created from craniocervical CT data of a female subject. They were used to train a residual network (ResNet) for landmark detection and a cycle generative adversarial network (GAN) to eliminate the style difference between DRRs and actual X-rays. Landmarks on the X-rays experiencing GAN style translation were detected by the ResNet, and were used in triangulation optimization for 3D-to-2D registration of the skull in actual dual-fluoroscope images (with a non-orthogonal setup, point X-ray sources, image distortions, and partially captured skull regions). The registration accuracy was evaluated in multiple scenarios of craniocervical motions. In walking, learning-based registration for the skull had angular/position errors of 3.9 ± 2.1°/4.6 ± 2.2 mm. However, the accuracy was lower during functional neck activity, due to overly small skull regions imaged on the dual fluoroscopic images at end-range positions. The methodology to strategically augment artificial training data can tackle the complicated skull registration scenario, and has potentials to extend to widespread registration scenarios.