Body Surface Motion Research Articles

Real-Time 3D Liver Tumor Localization via Combined Optical Surface Imaging and an X-Ray Projection from Arbitrary Imaging Angles

A deep learning (DL)-based, deformable registration-driven liver tumor localization technique was developed for onboard deformable motion tracking. The technique (Surf-X360-Bio) uses in-room optical surface imaging and an x-ray projection at an arbitrary scan angle to solve volumetric liver and liver tumor motion in real-time. Surf-X360-Bio solves the volumetric motion of the liver and localizes the liver tumor, through deforming liver and liver tumor meshes segmented on prior 4D-CTs/CBCTs. It uses real-time onboard information from an optical surface image and a simultaneously-acquired x-ray projection (from an arbitrary scan angle). Surf-X360-Bio localizes tumors via two steps: liver boundary motion estimation and intra-liver motion derivation. Surf-X360-Bio first estimates liver boundary motion by a patient-specific surface imaging model (Surf), utilizing the correlation between the external body surface motion and the internal liver boundary motion. As the correlation can be imperfect, the residual motion estimation errors were corrected by a patient-specific, angle-agnostic x-ray imaging model (X360). X360 deformed the liver boundary to match motion-related imaging features encoded in an arbitrarily-angled x-ray projection, using the Surf output for initialization. X360 adopted a geometry-aware learning module to extract and adapt to angle-varying features of arbitrarily-angled x-ray images, by calculating the projection system matrix of each x-ray image on-the-fly during model training and inference. After the liver boundary motion estimation by Surf and X360, intra-liver deformation was solved by a biomechanical model (Bio) to propagate the liver boundary motion inside to localize the tumors. The DL-based Bio model used domain knowledge of tissue biomechanics and finite element analysis (FEA) to solve intra-liver motion, with a much faster speed than conventional FEA methods to meet the real-time constraint. Surf-X360-Bio was evaluated using a dataset of 34 liver patients. Liver tumor localization accuracy was evaluated by the Dice similarity coefficient (DSC), the 95-percentile Hausdorff distance (HD95), and the center-of-mass error (COME). Using 3,306 motion scenarios spanning the 360 degree x-ray scan angles for each testing patient, Surf-X360-Bio localized the liver tumors to 0.81 (mean) ± 0.16 (s.d.) in DSC, 2.5 ± 1.7 mm in HD95, and 2.1 ± 1.8 mm in COME. In comparison, the prior reference image without deformable registration-driven localization yielded 0.42 ± 0.29 in DSC, 8.1 ± 5.2 mm in HD95, and 8.5 ± 5.2 mm in COME. Via Surf-X360-Bio, the overall inference time was below 230 ms for each case. Combining optical surface imaging and x-ray imaging, Surf-X360-Bio achieved fast (<250 ms inference time), accurate (mean error < 2.1 mm), and robust liver tumor localizations at arbitrary x-ray scan angles for real-time, marker-less 3D deformable motion tracking.

Novel Muscle Sensing by Radiomyography (RMG) and Its Application to Hand Gesture Recognition.

Conventional electromyography (EMG) measures the continuous neural activity during muscle contraction, but lacks explicit quantification of the actual contraction. Mechanomyography (MMG) and accelerometers only measure body surface motion, while ultrasound, CT-scan and MRI are restricted to in-clinic snapshots. Here we propose a novel radiomyography (RMG) for continuous muscle actuation sensing that can be wearable or touchless, capturing both superficial and deep muscle groups. We verified RMG experimentally by a wearable forearm sensor for hand gesture recognition (HGR). We first converted the sensor outputs to the time-frequency spectrogram, and then employed the vision transformer (ViT) deep learning network as the classification model, which can recognize 23 gestures with an average accuracy up to 99% on 8 subjects. By transfer learning, high adaptivity to user difference and sensor variation were achieved at an average accuracy up to 97%. We further extended RMG to monitor eye and leg muscles and achieved high accuracy for eye movement and body posture tracking. RMG can be used with synchronous EMG to derive stimulation-actuation waveforms for many potential applications in kinesiology, physiotherapy, rehabilitation, and human-machine interface.

Deep learning-based synthetization of real-time in-treatment 4D images using surface motion and pretreatment images: A proof-of-concept study.

To develop a deep learning model that maps body surface motion to internal anatomy deformation, which is potentially applicable to dose-free real-time 4D virtual image-guided radiotherapy based on skin surface data. Body contours were segmented out of 4DCT images. Deformable image registration algorithm was used to register the end-of-exhalation (EOE) phase to other phases. Deformation vector field was dimension-reduced to the first two principal components (PCs). A deep learning model was trained to predict the two PC scores of each phase from surface displacement. The instant deformation field can then be reconstructed, warping EOE image to obtain real-time CT image. This approach was validated on 4D XCAT phantom, the public DIR-Lab, and 4D-Lung dataset respectively, with and without simulated noise. Validation accuracy of the tumor centroid trajectory was observed as 0.04 ± 0.02mm on XCAT phantom. For the DIR-Lab dataset, 300 landmarks were annotated on the end-of-inhalation (EOI) images of each patient, and the mean displacements between their predicted and reference positions were below 2mm for all studied cases. For the 4D-Lung dataset, the average dice coefficients ± std between predicted and reference tumor contours at EOI phase were 0.835 ± 0.092 for all studied cases. A deep learning-based approach was proposed and validated to predict internal anatomy deformation from the surface motion, which is potentially applicable to on-line target navigation for accurate radiotherapy based on real-time 4D skin surface data and pretreatment images.

Stereotactic Body Radiotherapy for Locally Advanced Pancreatic Cancer Using Optical Surface Management System - AlignRT as an Optical Body Surface Motion Management in Deep Breath Hold Patients: Results from a Single-Arm Retrospective Study.

PurposeTo assess the efficacy and safety of stereotactic body radiotherapy for patients with unresectable, locally advanced pancreatic cancer using Optical Surface Management System – AlignRT (OSMS-AlignRT) as an optical body surface motion management in deep breath hold.Patients and MethodsForty-five patients diagnosed with locally advanced pancreatic cancer were treated with stereotactic body radiotherapy in 3 or 5 fractions, and received varying BED10 (median 79.5 Gy) from April 2017 to December 2020. All patients were treated in deep breath hold with OSMS-AlignRT used as optical body surface motion management. Thirty-three patients received systemic treatment before and/or after stereotactic body radiotherapy, and twelve patients received no systemic treatment. In this retrospective, observational, single-arm study, primary endpoints were overall survival and freedom from local progression (ie, local control). Secondary endpoints were progression-free survival and toxicity. Actuarial survival analysis and univariate analysis were investigated.ResultsData from forty-five patients were analyzed. Median follow-up was 15 months. One-year freedom from local progression and survival were 95.5% and 71.1%, respectively. Median progression-free survival was 14 months. Median overall survival from diagnosis for all patients was 17 months, and 19 months for patients alive at the time of analysis. No patient had >G2 toxicity.ConclusionStereotactic body radiotherapy for locally advanced pancreatic cancer using OSMS-AlignRT as optical body surface motion management in deep breath hold patients is an effective and safe local treatment option, with no >G2 toxicity, and could be a promising therapeutic option with acceptable toxicity, either as a single treatment or in a multimodal regimen. OSMS-AlignRT provided accurate and reliable body surface motion management during stereotactic body radiotherapy.

Towards automated ultrasound imaging—robotic image acquisition in liver and prostate for long-term motion monitoring

Real-time volumetric (4D) ultrasound has shown high potential for diagnostic and therapy guidance tasks. One of the main drawbacks of ultrasound imaging to date is the reliance on manual probe positioning and the resulting user dependence. Robotic assistance could help overcome this issue and facilitate the acquisition of long-term image data to observe dynamic processes in vivo over time. The aim of this study is to assess the feasibility of robotic probe manipulation and organ motion quantification during extended imaging sessions. The system consists of a collaborative robot and a 4D ultrasound system providing real-time data access. Five healthy volunteers received liver and prostate scans during free breathing over 30 min. Initial probe placement was performed with real-time remote control with a predefined contact force of 10 N. During scan acquisition, the probe position was continuously adjusted to the body surface motion using impedance control. Ultrasound volumes, the pose of the end-effector and the estimated contact forces were recorded. For motion analysis, one anatomical landmark was manually annotated in a subset of ultrasound frames for each experiment. Probe contact was uninterrupted over the entire scan duration in all ten sessions. Organ drift and imaging artefacts were successfully compensated using remote control. The median contact force along the probe’s longitudinal axis was 10.0 N with maximum values of 13.2 and 21.3 N for liver and prostate, respectively. Forces exceeding 11 N only occurred in 0.3% of the time. Probe and landmark motion were more pronounced in the liver, with median interquartile ranges of 1.5 and 9.6 mm, compared to 0.6 and 2.7 mm in the prostate. The results show that robotic ultrasound imaging with dynamic force control can be used for stable, long-term imaging of anatomical regions affected by motion. The system facilitates the acquisition of 4D image data in vivo over extended scanning periods for the first time and holds the potential to be used for motion monitoring for therapy guidance as well as diagnostic tasks.

The Novel Method to Reconstruct Three-Dimensional Target Motion From Body Surface Motion for Dynamic Moving Phantom

We have developed the new dynamic moving phantom (DMP) with three axis drives to reproduce three-dimensional(3D) target motion plus one axis drive for body surface motion. The motion data for operating this phantom is able to be acquired by rehearsing dynamic tracking therapy (DTT) of a patient. In this study, the novel approach which reconstructs the 3D long time information from respiratory motion (RPM) data on the body surface during planning CT and the target motion measured by 4DCT is developed. First, the RPM data on body surface during planning CT in each patient was obtained by laser ranger (u(t)). Then, the average wave of u(t) when 4DCT are scanned was calculated and 3D target motion was obtained by analyzing 4DCT. Next, correlation functions between average wave and 3D target motion in LR, AP and SI directions were created. Finally, long time 3D target motion was generated by the correlation function with u(t) in each patient. Two different correlation models were tested. One is dual polynomial model which is composed of polynomial functions separated by inspiratory and expiratory phases (P(u(t))). Another one is simple linear fitting function (L(u(t))). RPM data of 8 lung cancer patients who participated in clinical trial with DTT using gimbaled linac were used in this study. The average absolute difference between actual target position that was already known from tracking log data and reconstructed position were calculated to evaluate two correlation models. The significance between the average absolute difference of L (u (t)) and that of P (u (t)) were evaluated by Wilcoxon’s signed rank one-sided test. Table summarizes the result of the average of absolute difference in each patient. The difference between actual target position and reconstructed position by L(u(t)) in LR, AP and SI were 0.12±0.08 mm, 0.53±0.29 mm, 0.31±0.18 mm, respectively. On the other hand, those by P(u(t)) in LR, AP and SI were 0.23±0.14 mm, 0.87±0.54 mm, 0.69±0.40 mm, respectively (p = 0.018, 0.005, 0.027). The novel approach which reconstructs the 3D target motion from RPM data during planning CT and 3D target motion measured by 4DCT were shown. Dual polynomial model that we investigated in this study was better to reconstruct 3D target motion.Abstract 3619Pt.No.Dual polynomial modelLinear modelLR (mm)SI (mm)AP (mm)LR (mm)SI (mm)AP (mm)AVSDAVSDAVSDAVSDAVSDAVSD10.070.040.300.150.140.100.090.070.300.240.400.2720.220.130.490.300.390.250.220.130.520.351.070.5930.020.010.440.200.100.080.050.020.680.360.420.2740.130.060.480.270.060.060.220.090.630.360.160.1250.060.050.690.430.480.290.340.191.860.941.440.7460.200.170.530.250.210.140.540.320.500.290.430.2470.100.060.420.350.270.140.130.091.040.780.300.1680.120.130.870.420.840.410.270.171.400.981.290.81AV0.120.080.530.290.310.180.230.140.870.540.690.40 Open table in a new tab

Results from a clinical trial evaluating the efficacy of real-time body surface visual feedback in reducing patient motion during lung cancer radiotherapy

Introduction: Optical surface measurement devices are a maturing technology in radiotherapy. The challenge for such devices is to demonstrate how they can improve clinical care. We present results from a phase 1 clinical trial designed to test the hypothesis that if presented with live data from a novel optical measurement device, showing their deviation from an ideal radiotherapy treatment position, patients will be able to better control their motion and increase their geometrical conformance.Method and materials: Fourteen lung cancer patients were enrolled in a prospective clinical study and asked to use a variety of visual feedback schema from a novel in-house developed optical surface measurement device. The magnitude and regularity of their body surface motion using the different schema was compared to that when free-breathing at three time-points throughout their radiotherapy treatment schedule. Additionally, 4D Cone Beam CT data, acquired simultaneously with the optical measurements, was used to test if improvements in external motion are reflected in changes in internal tumor motion.Results: The primary endpoint of the trial, device tolerability assessed by the fraction of participants completing all study sessions, was 86%. Secondary endpoints showed that use of the visual feedback device was found to statistically significantly decrease body surface motion magnitude by an average of 17% over the study cohort, although not universally. Similarly body surface motion variability was decreased by 18% on average. Internal tumor motion magnitude was also found to be statistically significantly decreased by an average of 14% when using the feedback device. Reduction in external motion was predictive of reduced internal motion but no evidence of a simple correlation between changes in internal and external motion magnitude was found.Conclusions: Visual feedback of live motion is well tolerated by lung cancer patients and can reduce both body surface and tumor motion.

MoSh

Marker-based motion capture (mocap) is widely criticized as producing lifeless animations. We argue that important information about body surface motion is present in standard marker sets but is lost in extracting a skeleton. We demonstrate a new approach called MoSh (Motion and Shape capture), that automatically extracts this detail from mocap data. MoSh estimates body shape and pose together using sparse marker data by exploiting a parametric model of the human body. In contrast to previous work, MoSh solves for the marker locations relative to the body and estimates accurate body shape directly from the markers without the use of 3D scans; this effectively turns a mocap system into an approximate body scanner. MoSh is able to capture soft tissue motions directly from markers by allowing body shape to vary over time. We evaluate the effect of different marker sets on pose and shape accuracy and propose a new sparse marker set for capturing soft-tissue motion. We illustrate MoSh by recovering body shape, pose, and soft-tissue motion from archival mocap data and using this to produce animations with subtlety and realism. We also show soft-tissue motion retargeting to new characters and show how to magnify the 3D deformations of soft tissue to create animations with appealing exaggerations.

Investigation of sagittal image acquisition for 4D-MRI with body area as respiratory surrogate.

The authors have recently developed a novel 4D-MRI technique for imaging organ respiratory motion employing cine acquisition in the axial plane and using body area (BA) as a respiratory surrogate. A potential disadvantage associated with axial image acquisition is the space-dependent phase shift in the superior-inferior (SI) direction, i.e., different axial slice positions reach the respiratory peak at different respiratory phases. Since respiratory motion occurs mostly in the SI and anterior-posterior (AP) directions, sagittal image acquisition, which embeds motion information in these two directions, is expected to be more robust and less affected by phase-shift than axial image acquisition. This study aims to develop and evaluate a 4D-MRI technique using sagittal image acquisition. The authors evaluated axial BA and sagittal BA using both 4D-CT images (11 cancer patients) and cine MR images (6 healthy volunteers and 1 cancer patient) by comparing their corresponding space-dependent phase-shift in the SI direction (δSPS (SI)) and in the lateral direction (δSPS (LAT)), respectively. To evaluate sagittal BA 4D-MRI method, a motion phantom study and a digital phantom study were performed. Additionally, six patients who had cancer(s) in the liver were prospectively enrolled in this study. For each patient, multislice sagittal MR images were acquired for 4D-MRI reconstruction. 4D retrospective sorting was performed based on respiratory phases. Single-slice cine MRI was also acquired in the axial, coronal, and sagittal planes across the tumor center from which tumor motion trajectories in the SI, AP, and medial-lateral (ML) directions were extracted and used as references from comparison. All MR images were acquired in a 1.5 T scanner using a steady-state precession sequence (frame rate ∼ 3 frames/s). 4D-CT scans showed that δSPS (SI) was significantly greater than δSPS (LAT) (p-value: 0.012); the median phase-shift was 16.9% and 7.7%, respectively. Body surface motion measurement from axial and sagittal MR cines also showed δSPS (SI) was significantly greater than δSPS (LAT). The median δSPS (SI) and δSPS (LAT) was 11.0% and 9.2% (p-value = 0.008), respectively. Tumor motion trajectories from 4D-MRI matched with those from single-slice cine MRI: the mean (±SD) absolute differences in tumor motion amplitude between the two were 1.5 ± 1.6 mm, 2.1 ± 1.9 mm, and 1.1 ± 1.0 mm in the SI, ML, and AP directions from this patient study. Space-dependent phase shift is less problematic for sagittal acquisition than for axial acquisition. 4D-MRI using sagittal acquisition was successfully carried out in patients with hepatic tumors.

An investigation using high-precision CCD laser displacement sensor to measure body surface motion induced by heartbeat

The aim of this study was to measure the motion of the body surface induced by heartbeat by using a high-precision laser sensor in order to provide a reference for other research on medical remote sensing techniques that are unconstrained and non-contact. Body surface motions at some points on the chest and on the dorsal region were measured for eight men with a laser displacement sensor. For comparison, an electrocardiograph was also used to monitor the heartbeat. As a result, we confirmed cyclic motion of about 0.1 mm in amplitude everywhere on the body surface, not only from the front of the body but also from the back and the neck. In contrast, which part on the body was most suitable for measurement had to be taken into account because a difference in motion amplitudes was observed in different parts on the body. The amplitude of the motions also depended on the conditions of the body surface as well as underneath the skin. These results show the possibility of acquiring information about heartbeats from anywhere on the body by using an unconscious sensing technique, and the potential for the technique to monitor the condition of the body and personal physical characteristics.

SU-FF-J-34: Video-Surface-Guided Respiratory Motion Correction for 3D Static CT Images

Purpose: Three-dimensional (3D) static CT images of structures in the thorax and abdomen constantly suffer from artifacts caused by periodic respiratory motion. This study presents a novel method to correct respiratory motion for 3D CT images by correlating CT images with dynamic body surface models in which physiologic motion of body surface is recorded with the aid of a 3D video imaging system. Method and Materials: We introduce a new dynamic CT imaging technique that utilizes a commercially available high-speed 3D camera system to gather magnitude and frequency information of respiration cycle by acquiring the motion of body surface. A dynamic body surface model is built to record 3D surface geometry and texture information at different phases of respiration cycle. Retrospective gating technique is then adopted to correlate or register CT images with the dynamic body surface model. Multiple skin markers shown on CT and video surface images are used for verification of the dynamic model. Results: In this study, motion artifacts were remarkable reduced and accurate 4D CT datasets were generated for the further use by planning systems. Conclusion: Quantifying internal anatomy motion as a function of respiration cycle is important in conformal radiotherapy, especially for lung and breast tumors. Compared to conventional 3D CT, four-dimensional CT (4D CT) techniques present overwhelming advantage on imaging objects undergoing periodic motion. The feasibility of 4D CT techniques based on a 3D video imaging system has been demonstrated in this work. Correlating these images with the dynamic body surface models reduced respiratory motion artifacts for 3D CT images. Conflict of Interest: Partially supported by the camera company.

Evaluating regional body surface motion during breathing using stereophotogrammetry

The stereophotometric analysis of biological forms in medicine has been usefully employed for many years. Generally, the methods adopted have involved the use of stereo pairs of traditional photographs or radiographs. Although very successful, the traditional methods are time-consuming. Developments in solid-state and videoscan cameras, in computer hardware and in image processing technology have lead to real-time photogrammetry (RTP) or near-real-time photogrammetry (NRTP). It is anticipated that RTP and NRTP will play a vital role in biostereometric studies. This paper discusses the development of a low-cost PC-based NRTP system and the possibility of using NRTP in an ongoing biostereometric study which has been developed for studying regional body surface motion in the evaluation of respiratory muscle function and in which digital cameras will be substituted for analogue cameras and computer digitisation will be substituted for manual observation.