Motion Tracking Research Articles

A Real-time Motion Capture System for 3-D Virtual Characters

Motion tracking method is being issued as essential part of the entertainment, medical, sports, education and industry with the development of 3-D virtual reality. Virtual human character in the digital animation and game application has been controlled by interfacing devices; mouse, joysticks, midi-slider, and so on. Those devices could not enable virtual human character to move smoothly and naturally. Furthermore, high-end human motion capture systems in commercial market are expensive and complicated. In this paper, we proposed a practical and fast motion capturing system consisting of optic sensors, and linked the data with 3-D game character with real time. The prototype experiment setup is successfully applied to a boxing game which requires very fast movement of human character

Accurate prediction of neurologic changes in critically ill infants using pose AI.

Infant alertness and neurologic changes can reflect life-threatening pathology but are assessed by exam, which can be intermittent and subjective. Reliable, continuous methods are needed. We hypothesized that our computer vision method to track movement, pose AI, could predict neurologic changes in the neonatal intensive care unit (NICU). We collected 4,705 hours of video linked to electroencephalograms (EEG) from 115 infants. We trained a deep learning pose algorithm that accurately predicted anatomic landmarks in three evaluation sets (ROC-AUCs 0.83-0.94), showing feasibility of applying pose AI in an ICU. We then trained classifiers on landmarks from pose AI and observed high performance for sedation (ROC-AUCs 0.87-0.91) and cerebral dysfunction (ROC-AUCs 0.76-0.91), demonstrating that an EEG diagnosis can be predicted from video data alone. Taken together, deep learning with pose AI may offer a scalable, minimally invasive method for neuro-telemetry in the NICU.

Human Abnormal Activity Recognition from Video Using Motion Tracking

Human Abnormal Activity Recognition from Video Using Motion Tracking

Control of a painting robot containing parallelogram linkages based on its distinctive dynamic characteristics

To achieve high-performance control, the dynamics and control should be closely integrated. This paper focuses on dynamic-based control design for a hybrid painting robot containing parallelogram linkages to improve its tracking accuracy and motion consistency. The study derives the dynamics of the parallelogram component to investigate its distinctive attributes, which include time invariance, joint independence, and linear approximability. Based on these unique characteristics, a novel theory-experiment mixed two-step method for control design is presented. It involves identifying and visualizing an optimized space of control parameters through theoretical deduction, followed by determining the optimal control parameters within this space through experimental trials. Experimental results underscore a substantial improvement in the tracking accuracy and motion consistency of the painting robot.

Preliminary Experience with Three Alternative Motion Sensors for 0.55 Tesla MR Imaging.

Due to limitations in current motion tracking technologies and increasing interest in alternative sensors for motion tracking both inside and outside the MRI system, in this study we share our preliminary experience with three alternative sensors utilizing diverse technologies and interactions with tissue to monitor motion of the body surface, respiratory-related motion of major organs, and non-respiratory motion of deep-seated organs. These consist of (1) a Pilot-Tone RF transmitter combined with deep learning algorithms for tracking liver motion, (2) a single-channel ultrasound transducer with deep learning for monitoring bladder motion, and (3) a 3D Time-of-Flight camera for observing the motion of the anterior torso surface. Additionally, we demonstrate the capability of these sensors to simultaneously capture motion data outside the MRI environment, which is particularly relevant for procedures like radiation therapy, where motion status could be related to previously characterized cyclical anatomical data. Our findings indicate that the ultrasound sensor can track motion in deep-seated organs (bladder) as well as respiratory-related motion. The Time-of-Flight camera offers ease of interpretation and performs well in detecting surface motion (respiration). The Pilot-Tone demonstrates efficacy in tracking bulk respiratory motion and motion of major organs (liver). Simultaneous use of all three sensors could provide complementary motion information outside the MRI bore, providing potential value for motion tracking during position-sensitive treatments such as radiation therapy.

Virtual reality, electrophysiology & motion tracking technologies in mental illnesses

The following mini-review paper explores the integration of Virtual Reality (VR), motion tracking, and electrophysiological sensors in the context of exposure therapy and mental health treatment. These technologies collectively offer innovative approaches for enhancing therapeutic goals, providing immersive environments, and enabling precise monitoring of physiological responses. The paper discusses their applications, benefits, and potential challenges, underscoring their transformative impact on mental health care.

Investigating the use of comprehensive motion monitoring for intrafraction 3D drift assessment of hypofractionated prostate cancer patients on a 1.5T magnetic resonance imaging radiotherapy system

This work investigates the use of a multi-2D cine magnetic resonance imaging-based comprehensive motion monitoring (CMM) system for the assessment of prostate intrafraction 3D drifts. The data of six healthy volunteers were analyzed and the values of a clinically-relevant registration quality factor metric exported by CMM were presented. Additionally, the CMM-derived prostate motion was compared to a 3D-based reference and the 2D-3D tracking agreement was reported. Due to the low quality of SI motion tracking (often >2 mm tracking mismatch between anatomical planes) we conclude that further improvements are desirable prior to clinical introduction of CMM for prostate drift corrections.

Antegrade anterior column acetabulum fracture fixation with cannulated compression headless screws-A biomechanical study on standardized osteoporotic artificial bone.

Due to the increase in life expectancy and high-energy traumas, anterior column acetabular fractures (ACFs) are also increasing. While open reduction and internal fixation (ORIF) is still the standard surgical procedure, minimally invasive, percutaneous fixation of osteoporotic acetabulum fractures (AF) are growing in popularity. The aim of this biomechanical study was to evaluate the biomechanical competence following antegrade fixation with a standard screw versus a cannulated compression headless screw. Eight anatomical osteoporotic composite pelvises were given an anterior column fracture. Two groups of eight specimens each (n = 8) for fixation with either a 6.5 mm cannulated compression headless screw in group Anterior Acetabulum Canulated Compression Headless Screw (AACCH), or with a 6.5 mm partially threaded cannulated screw in group Anterior Acetabulum Standard Screw (AASS) where compared. Each specimen was biomechanically loaded cyclically at a rate of 2 Hz with monotonically increasing compressive load until failure. Motions were assessed by means of optical motion tracking. Initial construct stiffness trended higher in group AACCH at 152.4 ± 23.1 N/mm compared to group AASS at 118.5 ± 34.3 N/mm, p = 0.051. Numbers of cycles and corresponding peak load at failure, were significantly higher in group AACCH at 6734 ± 1669 cycles and 873.4 ± 166.9 N versus group AASS at 4440 ± 2063 cycles and 644.0 ± 206.3 N, p = 0.041. Failure modes were breakout of the screws around the proximal entry point. From a biomechanical perspective, group AACCH was associated with superior biomechanical competence compared to standard partially threaded cannulated screws and could therefore be considered as valid alternative for fixation of anterior acetabulum fractures.

Co2iAR: Co-located audio-visual enabled mobile collaborative industrial AR wiring harness assembly

Existing augmented reality (AR) assembly mainly provides visual instructions for operators from a first-person perspective, and it is hard to share individual working intents for co-located workers on the shop floor, especially for large-scale product assembly task that requires multiple operators working together. To bridge this gap for practical deployments, this paper proposes Co2iAR, a co-located audio-visual enabled mobile collaborative AR assembly. Firstly, according to the stereo visual-inertial fusion strategy, robust and accurate self-contained motion tracking is achieved for the resource-constrained mobile AR platform, followed by a co-located alignment from multiple mobile AR clients on the shop floor. Then, a lightweight text-aware network for online wiring harness character recognition is proposed, as well as the audio-based confirming strategy, enabling natural audio-visual interaction among co-located workers within a shared immersive workplace, which can also monitor the current wiring assembly status and activate the step-by-step tutorials automatically. The novelty of this work is focused on the deployment of audio-visual aware interaction using the same device that is being used to deploy the co-located collaborative AR work instructions, establishing shared operating intents among multiple co-located workers. Finally, comprehensive experiments are carried out on the collaborative performance among multiple AR clients, and results illustrate that the proposed Co2iAR can alleviate the cognitive load and achieve superior performance for the co-located AR assembly tasks, providing a more human-centric collaborative assembly performance.

Stereoscopic imaging of volatile organic compounds distribution in the region and tracing emission sources of volatile organic compounds using a novel movable single-photon ionization time-of-flight mass spectrometer.

This paper presents a newly developed high-performance mobile single-photon ionization time-of-flight mass spectrometry (M-SPI-TOFMS) system for on-line analysis and stereoscopic monitoring of complex gas mixtures. The system is designed for stereoscopic imaging to map the distribution of volatile organic compounds (VOCs) and trace their emission sources in urban areas and industrial parks. It mainly consists of a SPI-TOFMS instrument, a customized commercial vehicle, a meteorological five-parameter monitor with GPS, a high-power generator, and an uninterruptible power supply. The SPI technique, using a 118 nm VUV lamp, can ionize compounds with an ionization potential below 10.78 eV. Mass spectra obtained using this technique show the profiles of various VOCs and some inorganic compounds. The VOCs composition information and mobile location data are simultaneously sent to the GIS software. In GIS software, this data is used for real-time stereoscopic imaging of VOC distribution and precise tracking of VOC movement. The system can achieve a spatial data resolution of 0.69 mm at 25 km/h due to the microsecond detection speed of the M-SPI-TOFMS instrument. The laboratory test provides a rapid overview characterization of benzene, toluene, and xylene. The M-SPI-TOFMS has limits of detection and mass resolution of 33.7 pptv and 1060, respectively. Several field applications were carried out using M-SPI-TOFMS at various locations to identify VOC sources near different factories. The M-SPI-TOFMS system has a navigation monitoring speed of 25 km/h with a time resolution of 1 s. The widespread use of this system will provide accurate data to support environmental management departments in formulating VOCs pollution control policies and improving control efficiency.

Distributed prescribed-time coordinated control of spacecraft formation flying under input saturation

This paper studies the prescribed-time relative motion control problem of spacecraft formation flying under input saturation. Using prescribed-time theory, a prescribed-time sliding mode is designed such that the states on the sliding mode converge to the equilibrium in the prescribed time. Based on the prescribed-time sliding mode, a prescribed-time relative motion tracking controller is developed, which guarantees fast formation maneuvers with feasible fuel consumption and strong robustness under input saturation. Furthermore, a simulation example is carried out to verify the effectiveness of the proposed controller.

Combining IMU With Acoustics for Head Motion Tracking Leveraging Wireless Earphone

Combining IMU With Acoustics for Head Motion Tracking Leveraging Wireless Earphone

35‐3: Modeling Eye Movement and Reflection in Virtual Environments for Eye Tracking

Eye movement tracking plays a pivotal role in use interaction within Virtual Reality (VR) environments. Our research focuses on an advanced eye movement simulation using virtual environment for eye tracking training and eye tracking precision evaluating platform. We developed a real‐eye model, enhanced light refraction and corneal reflections under infrared LED light with eye rotation. Our study also customized screen,light source and cameras placement to capture eye images more faithfully. We also presents a noval testing methology and equipments of spatial accuracy testing of the eye‐tracking systems. The proposed proprietary eye‐tracking test system designed to recreate scenarios of human gaze and provide true values for eye gaze directions. Overall, the integration of Pupil Center Corneal Reflection (PCCR) based dataset generation and testing platform mark a significant advancement, enabling huge database, precise groudtruth training and accurate results testing.

Diffeomorphic transformer-based abdomen MRI-CT deformable image registration.

Stereotactic body radiotherapy (SBRT) is a well-established treatment modality for liver metastases in patients unsuitable for surgery. Both CT and MRI are useful during treatment planning for accurate target delineation and to reduce potential organs-at-risk (OAR) toxicity from radiation. MRI-CT deformable image registration (DIR) is required to propagate the contours defined on high-contrast MRI to CT images. An accurate DIR method could lead to more precisely defined treatment volumes and superior OAR sparing on the treatment plan. Therefore, it is beneficial to develop an accurate MRI-CT DIR for liver SBRT. To create a new deep learning model that can estimate the deformation vector field (DVF) for directly registering abdominal MRI-CT images. The proposed method assumed a diffeomorphic deformation. By using topology-preserved deformation features extracted from the probabilistic diffeomorphic registration model, abdominal motion can be accurately obtained and utilized for DVF estimation. The model integrated Swin transformers, which have demonstrated superior performance in motion tracking, into the convolutional neural network (CNN) for deformation feature extraction. The model was optimized using a cross-modality image similarity loss and a surface matching loss. To compute the image loss, a modality-independent neighborhood descriptor (MIND) was used between the deformed MRI and CT images. The surface matching loss was determined by measuring the distance between the warped coordinates of the surfaces of contoured structures on the MRI and CT images. To evaluate the performance of the model, a retrospective study was carried out on a group of 50 liver cases that underwent rigid registration of MRI and CT scans. The deformed MRI image was assessed against the CT image using the target registration error (TRE), Dice similarity coefficient (DSC), and mean surface distance (MSD) between the deformed contours of the MRI image and manual contours of the CT image. When compared to only rigid registration, DIR with the proposed method resulted in an increase of the mean DSC values of the liver and portal vein from 0.850±0.102 and 0.628±0.129 to 0.903±0.044 and 0.763±0.073, a decrease of the mean MSD of the liver from 7.216±4.513mm to 3.232±1.483mm, and a decrease of the TRE from 26.238±2.769mm to 8.492±1.058mm. The proposed DIR method based on a diffeomorphic transformer provides an effective and efficient way to generate an accurate DVF from an MRI-CT image pair of the abdomen. It could be utilized in the current treatment planning workflow for liver SBRT.

Alertness Assured: Sleep Detection and Video Control Technology

Abstract: The system utilizes advanced computer vision algorithms to analyze real-time video streams, employing techniques such as facial landmark detection and eye movement tracking. By continuously monitoring these visual cues, the system can accurately detect signs of drowsiness, such as drooping eyelids or prolonged eye closures, prompting timely interventions to prevent accidents or errors caused by fatigue. In addition to detecting drowsiness, the system also evaluates the user's attention levels by analysing head orientation and facial expressions. By tracking head movements and assessing changes in facial expressions indicative of engagement or distraction, the system provides valuable insights into the user's cognitive state. This attention assessment component enables the system to adapt its feedback and intervention strategies based on the user's level of alertness and focus. To enhance user awareness and responsiveness, the system employs graphical visualization techniques to display real-time feedback on drowsiness and attention status. Visual indicators, such as color-coded alerts or dynamic graphs depicting attention trends, provide users with intuitive insights into their cognitive performance, empowering them to make informed decisions about their work habits or driving patterns. Moreover, the system utilizes auditory alerts to supplement visual feedback, ensuring that users receive timely notifications even in noisy or visually demanding environments. Whether it's a gentle reminder to take a short break or a more urgent warning about escalating drowsiness levels, auditory cues serve as an effective means of alerting users to potential risks and encouraging proactive intervention

Camera-Based Dynamic Vibration Analysis Using Transformer-Based Model CoTracker and Dynamic Mode Decomposition.

Accelerometers are commonly used to measure vibrations for condition monitoring in mechanical and civil structures; however, their high cost and point-based measurement approach present practical limitations. With rapid advancements in computer vision and deep learning, research into tracking the motion of individual pixels with vision cameras has increased. The recently developed CoTracker, a transformer-based model, has demonstrated excellence in motion tracking, yet its performance in measuring structural vibrations has not been fully explored. This paper investigates the efficacy of the CoTracker model in extracting full-field structural vibrations using cameras. It is initially applied to capture the dense point movements in video sequences of a cantilever beam recorded using a high-speed camera. Subsequently, modal analysis using delay-embedding dynamic mode decomposition (DMD) is conducted to extract modal parameters including natural frequencies, damping ratios, and mode shapes. The results, benchmarked against those from a reference accelerometer and the Finite Element Method (FEM) result, demonstrate CoTracker's high potential for general applicability in structural vibration measurements.

Tip shape effects on the axial-flow-induced vibration of a cantilever rod

The influence of tip shape on flow-induced vibration of a cantilever rod subjected to axial water flow is experimentally investigated, through optical tracking of the rod movement and mapping of the instantaneous flow field around the rod tip. The experimental setup consists of a vertical cantilever rod housed within a tube. The rod tip shapes considered in this study include a blunt tip and cones with height-to-diameter ratios of 0.5, 1, and 2. The experiments were conducted across a Reynolds number range between 20k to 100k in both clamped-free and free-clamped configurations, representing opposite flow directions. The rod tip motion was captured using fast video image tracking, whereas the flow field near the rod tip was obtained using particle image velocimetry (PIV). The rod vibration dynamics exhibited a primarily fuzzy period-1 behavior, characterized by a periodic motion with a chaotic component. Flutter-like oscillation and buckling were also observed at higher Reynolds numbers, depending on the flow direction. The mechanisms of fluid-structure interaction involved turbulent buffeting and movement-induced excitations. Unsteady flow separation around the rod tip was identified as a further contributing mechanism to flow excitation. In the clamped-free configuration, unsteady flow separation was more pronounced for the cone tips due to the increased rod surface area in the wake region, leading to larger vibration amplitude. In the free-clamped configuration, flow separation effects were more prominent for the blunt tip, as the streamlined cone shapes were less prone to flow separations. Overall, the rod with a blunt tip resulted in smaller displacement in the clamped-free configuration, while the rods with cone tips led to smaller displacement in the free-clamped configuration.

Development of an ICT Laparoscopy System with Motion-Tracking Technology for Solo Laparoscopic Surgery: A Feasibility Study

The increasing demand for laparoscopic surgery due to its cosmetic benefits and rapid post-surgery recovery is juxtaposed with a shortage of surgical support staff. This juxtaposition highlights the necessity for improved camera management in laparoscopic procedures, encompassing positioning, zooming, and focusing. Our feasibility study introduces the information and communications technology (ICT) laparoscopy system designed to aid solo laparoscopic surgery. This system tracks a surgeon’s body motion using a controller, manipulating an embedded camera to focus on specific surgical areas. It comprises a camera module, a camera movement controller, and a motor within the main body, operating connected wires according to controller commands for camera movement. Surgeon movements are detected by an inertial measurement unit (IMU) sensor, facilitating precise camera control. Additional features include a foot pedal switch for motion tracking, a dedicated trocar for main body stability, and a display module. The system’s effectiveness was evaluated using an abdomen phantom model and animal experimentation with a porcine model. The camera responded to human movement within 100 ms, a delay that does not significantly affect procedural performance. The ICT laparoscopy system with advanced motion-tracking technology is a promising tool for solo laparoscopic surgery, potentially improving surgical outcomes and overcoming staff shortages.

Kinematic synergies show good consistency when extracted with a low-cost markerless device and a marker-based motion tracking system

Recently, markerless tracking systems, such as RGB-Depth cameras, have spread to overcome some of the limitations of the gold standard marker-based tracking systems. Although these systems are valuable substitutes for human motion analysis, as they guarantee higher flexibility, faster setup time and lower costs, their tracking accuracy is lower with respect to marker-based systems. Many studies quantified the error made by markerless systems in terms of body segment length estimation, articular angles, and biomechanics, concluding that they are appropriate for many clinical applications related to motion analysis. We propose an innovative approach to compare a markerless tracking system (Kinect V2) with a gold standard marker-based system (Vicon), based on motor control assessment. We quantified kinematic synergies from the tracking data of fifteen participants performing multi-directional upper limb movements. Kinematic synergy analysis decomposes the kinematic data into a reduced set of motor primitives that describe how the central nervous system coordinates motion at spatial and temporal level. Synergies were extracted with the recently released mixed-matrix factorization algorithm. Four synergies were extracted from both marker-based and markerless datasets and synergies were grouped in 6 clusters for each dataset. Cosine similarity in each cluster was ⩾0.60 in both systems, showing good consistency of synergies. Good matching was found between synergies extracted from markerless and from marker-based data, with a cosine similarity between matched synergies ⩾0.60 in 5 out 6 synergies. These results showed that the markerless sensor can be feasible for kinematic synergy analysis for gross movements, as it correctly estimates the number of synergies and in most cases also their spatial and temporal organization.

The Use of Knotless Suture Tape Construct vs Screw Fixation for Lisfranc Injuries: A Cadaveric Biomechanical Study.

Lisfranc injuries are often treated with open reduction and internal fixation using rigid fixation techniques. The use of flexible fixation to stabilize the Lisfranc joint is a newer technique. The purpose of this cadaveric study is to compare the amount of diastasis at the Lisfranc interval under diminished physiologic loads when treated with a knotless suture tape construct and a solid screw. Ten cadavers (20 feet) had native motion at the intact Lisfranc interval assessed at multiple increasing loads (69, 138, and 207 N). The Lisfranc ligamentous complex was then disrupted, and testing repeated to evaluate the amount of diastasis. Randomization was performed to determine the type of fixation for each cadaver: solid screw or knotless suture tape construct. Once fixation was completed, specimens were cyclically loaded for 10 000 cycles at loads, and diastasis was quantified after each load cycle to compare the interventions. Diastasis was measured using motion tracking cameras and retroreflective marker sets. A non-inferiority statistical analysis was performed. Diastasis mean values were confirmed to be >2 mm for all load bearing conditions in the injury model. Posttreatment, diastasis was significantly reduced when compared to the sectioned conditions (P < .01) for both treatment options. Non-inferiority analyses showed that the knotless suture tape construct did not perform inferior to screw fixation for diastasis at the Lisfranc interval at any of the compared load states. Under the loads tested, there is no significant difference in diastasis at the Lisfranc interval when treating ligamentous Lisfranc injuries with a knotless suture tape construct or solid screws. Both reduced diastasis from the injured state and were not different from the intact state. In this cadaveric model with ligamentous Lisfranc injury, diastasis of a knotless suture tape construct is compared to solid screw fixation as tested.