Head Tracking Research Articles

Head & Neck Oncology Track: Is Preoperative Serum Albumin Predictive of Adverse Outcomes in Head and Neck Cancer Surgery?

Head & Neck Oncology Track: Is Preoperative Serum Albumin Predictive of Adverse Outcomes in Head and Neck Cancer Surgery?

Head & Neck Oncology Track: Margin Status and Survival Outcomes in Virtually Planned Oncologic Jaw Surgery

Head & Neck Oncology Track: Margin Status and Survival Outcomes in Virtually Planned Oncologic Jaw Surgery

Head movement during natural group conversation and inter-annotator agreement on manual annotation

During speech communication and conversational turn-taking, listeners direct their head and eyes to receive meaningful auditory and visual cues. Features of these behaviors may convey listener intent. This study designed a test environment, data collection protocol and procedures, and investigated head movement behaviors during self-driven conversations among multiple partners. Nine participants were tested in cohorts of three. Participants wore a headset with sensors tracked by an infrared camera system. Participants watched an audio-video clip, followed by a 5-min undirected discussion. The entire session was video recorded for annotation purposes. Two annotators independently coded the video files using the EUDICO Linguistic Annotator software application. Annotations were then co-registered with the head tracking data in post processing. Inter-annotator agreement demonstrated the desired reliability, thereby validating the procedures designed. Movement trajectories showed that there were individual differences on the head yaw distribution. The combination of objective measures of head movement and manual annotation of conversation behaviors provides a rich data set for characterizing natural conversations in ecologically valid settings. The measurement procedures and coding system developed here is a first step towards characterizing head movements during conversations needed to predict listening intent and to create actions based on those predictions.

Discrepancy between subjective and objective health indicators when performing monotonous operator activities against the background of chronic sleep deprivation

The aim of the study was to investigate various psychophysiological predictors of the quality of monotonous activity performance against the background of sleep deprivation. Fourteen subjects aged 18–22 years took part in the experiments. Each subject participated in four experimental sessions with partially automated control scenarios in a computerized driving simulator: the first experiment (15 minutes) – training session in the simulator without sleep deprivation; the second experiment – sleep deprivation (90 minutes); the third experiment – sleep deprivation and periodic exposure to a massage seat cover (90 minutes); the fourth experiment – sleep deprivation and periodic spraying of peppermint oil from an aroma diffuser (90 minutes). Every 3–7 minutes, subjects were asked to respond to two types of stimuli: 1) gradually changing; 2) appearing unexpectedly and requiring maneuvering. Stimulus intervals and stimulus order were varied randomly. Car driving simulator data, electroencephalogram using the standard 10–20 system with a sampling rate of 1000 Hz, electrocardiogram and skin-galvanic response were recorded. A three-dimensional video camera was mounted on a monitor in front of the subject to track head position and changes of the subject's facial expressions throughout the experiment. Subjects completed questionnaires on sleepiness and general well-being, and reaction time was also tested before the experiment. According to the results of the questionnaires and test performance, the subjects were in a state of chronic sleep deprivation and chronic stress. No statistically significant differences between the experiments after partial sleep deprivation without additional stimulation, with the use of a massage seat cover and periodic spraying of peppermint oil every 10 minutes were seen. The eye closure rate (PERCLOS) and spectral index of heart rate variability showed a significant statistically reliable increase before errors. The values of physiological indices before errors indicate different causes of errors in experiments without and with exposure to peppermint oil or massage seat cover.

Saliency prediction in 360° architectural scenes: Performance and impact of daylight variations

Saliency models are image-based prediction models that estimate human visual attention. Such models, when applied to architectural spaces, could pave the way for design decisions where visual attention is taken into account. In this study, we tested the performance of eleven commonly used saliency models that combine traditional and deep learning methods on 126 rendered interior scenes with associated head tracking data. The data was extracted from three experiments conducted in virtual reality between 2016 and 2018. Two of these datasets pertain to the perceptual effects of daylight and include variations of daylighting conditions for a limited set of interior spaces, thereby allowing to test the influence of light conditions on human head movement. Ground truth maps were extracted from the collected head tracking logs, and the prediction accuracy of the models was tested via the correlation coefficient between ground truth and prediction maps. To address the possible inflation of results due to the equator bias, we conducted complementary analyses by restricting the area of investigation to the equatorial image regions. Although limited to immersive virtual environments, the promising performance of some traditional models such as GBVS360eq and BMS360eq for colored and textured architectural rendered spaces offers us the prospect of their possible integration into design tools. We also observed a strong correlation in head movements for the same space lit by different types of sky, a finding whose generalization requires further investigations based on datasets more specifically developed to address this question.

Learning Spatial-Frequency Transformer for Visual Object Tracking

Recently, some researchers have begun to adopt the Transformer to combine or replace the widely used ResNet as their new backbone network. As the Transformer captures the long-range relations between pixels well using the self-attention scheme, which complements the issues caused by the limited receptive field of CNN. Although their trackers work well in regular scenarios, they simply flatten the 2D features into a sequence to better match the Transformer. We believe these operations ignore the spatial prior of the target object, which may lead to sub-optimal results only. In addition, many works demonstrate that self-attention is actually a low-pass filter, which is independent of input features or keys/queries. That is to say, it may suppress the high-frequency component of the input features and preserve or even amplify the low-frequency information. To handle these issues, in this paper, we propose a unified Spatial-Frequency Transformer that models the Gaussian spatial Prior and High-frequency emphasis Attention (GPHA) simultaneously. To be specific, Gaussian spatial prior is generated using dual Multi-Layer Perceptrons (MLPs) and injected into the similarity matrix produced by multiplying Query and Key features in self-attention. The output will be fed into a softmax layer and then decomposed into two components, i.e., the direct and high-frequency signal. The low- and high-pass branches are rescaled and combined to achieve all-pass, therefore, the high-frequency features will be protected well in stacked self-attention layers. We further integrate the Spatial-Frequency Transformer into the Siamese tracking framework and propose a novel tracking algorithm termed SFTransT. The cross-scale fusion based SwinTransformer is adopted as the backbone, and also a multi-head cross-attention module is used to boost the interaction between search and template features. The output will be fed into the tracking head for target localization. Extensive experiments on short-term and long-term tracking benchmarks all demonstrate the effectiveness of our proposed framework. Source code will be released at https://github.com/Tchuanm/SFTransT.git.

Enhanced Wearable Force-Feedback Mechanism for Free-Range Haptic Experience Extended by Pass-Through Mixed Reality

We present an extended prototype of a wearable force-feedback mechanism coupled with a Meta Quest 2 head-mounted display to enhance immersion in virtual environments. Our study focuses on the development of devices and virtual experiences that place significant emphasis on personal sensing capabilities, such as precise inside-out optical hand, head, and controller tracking, as well as lifelike haptic feedback utilizing servos and vibration rumble motors, among others. The new prototype addresses various limitations and deficiencies identified in previous stages of development, resulting in significant user performance improvements. Key enhancements include weight reduction, wireless connectivity, optimized power delivery, refined haptic feedback intensity, improved stylus alignment, and smooth transitions between stylus use and hand-tracking. Furthermore, the integration of a mixed reality pass-through feature enables users to experience a comprehensive and immersive environment that blends physical and virtual worlds. These advancements pave the way for future exploration of mixed reality applications, opening up new possibilities for immersive and interactive experiences that combine useful aspects of real and virtual environments.

Identification of Driver Status Hazard Level and the System.

According to the survey statistics, most traffic accidents are caused by the driver's behavior and status irregularities. Because there is no multi-level dangerous state grading system at home and abroad, this paper proposes a complex state grading system for real-time detection and dynamic tracking of the driver's state. The system uses OpenMV as the acquisition camera combined with the cradle head tracking system to collect the driver's current driving image in real-time dynamically, combines the YOLOX algorithm with the OpenPose algorithm to judge the driver's dangerous driving behavior by detecting unsafe objects in the cab and the driver's posture, and combines the improved Retinaface face detection algorithm with the Dlib feature-point algorithm to discriminate the fatigue driving state of the driver. The experimental results show that the accuracy of the three driver danger levels (R1, R2, and R3) obtained by the proposed system reaches 95.8%, 94.5%, and 96.3%, respectively. The experimental results of this system have a specific practical significance in driver-distracted driving warnings.

Towards evaluation of the PID criteria based UAVs observation and tracking head within resizable selection by COA algorithm

The study of unmanned systems has garnered significant academic attention worldwide. The advancements in tracking technologies, particularly the unmanned aerial vehicle (UAV) developed by the US Air Force, have motivated scientists to extensively explore this field due to the abundance of available aerial resources. However, the UAV’s effectiveness in motion is influenced by its three fundamental motions: pitch, roll, and yaw. To address this, scientists have been developing optimization algorithms and models to enhance the UAV’s performance. Nonetheless, in practical scenarios, each motion requires specific measurements, and their relative importance varies. This presents several challenges, including the existence of multiple criteria for selecting optimization values, determining the criteria’s significance, and evaluating trade-offs between performance across different optimization cases. Consequently, the selection and evaluation of UAV motion control algorithms becomes complex. This study proposes a novel selection process that employs the cuckoo optimization algorithm (COA) and real measured resizable margins for goal detection instead of traditional fixed-size evaluation criteria. The framework consists of enhancing PID motion gains using the COA algorithm, focusing on altitude and attitude functional PID controllers. The results of the selection process demonstrate the significance of various criteria for different motions. Rigorous evaluation and analysis were conducted to validate the proposed research framework.

European NCAP Driver State Monitoring Protocols: Prevalence of Distraction in Naturalistic Driving.

examine the prevalence of driver distraction in naturalistic driving when implementing European New Car Assessment Program (Euro NCAP)-defined distraction behaviours. The 2023 introduction of Occupant Status monitoring (OSM) into Euro NCAP will accelerate uptake of Driver State Monitoring (DSM). Euro NCAP outlines distraction behaviours that DSM must detect to earn maximum safety points. Distraction behaviour prevalence and driver alerting and intervention frequency have yet to be examined in naturalistic driving. Twenty healthcare workers were provided with an instrumented vehicle for approximately two weeks. Data were continuously monitored with automotive grade DSM during daily work commutes, resulting in 168.8 hours of driver head, eye and gaze tracking. Single long distraction events were the most prevalent, with .89 events/hour. Implementing different thresholds for driving-related and driving-unrelated glance regions impacts alerting rates. Lizard glances (primarily gaze movement) occurred more frequently than owl glances (primarily head movement). Visual time-sharing events occurred at a rate of .21 events/hour. Euro NCAP-described driver distraction occurs naturalistically. Lizard glances, requiring gaze tracking, occurred in high frequency relative to owl glances, which only require head tracking, indicating that less sophisticated DSM will miss a substantial amount of distraction events. This work informs OEMs, DSM manufacturers and regulators of the expected alerting rate of Euro NCAP defined distraction behaviours. Alerting rates will vary with protocol implementation, technology capability, and HMI strategies adopted by the OEMs, in turn impacting safety outcomes, user experience and acceptance of DSM technology.

Correction of motion tracking errors for PET head rigid motion correction

Objective. In positron emission tomography (PET) rigid motion correction, erroneous tracking information translates into reduced quality in motion corrected reconstructions. We aim to improve the accuracy of the motion tracking data, to improve the quality of motion corrected reconstructions. Approach. We developed a method for correction of marker/skin displacement over the skull, for tracking methods which require multiple markers attached on the subject head. Additionally, we correct for small magnitude (∼1–2 mm) residual translation tracking errors that can still be present after other corrections. We performed [18F]FDG scans in awake mice (n = 8) and rats (n = 8), and dynamic [18F]SynVesT-1 scans in awake mice (n = 8). Head tracking was performed with the point source tracking method, attaching 3–4 radioactive fiducial markers on the animals’ heads. List-mode even-by-event motion correction reconstruction was performed using tracking data obtained from the point source tracking method (MC), tracking data corrected for marker displacement (MC-DC), and tracking data with additional correction for residual translation tracking errors (MC-DCT). Image contrast, and the image enhancement metric (IEM, with MC as reference) were calculated in these 3 reconstructions. Main results. In mice [18F]FDG scans, the contrast increased on average 3% from MC to MC-DC (IEM: 1.01), and 5% from MC to MC-DCT (IEM: 1.02). For mice [18F]SynVesT-1 scans the contrast increased 6% from MC to MC-DC (IEM: 1.03), and 7% from MC to MC-DCT (IEM: 1.05). In rat [18F]FDG scans contrast increased 5% (IEM: 1.04), and 9% (IEM: 1.05), respectively. Significance. The methods presented here serve to correct motion tracking errors in PET brain scans, which translates into improved image quality in motion corrected reconstructions.

Free gaze: co-recording of eye and head tracking with EEG to understand unconstrained vision

Free gaze: co-recording of eye and head tracking with EEG to understand unconstrained vision

Real-time fetal brain tracking for functional fetal MRI.

To improve motion robustness of functional fetal MRI scans by developing an intrinsic real-time motion correction method. MRI provides an ideal tool to characterize fetal brain development and growth. It is, however, a relatively slow imaging technique and therefore extremely susceptible to subject motion, particularly in functional MRI experiments acquiring multiple Echo-Planar-Imaging-based repetitions, for example, diffusion MRI or blood-oxygen-level-dependency MRI. A 3D UNet was trained on 125 fetal datasets to track the fetal brain position in each repetition of the scan in real time. This tracking, inserted into a Gadgetron pipeline on a clinical scanner, allows updating the position of the field of view in a modified echo-planar imaging sequence. The method was evaluated in real-time in controlled-motion phantom experiments and ten fetal MR studies (17 + 4-34 + 3 gestational weeks) at 3T. The localization network was additionally tested retrospectively on 29 low-field (0.55T) datasets. Our method achieved real-time fetal head tracking and prospective correction of the acquisition geometry. Localization performance achieved Dice scores of 84.4% and 82.3%, respectively for both the unseen 1.5T/3T and 0.55T fetal data, with values higher for cephalic fetuses and increasing with gestational age. Our technique was able to follow the fetal brain even for fetuses under 18 weeks GA in real-time at 3T and was successfully applied "offline" to new cohorts on 0.55T. Next, it will be deployed to other modalities such as fetal diffusion MRI and to cohorts of pregnant participants diagnosed with pregnancy complications, for example, pre-eclampsia and congenital heart disease.

Snapture—a Novel Neural Architecture for Combined Static and Dynamic Hand Gesture Recognition

As robots are expected to get more involved in people’s everyday lives, frameworks that enable intuitive user interfaces are in demand. Hand gesture recognition systems provide a natural way of communication and, thus, are an integral part of seamless human-robot interaction (HRI). Recent years have witnessed an immense evolution of computational models powered by deep learning. However, state-of-the-art models fall short of expanding across different gesture domains, such as emblems and co-speech. In this paper, we propose a novel hybrid hand gesture recognition system. Our Snapture architecture enables learning both static and dynamic gestures: by capturing a so-called snapshot of the gesture performance at its peak, we integrate the hand pose and the dynamic movement. Moreover, we present a method for analyzing the motion profile of a gesture to uncover its dynamic characteristics, which allows regulating a static channel based on the amount of motion. Our evaluation demonstrates the superiority of our approach on two gesture benchmarks compared to a state-of-the-art CNNLSTM baseline. Our analysis on a gesture class basis unveils the potential of our Snapture architecture for performance improvements using RGB data. Thanks to its modular implementation, our framework allows the integration of other multimodal data, like facial expressions and head tracking, which are essential cues in HRI scenarios, into one architecture. Thus, our work contributes both to integrative gesture recognition research and machine learning applications for non-verbal communication with robots.

On Rotation Gains Within and Beyond Perceptual Limitations for Seated VR.

Head tracking in head-mounted displays (HMDs) enables users to explore a 360-degree virtual scene with free head movements. However, for seated use of HMDs such as users sitting on a chair or a couch, physically turning around 360-degree is not possible. Redirection techniques decouple tracked physical motion and virtual motion, allowing users to explore virtual environments with more flexibility. In seated situations with only head movements available, the difference of stimulus might cause the detection thresholds of rotation gains to differ from that of redirected walking. Therefore we present an experiment with a two-alternative forced-choice (2AFC) design to compare the thresholds for seated and standing situations. Results indicate that users are unable to discriminate rotation gains between 0.89 and 1.28, a smaller range compared to the standing condition. We further treated head amplification as an interaction technique and found that a gain of 2.5, though not a hard threshold, was near the largest gain that users consider applicable. Overall, our work aims to better understand human perception of rotation gains in seated VR and the results provide guidance for future design choices of its applications.

Predictive saccades and decision making in the beetle-predating saffron robber fly.

Internal predictions about the sensory consequences of self-motion, encoded by corollary discharge, are ubiquitous in the animal kingdom, including for fruit flies, dragonflies, and humans. In contrast, predicting the future location of an independently moving external target requires an internal model. With the use of internal models for predictive gaze control, vertebrate predatory species compensate for their sluggish visual systems and long sensorimotor latencies. This ability is crucial for the timely and accurate decisions that underpin a successful attack. Here, we directly demonstrate that the robber fly Laphria saffrana, a specialized beetle predator, also uses predictive gaze control when head tracking potential prey. Laphria uses this predictive ability to perform the difficult categorization and perceptual decision task of differentiating a beetle from other flying insects with a low spatial resolution retina. Specifically, we show that (1) this predictive behavior is part of a saccade-and-fixate strategy, (2) the relative target angular position and velocity, acquired during fixation, inform the subsequent predictive saccade, and (3) the predictive saccade provides Laphria with additional fixation time to sample the frequency of the prey's specular wing reflections. We also demonstrate that Laphria uses such wing reflections as a proxy for the wingbeat frequency of the potential prey and that consecutively flashing LEDs to produce apparent motion elicits attacks when the LED flicker frequency matches that of the beetle's wingbeat cycle.

Self-organizing neural network for reproducing human postural mode alternation through deep reinforcement learning

A self-organized phenomenon in postural coordination is essential for understanding the auto-switching mechanism of in-phase and anti-phase postural coordination modes during standing and related supra-postural activities. Previously, a model-based approach was proposed to reproduce such self-organized phenomenon. However, if we set this problem including the process of how we establish the internal predictive model in our central nervous system, the learning process is critical to be considered for establishing a neural network for managing adaptive postural control. Particularly when body characteristics may change due to growth or aging or are initially unknown for infants, a learning capability can improve the hyper-adaptivity of human motor control for maintaining postural stability and saving energy in daily living. This study attempted to generate a self-organizing neural network that can adaptively coordinate the postural mode without assuming a prior body model regarding body dynamics and kinematics. Postural coordination modes are reproduced in head-target tracking tasks through a deep reinforcement learning algorithm. The transitions between the postural coordination types, i.e. in-phase and anti-phase coordination modes, could be reproduced by changing the task condition of the head tracking target, by changing the frequencies of the moving target. These modes are considered emergent phenomena existing in human head tracking tasks. Various evaluation indices, such as correlation, and relative phase of hip and ankle joint, are analyzed to verify the self-organizing neural network performance to produce the postural coordination transition between the in-phase and anti-phase modes. In addition, after learning, the neural network can also adapt to continuous task condition changes and even to unlearned body mass conditions keeping consistent in-phase and anti-phase mode alternation.

Playing With Others Using Headphones: Musicians Prefer Binaural Audio With Head Tracking Over Stereo

Immersive listening systems have grown significantly over the past decade and are now an established area of scientific, artistic, and industrial research. However, scarce research has been conducted on musicians' preferences for playing through headphones over binaural spatialization systems with the addition of head tracking, as opposed to classical stereophonic systems. This comparison is essential to optimally support the playing experience with others for cases of remote collaborative playing, individual instrumental practice, individual recreational music-making using backing tracks, and studio recording sessions. In this article, we study the preferences of playing musicians for a stereophonic system versus a binaural head-tracking system composed of Ambisonics technology and binaural synthesis with generalized head-related transfer functions. We conducted two experiments, each with 30 expert musicians, where participants were asked to rate and compare the 2 listening conditions while playing their instrument either seated or standing. Overall, the quantitative and qualitative results indicated a generalized preference for the binaural system with head tracking over the stereophonic system, with higher ratings for localization, immersion, social presence, realism, and connection with other musicians. Moreover, participants moved their heads significantly more in the binaural conditions. This phenomenon may be explained by the higher engagement and arousal due to the improved auditory experience, or alternatively by the presence of embodied music cognition mechanisms that cause a higher degree of exploration to better understand the action–perception loop. These findings highlight the need for progressing current commercial hardware and software systems used by musicians while playing over headphones.

Virtual reality and machine learning for predicting visual attention in a daylit exhibition space: A proof of concept

Lighting features act as driving forces that control visual attention and perception. VR can analyse the relationship between human behavior and environment stimuli, while machine learning (ML) can exploit these data to develop a predictive model. This paper aims to develop an approach for predicting visual attention using VR and ML algorithms. VR was applied to experiment 4 virtual 360 scenes of a generic daylit exhibition space, and 41 participants were recruited. Measurements of head tracking were investigated to capture the areas of interest (AoI) within the space under different daylight conditions. Features were extracted to train the ML models through luminance, spatial contrast, and center bias. Results showed the potential of ML for predicting the visual behaviour. When comparing predicted with measured data, the accuracy reached 71 % using ensemble bagged trees algorithm. This study acts as a proof-of-concept for predicting visual attention using ML algorithms, highlighting the potentials of ML and VR for adopting human-centric design. Thus, it allows architects and lighting designers to promote more interesting and interacting visual experience.

Quantifying MR head motion in the Rhineland Study – A robust method for population cohorts

Head motion during MR acquisition reduces image quality and has been shown to bias neuromorphometric analysis. The quantification of head motion, therefore, has both neuroscientific as well as clinical applications, for example, to control for motion in statistical analyses of brain morphology, or as a variable of interest in neurological studies. The accuracy of markerless optical head tracking, however, is largely unexplored. Furthermore, no quantitative analysis of head motion in a general, mostly healthy population cohort exists thus far. In this work, we present a robust registration method for the alignment of depth camera data that sensitively estimates even small head movements of compliant participants. Our method outperforms the vendor-supplied method in three validation experiments: 1. similarity to fMRI motion traces as a low-frequency reference, 2. recovery of the independently acquired breathing signal as a high-frequency reference, and 3. correlation with image-based quality metrics in structural T1-weighted MRI. In addition to the core algorithm, we establish an analysis pipeline that computes average motion scores per time interval or per sequence for inclusion in downstream analyses. We apply the pipeline in the Rhineland Study, a large population cohort study, where we replicate age and body mass index (BMI) as motion correlates and show that head motion significantly increases over the duration of the scan session. We observe weak, yet significant interactions between this within-session increase and age, BMI, and sex. High correlations between fMRI and camera-based motion scores of proceeding sequences further suggest that fMRI motion estimates can be used as a surrogate score in the absence of better measures to control for motion in statistical analyses.