Head Orientation Research Articles

Toward Robust and Unconstrained Full Range of Rotation Head Pose Estimation.

Estimating the head pose of a person is a crucial problem for numerous applications that is yet mainly addressed as a subtask of frontal pose prediction. We present a novel method for unconstrained end-to-end head pose estimation to tackle the challenging task of full range of orientation head pose prediction. We address the issue of ambiguous rotation labels by introducing the rotation matrix formalism for our ground truth data and propose a continuous 6D rotation matrix representation for efficient and robust direct regression. This allows to efficiently learn full rotation appearance and to overcome the limitations of the current state-of-the-art. Together with new accumulated training data that provides full head pose rotation data and a geodesic loss approach for stable learning, we design an advanced model that is able to predict an extended range of head orientations. An extensive evaluation on public datasets demonstrates that our method significantly outperforms other state-of-the-art methods in an efficient and robust manner, while its advanced prediction range allows the expansion of the application area. We open-source our training and testing code along with our trained models: https://github.com/thohemp/6DRepNet360.

Body and Head Orientation Estimation From Low-Resolution Point Clouds in Surveillance Settings

Body and Head Orientation Estimation From Low-Resolution Point Clouds in Surveillance Settings

The sense-making of social motor competence in autism spectrum disorder

Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterised by deficits in social motor development. Social motor competence plays a vital role in social communication, especially in face-to-face communication. It's an effective way to convey language and emotions so that we are more easily able to understand each other. However, many individuals with ASD demonstrate pervasive deficits in social motor skills. Moreover, the disintegration of the sense-making about social motor competence also affects the coordination between actions. They show sense-making deficits in facial actions, such as gaze direction, gaze following, and mutual gaze. Besides, they also fail to use pointing gestures, head orientation, and body postures to infer other's goals, desires, or interests. This is also their special way of sense-making. Finally, based on the "Qi" theory, we suggest that treatments such as Tai Chi, Wu Qin Xi, and Nei Yang Gong could be more widely used in correcting the social motor competence of individuals with ASD and enhance their social abilities, so they can participate in making sense of others.

Modeling and Classification of the Behavioral Patterns of Students Participating in Online Examination

Online education has become an essential part of the modern education system, but keeping the integrity of the online examination remains a challenge. A significant increase in cheating in online examinations (from 29.9% before COVID-19 to 54.7% during COVID-19, as per a recent survey) points out the necessity of online exam proctoring systems. Traditionally, educational institutes utilize different questions in onsite exams: multiple-choice questions (MCQs), analytical questions, descriptive questions, etc. For online exams, form-based exams using MCQs are popular though in disciplines like math, engineering, architecture, art, or other courses, paper and pen tests are typical for proper assessment. In form-based exams, students’ attention is toward display devices, and cheating behavior is identified as the deviation of head and eye gaze direction from the display device. In paper- and pen-based exams, students’ main attention is on the answer script not on the device. Identifying cheating behavior in such exams is not a trivial task since complex body movements need to be observed to identify cheating. Previous research works focused on the deviation of the head and eyes from the screen which is more suited for form-based exams. Most of them are very resource-intensive; along with a webcam, they require additional hardware such as sensors, microphones, and security cameras. In this work, we propose an automated proctoring solution for paper- and pen-based online exams considering specific requirements of pen-and-paper exams. Our approach tracks head and eye orientations and lip movements in each frame and defines the movement as the change of orientation. We relate cheating with frequent coordinated movements of the head, eyes, and lips. We calculate a cheating score indicative of the frequency of movements. A case is marked as a cheating case if the cheating score is higher than the proctor-defined threshold (which may vary depending on the specific requirement of the discipline). The proposed system has five major parts: (1) identification and coordinate extraction of selected facial landmarks using MediaPipe; (2) orientation classification of the head, eye, and lips with K-NN classifier, based on the landmarks; (3) identification of abnormal movements; (4) calculation of a cheating score based on abnormal movement patterns; and (5) a visual representation of students’ behavior to support the proctor for early intervention. Our system is robust since it observes the pattern of movement over a sequence of frames and considers the coordinated movement pattern of the head, eye, and lips rather than considering a single deviation as a cheating behavior which will minimize the false positive cases. Visualization of the student behavior is another strength of our system that enables the human proctor to take preventive measures rather than punishing the student for the final cheating score. We collected video data with the help of 16 student volunteers from the authors’ university who participated in the two well-instructed mock exams: one with cheating and another without cheating. We achieved 100% accuracy in detecting noncheating cases and 87.5% accuracy for cheating cases when the threshold was set to 40.

Head-centric computing for vestibular stimulation under head-free conditions.

Background: The vestibular end organs (semicircular canals, saccule and utricle) monitor head orientation and motion. Vestibular stimulation by means of controlled translations, rotations or tilts of the head represents a routine manoeuvre to test the vestibular apparatus in a laboratory or clinical setting. In diagnostics, it is used to assess oculomotor postural or perceptual responses, whose abnormalities can reveal subclinical vestibular dysfunctions due to pathology, aging or drugs. Objective: The assessment of the vestibular function requires the alignment of the motion stimuli as close as possible with reference axes of the head, for instance the cardinal axes naso-occipital, interaural, cranio-caudal. This is often achieved by using a head restraint, such as a helmet or strap holding the head tightly in a predefined posture that guarantees the alignment described above. However, such restraints may be quite uncomfortable, especially for elderly or claustrophobic patients. Moreover, it might be desirable to test the vestibular function under the more natural conditions in which the head is free to move, as when subjects are tracking a visual target or they are standing erect on the moving platform. Here, we document algorithms that allow delivering motion stimuli aligned with head-fixed axes under head-free conditions. Methods: We implemented and validated these algorithms using a MOOG-6DOF motion platform in two different conditions. 1) The participant kept the head in a resting, fully unrestrained posture, while inter-aural, naso-occipital or cranio-caudal translations were applied. 2) The participant moved the head continuously while a naso-occipital translation was applied. Head and platform motion were monitored in real-time using Vicon. Results: The results for both conditions showed excellent agreement between the theoretical spatio-temporal profile of the motion stimuli and the corresponding profile of actual motion as measured in real-time. Conclusion: We propose our approach as a safe, non-intrusive method to test the vestibular system under the natural head-free conditions required by the experiential perspective of the patients.

Ensemble coding of facial identity is robust, but may not contribute to face learning

Ensemble coding – the rapid extraction of a perceptual average – has been proposed as a potential mechanism underlying face learning. We tested this proposal across five pre-registered experiments in which four ambient images of an identity were presented in the study phase. In Experiments 1 and 2a-c, participants were asked whether a test image was in the study array; these experiments examined the robustness of ensemble coding. Experiment 1 replicated ensemble coding in an online sample; participants recognize images from the study array and the average of those images. Experiments 2a-c provide evidence that ensemble coding meets several criteria of a possible learning mechanism: It is robust to changes in head orientation (± 60o), survives a short (30s) delay, and persists when images of two identities are interleaved during the study phase. Experiment 3 examined whether ensemble coding is sufficient for face learning (i.e., facilitates recognition of novel images of a target identity). Each study array comprised four ambient images (variability + average), a single image, or an average of four images (average only). Participants were asked whether a novel test image showed the identity from a study array. Performance was best in the four-image condition, with no difference between the single-image and average-only conditions. We conclude that ensemble coding of facial identity is robust but that the perceptual average per se is not sufficient for face learning.

Spatial navigation performance differentiates Alzheimer´s disease biomarker positive and biomarker negative cognitively impaired older adults

AbstractBackgroundSpatial navigation deficits including egocentric (body‐centered) and allocentric (world‐centered) navigation impairment are present early in Alzheimer´s disease (AD). We evaluated a potential of a realistic‐looking spatial navigation task to differentiate older adults with amnestic mild cognitive impairment (aMCI) who have positive AD biomarkers (AD aMCI) from those with negative AD biomarkers (non‐AD aMCI).MethodParticipants with AD aMCI (n = 24), non‐AD aMCI (n = 25), mild AD dementia (n = 14) and cognitively normal (CN) older adults (n = 35) underwent spatial navigation assessment in the virtual Supermarket task. Participants passively travelled through the supermarket and followed different routes to reach specific locations. In the final location, the participants had to indicate their starting location (egocentric heading) and their current location and final heading orientation on an aerial spatial map of the supermarket (distance from the correct location and allocentric heading, respectively, both measures of allocentric navigation).ResultIn the egocentric heading, the AD aMCI and mild AD dementia groups indicated their original starting location less accurately than the CN group (p<0.001) and with similar accuracy as the non‐AD aMCI group (p> = 0.219). The non‐AD aMCI group did not significantly differ from the CN group (p = 0.055). In the first allocentric task (distance from the correct location), the AD aMCI group and mild AD dementia groups indicated their positions on a paper aerial map of the supermarket with greater distance errors compared to the non‐AD aMCI group (p< = 0.013) and CN (p<0.001) groups. The non‐AD aMCI group had comparable performance to the CN group (p = 0.172). In the second allocentric task (allocentric heading), both the AD aMCI and mild AD dementia groups made more errors indicating final heading orientation than the non‐AD aMCI (p< = 0.040) and the CN (p<0.001) groups. The non‐AD aMCI group had worse performance than the CN group (p = 0.004). The sensitivity and specificity for discriminating AD aMCI and non‐AD aMCI was 72% and 71%, respectively.ConclusionBoth, egocentric and allocentric spatial navigation tasks detected spatial navigation impairment in older adults with AD aMCI. However, only the allocentric tasks differentiated AD aMCI from non‐AD aMCI. The virtual Supermarket task has a potential to become a screening tool for early AD‐related spatial navigation deficits.

Combining Mindwave, MPU6050, Ultrasonic, and Internet of Things for a reliable, safe, and monitored wheelchair control system

This paper proposes the development of a wheelchair control system by combining a Mindwave EEG (electroencephalogram) sensor, an MPU6050 inertial sensor, an ultrasonic sensor, and IoT technology. Stroke survivors often face challenges in controlling their wheelchairs due to cognitive and physical impairments, requiring specific solutions to ensure their safety and comfort. The integration of the Mindwave EEG sensor captures the user's cognitive state, such as level of attention and focus, while the MPU6050's inertial sensor monitors physical movement and changes in head orientation. Additionally, the ultrasonic sensor is incorporated to detect obstacles and provide proximity information for safe navigation. This wheelchair control system is also linked with IoT technology to enable monitoring of the user's cognitive, physical, and environmental condition and remote assistance. In this paper, to implement this wheelchair control system, a T-BEAM with an ESP32 controller is used, which is equipped with GPS to provide wheelchair location data and LORA radio communication for remote data transmission, and a TTGO LORA32 to forward the data to Arduino IoT Cloud service over Wi-Fi. The results of the development show that the wheelchair control system is safer and more monitored, and users are easier and more flexible in operating wheelchairs.

Exploration of actual sky polarization patterns: From influencing factor analyses to polarized light-aided navigation

The polarized light scattered by the sunlight and atmosphere is a general natural property of the atmospheric space of Earth. This study is devoted to an exploration of actual sky polarization pattern-oriented navigation. For the polarized light-aided VINS (abbreviated ‘visual-inertial navigation system’), one of our major concerns is to prove the heading (orientation) reference stays stable under the actual complex atmospheric environments. The influencing factors that contribute to the variation of the polarization pattern symmetry have been taken into account. The analytic AOP (abbreviated ‘angle of polarization’) and DOP (abbreviated ‘degree of polarization’) maps and quantitative analyses that concern spectrum character, cloudage, and aerosol optical depth relevant to light scattering effects are simultaneously presented. Given the globally referenced heading, one intuitive way of modeling multi-sensor fusion is to use a graph structure. Following the pipeline design of VINS Fusion, a polarization imaging factor constraint (expressed by the polarization measurement residual-encoded edges) is added in the state estimator and the cost function to be minimized during optimization is formulated. Via campus road experiments on the built robot platform, our proposal is compared against VINS Fusion under different weather conditions, revealing that over the entire closed path, our polarized light-aided VINS achieves high rate of both locally and globally consistent estimates. To our best knowledge, this is the first work to illuminate the insights into a practical, polarization imager-integrated navigating application in which the localization results can be optically explained, evaluated, and optimized.

Difference in the Static Postural Control according to the Subjective Visual Vertical Deviation and Head Orientations

Difference in the Static Postural Control according to the Subjective Visual Vertical Deviation and Head Orientations

Computerized analysis of facial expression reveals objective indices of blunted facial affect.

Blunted affect is associated with severe mental illness, particularly schizophrenia. Mechanisms of blunted affect are poorly understood, potentially due to a lack of phenomenological clarity. Here, we examine clinician rated blunted affect and computerized facial metrics derived from ambulatory video assessment using machine learning. With high predictive accuracy (80-82%), we found that head orientation, eye movement, and facets of mouth movement were associated with clinical ratings of blunted affect. Features denoting larger muscle movements were associated with social cognition (R2 = 0.37) and cognition (R2 = 0.40). Findings provide potential insights on psychological and pathophysiological contributors to blunted affect.

How Do Background and Remote User Representations Affect Social Telepresence in Remote Collaboration?: A Study with Portal Display, a Head Pose-Responsive Video Teleconferencing System

This study presents Portal Display, a screen-based telepresence system that mediates the interaction between two distinct spaces, each using a single display system. The system synchronizes the users’ viewpoint with their head position and orientation to provide stereoscopic vision through this single monitor. This research evaluates the impact of graphically rendered and video-streamed backgrounds and remote user representations on social telepresence, usability, and concentration during conversations and collaborative tasks. Our results indicate that the type of background has a negligible impact on these metrics. However, point cloud streaming of remote users significantly improves social telepresence, usability, and concentration compared with graphical avatars. This study implies that Portal Display can operate more efficiently by substituting the background with graphical rendering and focusing on higher-resolution 3D point cloud streaming for narrower regions for remote user representations. This configuration may be especially advantageous for applications where the remote user’s background is not essential to the task, potentially enhancing social telepresence.

The interaction between talker head orientation, spatial separation, and extended high frequencies for speech-in-speech recognition

Speech-in-speech recognition experiments generally involve presentation of stimuli as if target and masker talkers are facing the listener. This is because speech signals are recorded with microphones located in front of the talkers. In real-world situations, masker talkers are often rotated away from the listener, facing their own conversational partners. This head orientation mismatch between the target and masker talkers provides cues that enhance speech recognition, including cues at extended high frequencies (EHFs; >8 kHz) due to the directional nature of EHFs in speech radiation. However, it is unclear how head orientation and EHF cues affect speech recognition when target and masker speech come from different spatial locations, as is the case in realistic multi-talker environments. In the present study, we investigated the EHF benefit in a complex auditory scene involving differences in target and masker head orientation and spatial location. Masker head orientation was either 0° (facing) or 90° (non-facing). Target speech location was 0° azimuth, while masker speech location was either 0° or ±45° azimuth. Preliminary analyses suggest spatial cues provide greater benefit than head orientation cues, and that the EHF benefit is diminished when talkers are spatially separated. [Work supported by NIH grant R01DC019745.]

The impact of head orientation with respect to B0 on diffusion tensor MRI measures.

Diffusion tensor MRI (DT-MRI) remains the most commonly used approach to characterise white matter (WM) anisotropy. However, DT estimates may be affected by tissue orientation w.r.t. due to local gradients and intrinsic orientation dependence induced by the microstructure. This work aimed to investigate whether and how diffusion tensor MRI-derived measures depend on the orientation of the head with respect to the static magnetic field, . By simulating WM as two compartments, we demonstrated that compartmental anisotropy can induce the dependence of diffusion tensor measures on the angle between WM fibres and the magnetic field. In in vivo experiments, reduced radial diffusivity and increased axial diffusivity were observed in white matter fibres perpendicular to compared to those parallel to . Fractional anisotropy varied by up to as a function of the angle between WM fibres and the orientation of the main magnetic field. To conclude, fibre orientation w.r.t. is responsible for up to variance in diffusion tensor measures across the whole brain white matter from all subjects and head orientations. Fibre orientation w.r.t. may introduce additional variance in clinical research studies using diffusion tensor imaging, particularly when it is difficult to control for (e.g., fetal or neonatal imaging, or when the trajectories of fibres change due to, e.g., space occupying lesions).

Influence of the Human Field of View on Visual and Non-Visual Quantities in Indoor Environments.

The visual and non-visual effectiveness of light is often determined by measuring the spectrally weighed irradiance on the corneal plane. This is typically achieved using spectral irradiance or illuminance measurements, captured in a hemispheric (2π) geometry with a diffuser. However, the human binocular field of view (FOV) is not a perfect hemisphere, as it is occluded both upward and downward. Previous research on FOV-restricted measurements is limited, leaving the error from using hemispheric measurements for non-visual quantities undefined. In our study, we tackled this issue by designing and 3D printing FOV occlusions as attachments to spectral measurement devices. We took measurements with and without the occlusion in various laboratory (light from different directions) and real-world lighting situations (light typically from above). Our findings reveal a reduction of visual and melanopic values due to the FOV occlusion. These ranged from negligible to more than 60% in realistic scenarios. Interestingly, the reduction was consistent for both visual and melanopic parameters, as the distribution of light in the FOV was generally spectrally homogeneous. An exception occurred in a specific artificial laboratory situation, where the melanopic daylight (D65) efficacy ratio changed by more than a factor of 2 solely because of the FOV occlusion. Additionally, we observed that head orientation had a marked effect on all quantities measured. In conclusion, our results highlight the potential for substantial errors when solely relying on vertical, hemispheric measurements in experiments and non-visual lighting design projects. We encourage the (additional) use of FOV occlusion in eye-level measurements for typical viewing directions, and we are providing open-source 3D-print files to facilitate this practice.

Multi-Point Fiber-Optic Distance Sensor for Endoscopic Surgery Monitoring

We present a sub-mm diameter multi-point endoscopic sensor for simultaneous measurement of the minimum distance and orientation of the sensor head with respect to a flat tissue surface. The approach is based on frequency modulated continuous wave (FMCW) LIDAR, which is a coherent measurement method immune to stray light interference. As a swept source, a commercially available 850 nm VCSEL is wavelength modulated by self-heating to achieve a tuning range of 4.1 nm at up to 10 kHz repetition rate. The resulting interferograms are calibrated in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -space using a reference Mach-Zender interferometer. Multiple measurement beams are split at the sensor tip using a 3D nano-printed microprism array, which achieves this refractively for higher collection efficiency and a larger measurement range compared to a diffractive element. Preliminary experiments under controlled conditions validate the sensor’s performance.

On the Dynamic Mechanism of Long-Flexible Fatty Acid Binding to Fatty Acid Binding Protein: Resolving the Long-Standing Debate.

Fatty acids (FAs) are one of the essential energy sources for physiological processes, and they play a vital role in regulating immune and inflammatory responses, promoting cell differentiation and apoptosis, and inhibiting tumor growth. These functions are carried out by FA binding proteins (FABPs) that recognize and transport FAs. Although the crystal structure of the FA-FABPs complex has long been characterized, the mechanism behind FA binding and dissociation from FABP remains unclear. This study employed conventional MD simulations and enhanced sampling technologies to investigate the atomic-scale complexes of heart fatty acid binding proteins and stearic acid (SA). The results revealed two primary pathways for the binding or dissociation of the flexible long-chain ligand, with the orientation of the SA carboxyl head during dissociation determining the chosen path. Conformational changes in the portal region of FABP during the ligand binding/unbinding were found to be trivial, and the overturn of the ″cap″ or the unfolding of the α2 helix was not required. This study resolves the long-standing debate on the binding mechanism of SA with the long-flexible tail to FABP, which significantly improves the understanding of the transport mechanism of FABPs and the development of related therapeutic agents.

Behavior-Aware Pedestrian Trajectory Prediction in Ego-Centric Camera Views with Spatio-Temporal Ego-Motion Estimation

With the ongoing development of automated driving systems, the crucial task of predicting pedestrian behavior is attracting growing attention. The prediction of future pedestrian trajectories from the ego-vehicle camera perspective is particularly challenging due to the dynamically changing scene. Therefore, we present Behavior-Aware Pedestrian Trajectory Prediction (BA-PTP), a novel approach to pedestrian trajectory prediction for ego-centric camera views. It incorporates behavioral features extracted from real-world traffic scene observations such as the body and head orientation of pedestrians, as well as their pose, in addition to positional information from body and head bounding boxes. For each input modality, we employed independent encoding streams that are combined through a modality attention mechanism. To account for the ego-motion of the camera in an ego-centric view, we introduced Spatio-Temporal Ego-Motion Module (STEMM), a novel approach to ego-motion prediction. Compared to the related works, it utilizes spatial goal points of the ego-vehicle that are sampled from its intended route. We experimentally validated the effectiveness of our approach using two datasets for pedestrian behavior prediction in urban traffic scenes. Based on ablation studies, we show the advantages of incorporating different behavioral features for pedestrian trajectory prediction in the image plane. Moreover, we demonstrate the benefit of integrating STEMM into our pedestrian trajectory prediction method, BA-PTP. BA-PTP achieves state-of-the-art performance on the PIE dataset, outperforming prior work by 7% in MSE-1.5 s and CMSE as well as 9% in CFMSE.

Responses of wild skuas (Catharacta antarctica ssp. lonnbergi) to human cues in cooperative and competitive social contexts.

Many animals respond to and use social cues emitted by other species (e.g., head direction). In the context of human-animal communication, these capacities have been attributed to regular and longstanding exposure to humans. We presented wild brown skuas (Catharacta antarctica ssp. lonnbergi) with two versions of an object-choice paradigm. In the cooperative version (Experiment 1), one human experimenter provided a simple and salient cue indicating which of two containers covered a food reward. The cues administered consisted of touching, looking at, pointing at, or pointing and looking at the container hiding food. In Experiment 1, skuas could thus cooperate with an experimenter by using the cues provided to locate the rewarded container. In the competitive version (Experiment 2), two human experimenters presented a platform with a visible food reward. In six experimental conditions, we varied experimenters' body orientation, head orientation, eye-gaze direction, face occlusion, and mouth occlusion, as well as the platform's location, ensuring that in each case only one experimenter had visual access to the rewarded platform. Here, birds could compete with the experimenters by robbing the human who does not see the food. Skuas failed to use human-given cues spontaneously in Experiment 1, and took the reward regardless of whether the experimenters could see in Experiment 2. Our results contrast with those obtained on other wild birds with pre-experience with humans. Hopefully, our findings will stimulate further research in order to illuminate the potential role of such experience in the capacity to respond to and use human-given cues. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Is the processing of facial expression and head orientation dissociated in the human brain?

Is the processing of facial expression and head orientation dissociated in the human brain?