Head Gestures Research Articles

Gesture centric interaction: evaluating hand and head gestures in touchless cursor control

Touchless interfaces have gained considerable importance in the modern era, particularly due to their user-friendly and hygienic nature of interaction. This article presents the designing of two touchless cursor control systems based on hand gestures and head movements utilising the MediaPipe framework to extract the key landmarks of the hand and face utilising a laptop camera. The index finger’s landmark points are tracked and converted to corresponding screen coordinates for cursor control. Similarly, yaw and pitch angles of head movements are computed in the head movement-based cursor design. A comprehensive performance evaluation of the two proposed systems based on a two-dimensional (2D) Fitts’ law experiment revealed superior performance for the hand gesture-controlled cursor compared to the head movement-controlled cursor with throughputs of 0.59 bps and 0.53 bps respectively. Participants also favoured hand gesture-based cursor control over head movement-based cursor control in terms of overall experience and task difficulty.

FacePsy: An Open-Source Affective Mobile Sensing System - Analyzing Facial Behavior and Head Gesture for Depression Detection in Naturalistic Settings

Depression, a prevalent and complex mental health issue affecting millions worldwide, presents significant challenges for detection and monitoring. While facial expressions have shown promise in laboratory settings for identifying depression, their potential in real-world applications remains largely unexplored due to the difficulties in developing efficient mobile systems. In this study, we aim to introduce FacePsy, an open-source mobile sensing system designed to capture affective inferences by analyzing sophisticated features and generating real-time data on facial behavior landmarks, eye movements, and head gestures - all within the naturalistic context of smartphone usage with 25 participants. Through rigorous development, testing, and optimization, we identified eye-open states, head gestures, smile expressions, and specific Action Units (2, 6, 7, 12, 15, and 17) as significant indicators of depressive episodes (AUROC=81%). Our regression model predicting PHQ-9 scores achieved moderate accuracy, with a Mean Absolute Error of 3.08. Our findings offer valuable insights and implications for enhancing deployable and usable mobile affective sensing systems, ultimately improving mental health monitoring, prediction, and just-in-time adaptive interventions for researchers and developers in healthcare.

S3: Speech, Script and Scene driven Head and Eye Animation

We present S 3 , a novel approach to generating expressive, animator-centric 3D head and eye animation of characters in conversation. Given speech audio, a Directorial script and a cinematographic 3D scene as input, we automatically output the animated 3D rotation of each character's head and eyes. S 3 distills animation and psycho-linguistic insights into a novel modular framework for conversational gaze capturing: audio-driven rhythmic head motion; narrative script-driven emblematic head and eye gestures; and gaze trajectories computed from audio-driven gaze focus/aversion and 3D visual scene salience. Our evaluation is four-fold: we quantitatively validate our algorithm against ground truth data and baseline alternatives; we conduct a perceptual study showing our results to compare favourably to prior art; we present examples of animator control and critique of S 3 output; and present a large number of compelling and varied animations of conversational gaze.

Development of a Motion Controller for the Electric Wheelchair of Quadriplegic Patients Using Head Movements Recognition

The robotic wheelchair is designed to allow independent movement for patients with severe musculoskeletal and neuromuscular disorders that interfere with normal wheelchair propulsion by hands. Control of the movement of the wheelchair necessities residual motion of the upper extremity which was not applicable for patients with upper extremity disability as in Quadriplegic patients. Several research efforts have been proposed to help quadriplegic patients independently use the robotic wheelchair. However, the proposed solutions had many problems with accuracy, simplicity, and cost. The current work aims to build an accurate, simple, and low-cost control system for robotic wheelchairs using head movement. The proposed system was based on the use of the MEMs Inertial measurement unit (IMU) to sense the head movement gestures and transfer the gestures into signals to control the wheelchair movements. A new lightweight algorithm for the detection of head gestures was proposed to improve the system’s accuracy. The proposed system has been applied and tested on volunteers. The results revealed that the accuracy of the proposed method in recognizing head movement was 97% on average.

Teknologi Kendali Kamera Fpv 2 DOF Melalui Gesture Kepala Dengan Sensor IMU (Inertial Measurement Unit)

In taking videos or photos while in the air, there are currently very many tools or media that can be used and with the current technology that is very rapidly developing, especially in science, it can do everything easily. In the electronic field, a drone is a device that can fly and be controlled unmanned using a remote control. This object has a designation that is called a UAV (Unmanned Aerial Vehicle). This drone has a variety starting from the shape, size, different characters, and functions. FPV is one of the important parts of the 2DOF FPV (first person view) drone as a head gesture combined with parts such as nano arduino, Blueotooth HC-04, Servo, servo tilt up, IP camera, nano arduino must be connected to bluetooth on the cellphone, servo functions as moving right and left in accordance with the user's head gesture, servo tilt up functions as a gesture from top to bottom according to the gesture of the user and IP camera functions as video streaming, VR Glasses or commonly called Googles are connected using HC-04 bluetooth. According to the function in VR Glasses, it will bring up the sensitivity and movement value on the servo and servo tilt up and display the camera display on the IP camera.

GazeSwitch: Automatic Eye-Head Mode Switching for Optimised Hands-Free Pointing

This paper contributes GazeSwitch, an ML-based technique that optimises the real-time switching between eye and head modes for fast and precise hands-free pointing. GazeSwitch reduces false positives from natural head movements and efficiently detects head gestures for input, resulting in an effective hands-free and adaptive technique for interaction. We conducted two user studies to evaluate its performance and user experience. Comparative analyses with baseline switching techniques, Eye+Head Pinpointing (manual) and BimodalGaze (threshold-based) revealed several trade-offs. We found that GazeSwitch provides a natural and effortless experience but trades off control and stability compared to manual mode switching, and requires less head movement compared to BimodalGaze. This work demonstrates the effectiveness of machine learning approach to learn and adapt to patterns in head movement, allowing us to better leverage the synergistic relation between eye and head input modalities for interaction in mixed and extended reality.

Head Gesture Recognition Combining Activity Detection and Dynamic Time Warping.

The recognition of head movements plays an important role in human-computer interface domains. The data collected with image sensors or inertial measurement unit (IMU) sensors are often used for identifying these types of actions. Compared with image processing methods, a recognition system using an IMU sensor has obvious advantages in terms of complexity, processing speed, and cost. In this paper, an IMU sensor is used to collect head movement data on the legs of glasses, and a new approach for recognizing head movements is proposed by combining activity detection and dynamic time warping (DTW). The activity detection of the time series of head movements is essentially based on the different characteristics exhibited by actions and noises. The DTW method estimates the warp path distances between the time series of the actions and the templates by warping under the time axis. Then, the types of head movements are determined by the minimum of these distances. The results show that a 100% accuracy was achieved in the task of classifying six types of head movements. This method provides a new option for head gesture recognition in current human-computer interfaces.

Drowsiness Detection using Python

The Driver Drowsiness Detection System is a sophisticated safety solution designed to mitigate the risks associated with driver fatigue and drowsiness while operating a vehicle. This project aims to develop an intelligent system capable of monitoring driver behavior and physiological signals in real-time, detecting signs of drowsiness, and issuing timely alerts to prevent accidents. The system utilizes a combination of hardware sensors, including cameras, infrared sensors, EEG sensors, and heart rate monitors, to capture and analyze various indicators of driver drowsiness, such as facial expressions, eye movements, head gestures, body temperature, brainwave patterns, and heart rate variability.

Use and Function of Gestures in Persuasive Business Communication: A Contrastive Study between Spanish and English

This study sheds light on the types and frequencies of kinesic signs used in business pitches by entrepreneurs in Spanish and English, as well as the functions these nonverbal signs fulfil to contribute to the persuasiveness of their presentations. The corpus consists of 20 pitches (10 in each language) from two equivalent TV programs (the reality shows Dragons’ Den and Tu Oportunidad from the UK and Spain, respectively). A specific method of analysis previously developed by Cestero Mancera is used to identify signs in an inductive, qualitative way that allows us to apply a quantitative methodology to distinguish among the different subpopulations of the subjects studied (male and female presenters in Spanish and English presentations). The results show that facial gesturing is the most frequent type used by all presenters, although speakers of each language seem to have diverse preferences for the rest of the signs (head, hand and other body gestures). Kinesic signs apparently display the same pragmatic functions in Spanish and English, although they are more frequently used in Spanish (63.78% of the 784 signs found) than English (36.22%). Gender may be considered an influential factor when observed within the same language but not as a variable; furthermore, gender preferences are reversed when associated with language (i.e., female Spanish presentations display the most signs, while English presentations by females display the least; male presentations are in between these, there being more in Spanish than in English presentations).

AI and augmented reality for 3D Indian dance pose reconstruction cultural revival

This paper delves into the specialized domain of human action recognition, focusing on the Identification of Indian classical dance poses, specifically Bharatanatyam. Within the dance context, a “Karana” embodies a synchronized and harmonious movement encompassing body, hands, and feet, as defined by the Natyashastra. The essence of Karana lies in the amalgamation of nritta hasta (hand movements), sthaana (body postures), and chaari (leg movements). Although numerous, Natyashastra codifies 108 karanas, showcased in the intricate stone carvings adorning the Nataraj temples of Chidambaram, where Lord Shiva’s association with these movements is depicted. Automating pose identification in Bharatanatyam poses challenges due to the vast array of variations, encompassing hand and body postures, mudras (hand gestures), facial expressions, and head gestures. To simplify this intricate task, this research employs image processing and automation techniques. The proposed methodology comprises four stages: acquisition and pre-processing of images involving skeletonization and Data Augmentation techniques, feature extraction from images, classification of dance poses using a deep learning network-based convolution neural network model (InceptionResNetV2), and visualization of 3D models through mesh creation from point clouds. The use of advanced technologies, such as the MediaPipe library for body key point detection and deep learning networks, streamlines the identification process. Data augmentation, a pivotal step, expands small datasets, enhancing the model’s accuracy. The convolution neural network model showcased its effectiveness in accurately recognizing intricate dance movements, paving the way for streamlined analysis and interpretation. This innovative approach not only simplifies the identification of Bharatanatyam poses but also sets a precedent for enhancing accessibility and efficiency for practitioners and researchers in the Indian classical dance.

A Multimodal Approach to Catalan Pragmatic Markers: An Exploratory Study

This exploratory study aims to investigate the co-occurrence of Catalan pragmatic markers that fulfill an interactive function and multimodal cues, such as manual gestures, adaptors, head gestures, and eye gaze. To do so, we utilized spontaneous face-to-face conversations between ten different dyads, all of whom were undergraduate students discussing movies that they had recently seen. We processed the data with descriptive quantitative statistics, specifically employing exploratory multifactorial statistical visualization techniques, including multiple correspondence analysis (MCA) and cluster analysis. The results showed that pragmatic markers generally co-occur with multimodal cues. Although the multimodal system works independently, when pragmatic markers are uttered, both systems tend to harmonize in accordance with general communicative tasks, particularly turn management, alignment, and social affiliation.

Use of a Novel Beyeonics One Three-dimensional Head-mounted Digital Visualization Platform in Vitreoretinal surgeries.

Ophthalmic microscopes have been crucial in visualizing surgical fields, but their limitations in enhancing the surgical view through digital image processing have prompted the development of digital surgical microscopes. The Beyeonics One microscope, a novel digital microscope, offers ophthalmic surgeons a 3D visualization platform and an augmented reality (AR) surgical headset, potentially improving surgical decision-making and outcomes. While its initial use has been described in cataract and corneal surgeries, its application in vitreoretinal surgery remains relatively unexplored. In this interventional case series, we collected data from the medical records of patients who underwent vitreoretinal surgery using the Beyeonics One 3D visualization platform at the Tel Aviv Medical Center. A total of 36 eyes from 36 subjects were included. Surgical techniques included retinal detachment surgeries and macular surgeries, performed by experienced surgeons. The surgical visualization was facilitated by the Beyeonics One 3D head-mounted display (HMD) platform. The procedures were uneventful, and none intra- or postoperative complications were reported, and surgeons did not experience any signal delay in the real-time video. The Beyeonics One microscope offers several potential advantages in vitreoretinal surgery, including digital image processing, enhanced depth perception through the 3D HMD platform, and hands-free image control using head gestures. While this study demonstrates the feasibility and safety of the Beyeonics One microscope, addressing limitations related to hazy views and optimizing image quality are crucial for consistent visualization.

Mining Bodily Cues to Deception

A significant body of research has investigated potential correlates of deception and bodily behavior. The vast majority of these studies consider discrete, subjectively coded bodily movements such as specific hand or head gestures. Such studies fail to consider quantitative aspects of body movement such as the precise movement direction, magnitude and timing. In this paper, we employ an innovative data mining approach to systematically study bodily correlates of deception. We re-analyze motion capture data from a previously published deception study, and experiment with different data coding options. We report how deception detection rates are affected by variables such as body part, the coding of the pose and movement, the length of the observation, and the amount of measurement noise. Our results demonstrate the feasibility of a data mining approach, with detection rates above 65%, significantly outperforming human judgement (52.80%). Owing to the systematic analysis, our analyses allow for an understanding of the importance of various coding factor. Moreover, we can reconcile seemingly discrepant findings in previous research. Our approach highlights the merits of data-driven research to support the validation and development of deception theory.

Construction of an Interactive Platform for Piano Teaching in Virtual Reality Environment

Abstract Under the impact of the wave of new technology, the form of piano teaching is no longer limited to the classroom and paper textbooks, and modern information technology, such as the Internet and virtual reality, is constantly integrated into traditional piano teaching. This paper utilizes the Unity3D engine to create a virtual reality environment for piano teaching interaction, based on VR technology, and to construct an interactive platform for piano teaching. Firstly, based on binocular stereo vision for the three-dimensional reconstruction of the teaching interactive scene, based on the improvement of the LeNet5 network for the user’s head gesture tracking, the constructed virtual scene realizes the real-time movement with the movement of the human head. The test set images using CPU-i76700HQ and GTX960 graphics cards consume 78ms and 25ms per frame, respectively, which meets the real-time system requirements. In this paper, the improved LeNet5 algorithm for head pose estimation without face detection is applicable and has a high detection accuracy of 99.31%, which is better than the algorithms such as Adaboost+Hough, LBF+BP, CNN+Heatmaps, DCNN, and FSA-Net, and is able to satisfy the real-time detection requirements of the virtual scene. The virtual piano teaching interactive platform has an average rating of more than 4 in attractiveness, subjectivity, interactivity, and convenience, but the running speed is only 3.46, so it is necessary to strengthen the technical support and improve the software and hardware level to break through the development bottleneck. Traditional piano teaching is given new vitality and kinetic energy by the interactive platform constructed in this paper for piano teaching in a virtual reality environment.

Intelligent Head-bot, towards the Development of an AI Based Cognitive Platform

A cognitive humanoid head is an AI enabled head-bot platform that resembles human's cognitive abilities, such as perception, thinking, learning, and decision-making. The platform is able to interact with human through natural language processing and recognize individuals, thus allowing seamless communication between two parties. No such cognitive platform has been introduced in Bangladesh, thus creating an open field to contribute to the field of Machine Intelligence. This study aims to develop an AI based humanoid head (head-bot) capable of imitating a range of expressions, recognizing individuals, and interacts with visitors through general conversation. The head-bot skeleton is developed using a number of hexagonal blocks of PVC sheet to mimic a human-head-like structure where LCD, camera, microphones, and speaker are mounted. Two separate Machine Learning models are designed for face detection and recognitions, and voice enabled chat-bot implementation. The head-bot platform incorporates 2-DoF neck movements for various head gestures and face tracking. The Artificial Neural Network models are tested with accuracy of 95.05%, and 99.0%, for face detection and recognitions, and speech recognitions and response generation, respectively. According to the overall results and system performances, it seems that the proposed system has a number of good potentials for real life applications such as entertainment, guidance, conversations, interactive receptionists, personal companion, medical assistance, and so on.

Personalized teaching of college psychology based on information fusion technology

Abstract This paper explores the cognitive attention mechanism in mental activities, analyzes the learning emotion contained therein as well as the information of thinking activity that carries out the whole cognitive process of mental learning, and obtains the information construction data of the three-dimensional interest model after the estimation of the learner’s head gesture and the recognition of dynamic expression. A conditional random forest model is built using multimodal information fusion technology, and a natural smile detection method is proposed based on it. After training, it realizes the information estimation of the learner’s head pose data, generates the smiley face classifier based on conditional random forest, and determines the teaching decision boundary using K-Means clustering. To analyze the psychological personalized teaching results of the model, an empirical study is conducted through a controlled experiment. The experimental results show that the learning efficiency of the model on the CelebA dataset and SMILEsmileD dataset is improved, the accuracy rate is stabilized at 95% after the number of iterations 10, and the model’s performance is superior. The majority of the students in the experiment have a mastery of psychological knowledge that is around 0.85, and there is a significant positive correlation, and personalized teaching has a more significant effect.

Utilizing EEG Signal Data and Motion to Aid in Prosthetic Hand Motion

Current prosthetic arm technologies are often difficult to use intuitively by amputees, require invasive surgical procedures, and can be extremely costly with prices ranging from $20,000 to $80,000. To address these challenges, the engineering goal of this project aims to design a smart, low-cost, mind-controlled transhumeral prosthesis by integrating the brain-interfacing capabilities of electroencephalography (EEG), the economical means of 3D printing technology, and gesture-detecting attributes of an accelerometer-gyroscope. Single-channel brain signals are transmitted through Bluetooth to be interpreted by a novel EEG decoding algorithm and head gestures from the inertial measurement unit actuate movement within the arm. Force sensitive resistors were employed to regulate force control in real-time to optimize grasp type. An LCD screen is integrated within the arm’s design to display the type of touch it is exerting on an object. The arm itself was printed with an original design utilizing PLA plastic filament making it extremely durable and lightweight. After thoroughly testing the prosthesis, the novel EEG decoding system boasts an accuracy of 94.7% and a user cognition to machine delay of 1.64 + - 0.37 seconds. The inertial measurement unit system recognizes user gestures with a delay of .136 seconds. With a bill of materials approximately $375 USD and ability to be moved in 4 degrees of motion, this novel upper limb prosthesis serves as a promising alternative to existing units on the market. The algorithms developed within this project have a wide range of use within other brain control interface projects.

A Yaw tracking algorithm for head movement from inertial sensors data

Monitoring head movements is important in many aspects of life from medicine and rehabilitation to sports, and VR entertainment. In this study, we used recordings from two sensors, i.e. an accelerometer and a gyroscope, to calculate the angles of movement of the gesturing person’s head. For the yaw motion, we proposed an original algorithm using only these two inertial sensors and the detected motion type obtained from a pre-trained SVM classifier. The combination of the gyroscope data and the detected motion type allowed us to calculate the yaw angle without the need for other sensors, such as a magnetometer or a video camera. To verify the accuracy of our algorithm, we used a robotic arm that simulated head gestures where the angle values were read out from the robot kinematics. The calculated yaw angles differed from the robot’s readings with a mean absolute error of approx. 1 degree and the rate of differences between these values exceeding 5 degrees was significantly below 1 percent except for one outlier at 1.12%. This level of accuracy is sufficient for many applications, such as VR systems, human-system interfaces, or rehabilitation.

Headar

Nod and shake of one's head are intuitive and universal gestures in communication. As smartwatches become increasingly intelligent through advances in user activity sensing technologies, many use scenarios of smartwatches demand quick responses from users in confirmation dialogs, to accept or dismiss proposed actions. Such proposed actions include making emergency calls, taking service recommendations, and starting or stopping exercise timers. Head gestures in these scenarios could be preferable to touch interactions for being hands-free and easy to perform. We propose Headar to recognize these gestures on smartwatches using wearable millimeter wave sensing. We first surveyed head gestures to understand how they are performed in conversational settings. We then investigated positions and orientations to which users raise their smartwatches. Insights from these studies guided the implementation of Headar. Additionally, we conducted modeling and simulation to verify our sensing principle. We developed a real-time sensing and inference pipeline using contemporary deep learning techniques, and proved the feasibility of our proposed approach with a user study (n=15) and a live test (n=8). Our evaluation yielded an average accuracy of 84.0% in the user study across 9 classes including nod and shake as well as seven other signals -- still, speech, touch interaction, and four non-gestural head motions (i.e., head up, left, right, and down). Furthermore, we obtained an accuracy of 72.6% in the live test which reveals rich insights into the performance of our approach in various realistic conditions.

PERAN PENDIDIKAN AGAMA ISLAM FORMAL DAN NON FORMAL DALAM MENINGKATKAN AKHLAKUL KARIMAH

Islamic religious education plays a very important role in the formation of a person's morals. And education, especially morals and morals, must be given from an early age to foster children's spiritual and physical growth, both spiritual, motoric, emotional, social and shape children's characteristics into good ones. It is not only in educational institutions that the family plays an important role in educating children, the main thing is in forming their characteristics properly and correctly.There are three forms of educational institutions currently developing in our country, namely formal, non-formal and informal. Formal educational institutions are educational institutions whose curriculum is created and managed by the government from pre-school level to university level. Then non-formal educational institutions are educational institutions whose curriculum is created and managed by the community, including Islamic boarding school or diniyah educational institutions. Meanwhile, informal education is education that is built or organized by the family and environment only for education without structure and does not require permission from the government.This article discusses the role of formal and non-formal Islamic religious education in improving morals. There are still students whose morals are not good, especially morals towards teachers. Researchers want to improve good morals towards teachers in order to gain useful knowledge and blessings. This research uses descriptive qualitative using data analysis techniques in the form of observation where the researcher takes qualitative primary data sources obtained from teachers of morals and aqidah. In this research the data source is secondary. The first research is that a student must obey the teacher by getting used to it when the teacher speaks or walks, the students bow their heads while giving respect with a bowed head gesture. To produce results such as when in class students or students must enter the class first to greet and prepare the place for the incoming teacher. The second is the teacher's guidance to students, namely using concrete story methods such as studying and giving sermons and this must also be used as an example by the teacher, so that from this method students who previously behaved badly can become good so that they can be applied in society. Third, if there are students or students who are difficult to manage, namely by using the approach of providing advice and motivation so that a student has good morals, it cannot be denied that stubborn students can occur due to internal factors such as lack of love for parents, then it will be fun for the teacher to talk to the student. Thank God, in this way students can change from bad morals to good morals..