Human-computer Interaction Systems Research Articles

Eye Movement Classification using Feature Engineering and Ensemble Machine Learning

This paper explores the classification of gaze direction using electrooculography (EOG) signals, integrating signal processing, deep learning, and ensemble learning techniques to enhance accuracy and reliability. A complex technique is proposed in which several feature types are derived from EOG data. Spectral properties generated from power spectral density analysis augment basic statistical characteristics such as mean and standard deviation, revealing the frequency content of the signal. Skewness, kurtosis, and cross-channel correlations are also used to represent intricate nonlinear dynamics and inter-channel interactions. These characteristics are then reformatted into a two-dimensional array imitating picture data, enabling the use of the pre-trained ResNet50 model to extract deep and high-level characteristics. Using these deep features, an ensemble of bagging-trained decision trees classifies gaze directions, lowering model variance and increasing prediction accuracy. The results show that the ensemble deep learning model obtained outstanding performance metrics, with accuracy and sensitivity ratings exceeding 97% and F1-score of 98%. These results not only confirm the effectiveness of the proposed approach in managing challenging EOG signal classification tasks but also imply important consequences for the improvement of Human-Computer Interaction (HCI) systems, especially in assistive technologies where accurate gaze tracking is fundamental.

Flexible pressure sensor with metallic reinforcement and graphene nanowalls for wearable electronics device

In recent years, flexible pressure sensors have been seen widespread adoption in various fields such as electronic skin, smart wearables, and human-computer interaction systems. Owing to the electrical conductivity and adaptability to flexible substrates, vertical graphene nanowalls (VGNs) have recently been recognized as promising materials for pressure-sensing applications. Our study presented the synthesis of high-quality VGNs via plasma enhanced chemical vapor deposition and the incorporation of a metal layer by electron beam evaporation, forming a stacked structure of VGNs/Metal/VGNs. Metal nanoparticles attached to the edges and surfaces of graphene nanosheets can alter the charge transport paths within the material to enhance the responsiveness of the sensor. This layered structure effectively fulfilled the requirements of flexible pressure sensors, exhibiting high sensitivity (40.15 kPa-1), low response time (88 ms), and short recovery time (97 ms). The pressure sensitivity remained intact even after 1000 bending cycles. Additionally, the factors contributing to the impressive pressure-sensing performance of this composite were found and its capability to detect human pulse and finger flexion signals was demonstrated, making it a promising candidate for applications of wearable electronics devices.

A Review of Humor Recognition Research

Abstract. Humor recognition is a popular research area in Natural Language Processing (NLP), with the fundamental goal of detecting whether humor is present. With the rapid development of artificial intelligence and the growing demand for human-computer interaction, humor recognition has broad applications in areas such as social media analysis, human-computer interaction systems, and intelligent question-answering assistants. However, due to the highly subjective and complex nature of humor, teaching computers to understand and recognize humor in a manner similar to humans is an exceptionally challenging task. This paper reviews the major research progress in the field of humor recognition and the construction of humor corpora. It discusses humor recognition methods based on traditional machine learning, deep learning, and multimodal approaches, and then highlights the advantages and disadvantages of each. Based on these analyses, the paper identifies the main challenges and difficulties in current research and provides suggestions and prospects for the future development of humor recognition systems.

Three-dimensional numerical calculation and case application of IP five-directional well logging

Abstract Conventional well logging can effectively detect the vertical distribution range of ore bodies near the well by placing detectors in the well. However, its detection capability for the horizontal extension of ore bodies is limited. Therefore, starting from the differential equations satisfied by the point source in three-dimensional structures, this paper deduced the variational problem satisfied by the total potential under three-dimensional complex conditions. By focusing on well logging, tetrahedral cross-grid partitioning technology was employed to achieve high-precision three-dimensional finite element forward modeling. Simulations were carried out for different forms of ore bodies using the induced polarization five-direction observation method, considering the presence of ore bodies near the well and the case where the measurement well passes through the ore body. The paper analyzed and summarized the induced polarization anomaly patterns formed by ore bodies with low resistance and high polarization in different horizontal extension directions under induced polarization five-direction logging with power supply measurements from different directions. Based on this, a human-computer interaction system was constructed to perform forward fitting and inversion interpretation of the measured induced polarization data, resulting in a good application effect.

Vehicle Trajectory Prediction Using Residual Diffusion Model Based on Image Information

In automatic driving, accurate prediction of vehicle trajectory is the key to achieve automatic driving, and multi-vehicle joint trajectory prediction has become an important part of modern human-computer interaction systems such as automatic driving. In order to better predict vehicle trajectories, we propose a new residual diffusion model to infer the joint distribution of future multi-vehicle trajectories. This approach has several major advantages. First, the model is able to learn multiple probability distributions from trajectory data to obtain potential outcomes for vehicles to multiple future trajectories. Secondly, in order to integrate the motion characteristics of multiple vehicles in the same scene, we use the method of combining the reference denoising and multiple residual denoising to improve the model performance and prediction speed. Finally, on this basis, a general trajectory constraint function is introduced, so that the generated trajectories of multiple vehicles will not collide with each other. We perform a rich experimental comparison of various existing methods on the NGSIM dataset and demonstrate that the proposed algorithm achieves a 26% improvement on mAP.

Re-examining the chatBot Usability Scale (BUS-11) to assess user experience with customer relationship management chatbots

AbstractIntelligent systems, such as chatbots, are likely to strike new qualities of UX that are not covered by instruments validated for legacy human–computer interaction systems. A new validated tool to evaluate the interaction quality of chatbots is the chatBot Usability Scale (BUS) composed of 11 items in five subscales. The BUS-11 was developed mainly from a psychometric perspective, focusing on ranking people by their responses and also by comparing designs’ properties (designometric). In this article, 3186 observations (BUS-11) on 44 chatbots are used to re-evaluate the inventory looking at its factorial structure, and reliability from the psychometric and designometric perspectives. We were able to identify a simpler factor structure of the scale, as previously thought. With the new structure, the psychometric and the designometric perspectives coincide, with good to excellent reliability. Moreover, we provided standardized scores to interpret the outcomes of the scale. We conclude that BUS-11 is a reliable and universal scale, meaning that it can be used to rank people and designs, whatever the purpose of the research.

Three-dimensional dynamic gesture recognition method based on convolutional neural network

With the rapid advancement of virtual reality, dynamic gesture recognition technology has become an indispensable and critical technique for users to achieve human–computer interaction in virtual environments. The recognition of dynamic gestures is a challenging task due to the high degree of freedom and the influence of individual differences and the change of gesture space. To solve the problem of low recognition accuracy of existing networks, an improved dynamic gesture recognition algorithm based on ResNeXt architecture is proposed. The algorithm employs three-dimensional convolution techniques to effectively capture the spatiotemporal features intrinsic to dynamic gestures. Additionally, to enhance the model’s focus and improve its accuracy in identifying dynamic gestures, a lightweight convolutional attention mechanism is introduced. This mechanism not only augments the model’s precision but also facilitates faster convergence during the training phase. In order to further optimize the performance of the model, a deep attention submodule is added to the convolutional attention mechanism module to strengthen the network’s capability in temporal feature extraction. Empirical evaluations on EgoGesture and NvGesture datasets show that the accuracy of the proposed model in dynamic gesture recognition reaches 95.03% and 86.21%, respectively. When operating in RGB mode, the accuracy reached 93.49% and 80.22%, respectively. These results underscore the effectiveness of the proposed algorithm in recognizing dynamic gestures with high accuracy, showcasing its potential for applications in advanced human–computer interaction systems.

Enhanced facial expression recognition using transfer learning and M-CLAHE

In recent years, Facial Expression Recognition (FER) techniques have gained substantial attention within the realm of biometric technology due to their wide range of applications, including emotion analysis, human-computer interaction, and surveillance systems. This paper presents a robust and efficient FER system composed of three key steps. First, precise face detection is performed using the Viola-Jones algorithm, a well-established method for detecting facial features in real-time. Second, the detected images are enhanced using a Modified Contrast Limited Adaptive Histogram Equalization (M-CLAHE) technique to improve contrast and visibility. Finally, feature extraction and classification are carried out using three powerful Convolutional Neural Network (CNN) architectures: VGG16, ResNet50, and Inception-v3, all benefiting from Transfer Learning to boost performance. Experiments were conducted on two widely-used datasets, JAFEE and CK+, with Inception-v3 achieving remarkable accuracy, reaching 98.43% and 99.93% on the respective databases. These results underline the effectiveness and robustness of our approach, particularly through the use of Transfer Learning, which significantly enhances the overall performance of the model compared to existing methods.

Design of a Capacitive Tactile Sensor Array System for Human-Computer Interaction.

This paper introduces a novel capacitive sensor array designed for tactile perception applications. Utilizing an all-in-one inkjet deposition printing process, the sensor array exhibited exceptional flexibility and accuracy. With a resolution of up to 32.7 dpi, the sensor array was capable of capturing the fine details of touch inputs, making it suitable for applications requiring high spatial resolution. The design incorporates two multiplexers to achieve a scanning rate of 100 Hz, ensuring the rapid and responsive data acquisition that is essential for real-time feedback in interactive applications, such as gesture recognition and haptic interfaces. To evaluate the performance of the capacitive sensor array, an experiment that involved handwritten number recognition was conducted. The results demonstrated that the sensor accurately captured fingertip inputs with a high precision. When combined with an Auxiliary Classifier Generative Adversarial Network (ACGAN) algorithm, the sensor system achieved a recognition accuracy of 98% for various handwritten numbers from "0" to "9". These results show the potential of the capacitive sensor array for advanced human-computer interaction applications.

Working Memory Capacity for Mid-Air Gestures in Human–Computer Interaction

Mid-air gesture interactions have emerged as one of the predominant modalities used in human-computer interaction (HCI). However, extant research on gestures predominantly concentrates on the design of gestures for singular functionalities, which requires a comprehensive investigation into gesture interaction design from an integrative HCI system perspective. Working memory capacity plays a pivotal role in constraining the number of gestures accommodated within an HCI system. This study adopts the change detection paradigm to conduct an empirical investigation into the working memory capacities associated with mid-air gestures, aiming to elucidate the working memory capacity for such gestures. The experimental design included two independent variables: five distinct categories of mid-air gestures (physical, symbolic, metaphorical, abstract, and mixed) and memory set sizes ranging from two to nine items. The findings revealed that working memory capacities for different types of gestures fluctuate between three and five items, with symbolic gestures presenting the most negligible cognitive load for recall and abstract gestures proving to be the most challenging. The outcomes of this research have significant implications for both the design of individual gestures and the overarching design of gesture interaction systems, serving as a valuable reference for future endeavors in HCI design optimization.

EEG Dataset for the Recognition of Different Emotions Induced in Voice-User Interaction

Electroencephalography (EEG)-based open-access datasets are available for emotion recognition studies, where external auditory/visual stimuli are used to artificially evoke pre-defined emotions. In this study, we provide a novel EEG dataset containing the emotional information induced during a realistic human-computer interaction (HCI) using a voice user interface system that mimics natural human-to-human communication. To validate our dataset via neurophysiological investigation and binary emotion classification, we applied a series of signal processing and machine learning methods to the EEG data. The maximum classification accuracy ranged from 43.3% to 90.8% over 38 subjects and classification features could be interpreted neurophysiologically. Our EEG data could be used to develop a reliable HCI system because they were acquired in a natural HCI environment. In addition, auxiliary physiological data measured simultaneously with the EEG data also showed plausible results, i.e., electrocardiogram, photoplethysmogram, galvanic skin response, and facial images, which could be utilized for automatic emotion discrimination independently from, as well as together with the EEG data via the fusion of multi-modal physiological datasets.

Research on click enhancement strategy of hand-eye dual-channel human-computer interaction system: Trade-off between sensing area and area cursor

This study aims to explore the application of click enhancement strategies in a “Sight Line Localization + Hand Triggered” eye-control human–computer interaction system, proposes a click enhancement strategy to solve two critical problems in eye-control human–computer interaction: Midas touch and low spatial accuracy. By conducting ergonomics experiments, we verify that the proposed click enhancement strategy can effectively improve the operational performance of hand-eye dual-channel HCI systems. The experimental results show that the accuracy of the operation can be significantly improved by using a cursor size equal to the diameter of the interaction control and a sensing area size of 1.8 times the diameter of the control. Based on the comprehensive consideration of operation efficiency and comfort, 0.75 times the control diameter of the cursor and 1.8 times the control diameter of the sensing area are the optimal parameter configurations. The results not only solve the problems of Midas touch and low spatial accuracy but also significantly reduce visual fatigue, thus improving the ease of use and robustness of the hand-eye dual-channel human–computer interaction system.

أنظمة التفاعل التكيفي بين الإنسان والحاسوب مع التعرف على عواطف المستخدم والتنبؤ بالاستجابة المستندة إلى GRU

Human-Computer Interaction (HCI) has evolved to incorporate sophisticated systems capable of recognizing and responding to user emotions, enhancing the user experience by making interactions more intuitive and engaging. This paper explores the development of adaptive HCI systems utilizing Gated Recurrent Unit (GRU) neural networks for emotion recognition and response prediction. The core objective is to create interfaces that dynamically adjust based on the user's emotional state, thereby improving usability and satisfaction across various applications, including healthcare, education, and customer service. GRU-based models are particularly effective for this task due to their ability to handle sequential data and capture temporal dependencies inherent in emotional expressions. By processing multimodal inputs such as facial expressions, voice intonations, text, and physiological signals, these systems can accurately detect and predict emotions in real-time. The DEAP dataset, which includes EEG and other physiological recordings along with self-assessed emotional ratings, serves as a foundational resource for training and validating these models. The proposed approach involves pre-processing the physiological data using techniques like min-max normalization to ensure consistency and stability during model training. The GRU models then learn to map these inputs to corresponding emotional states, leveraging their memory capabilities to retain and interpret temporal patterns. The adaptive nature of these systems enables them to provide personalized responses, such as adjusting the interface layout or offering support when negative emotions are detected. The implementation of GRU-based emotion recognition and response prediction in HCI systems holds significant potential for enhancing user interactions by making them more responsive and emotionally intelligent. This paper demonstrates the effectiveness of GRU models in real-time emotion monitoring and highlights their applications in creating more adaptive and empathetic technology interfaces. The proposed model is implemented in Python and has an accuracy of about 99.12% which is higher than other existing methods. Keywords: Human-Computer Interaction (HCI), Gated Recurrent Unit (GRU), Neural Networks, Emotional Detection.

Automobile Intelligent Vehicle-machine and Human-computer Interaction System based on Big Data

This paper measures the accelerator, brake pedal, clutch, transmission device and steering wheel under different driving conditions in real-time and accurately on the simulation experiment platform. The goal is to conduct human-machine-road system interaction in a virtual simulation of human-vehicle-road systems. Fitting test data establishes the mathematical model of traffic control parameters. In terms of hardware, the distributed architecture of upper and lower computers is utilized. The system communicates point-to-point with the host computer through the RS-232 serial port. The system adopts multi-thread technology and serial communication technology. The simulation system of driving operation is designed with the visual central controller. The system takes 89S52 as the core. The slave program is written in C language. Then, the system establishes a multi-target coordinated obstacle avoidance method based on a multi-sensor information vehicle cooperation collision avoidance method. The multi-vehicle cooperation obstacle avoidance problem is transformed into an optimal control problem under multiple constraints. Simulation analysis shows that the velocity and displacement obtained by the multi-robot collaborative collision avoidance method are in good agreement with the measured values. Compared with the time series algorithm, the output accuracy of the proposed collaborative collision avoidance algorithm is significantly reduced, and the changes in velocity and displacement in the time domain are more stable.

RETRACTED ARTICLEResearch Progress on 2D Human Pose Estimation Based on Deep Learning

Human pose estimation (HPE) is a significant area of study within computer vision and artificial intelligence, which locates the position and pose of human joints through images or videos. HPE has advanced progress in computer vision and deep learning fields at the theoretical level, improved the efficiency and accuracy of neural networks, and supported the development of natural and efficient human-computer interaction systems. At the application level, HPE has extensive value in many fields, including security monitoring, motion analysis and health management, VR&AR, robotics technology, film and television production and animation, intelligent driving and assisted driving. This paper divides 2D HPE tasks into three categories: two-stage (top-down and bottom-up), single-stage, and other improvement strategies (Transformer-based methods and post-processing optimization methods). Although deep learning-based pose estimation algorithms have achieved remarkable results, they still face challenges, such as prediction in complex scenes, crowd and joint occlusion problems, and lightweight design of models. Future work should pay more attention to these challenges and extend pose estimation technology from 2D to 3D contexts.

Multifunctional Human-Computer Interaction System Based on Deep Learning-Assisted Strain Sensing Array.

Continuous and reliable monitoring of gait is crucial for health monitoring, such as postoperative recovery of bone joint surgery and early diagnosis of disease. However, existing gait analysis systems often suffer from large volumes and the requirement of special space for setting motion capture systems, limiting their application in daily life. Here, we develop an intelligent gait monitoring and analysis prediction system based on flexible piezoelectric sensors and deep learning neural networks with high sensitivity (241.29 mV/N), quick response (66 ms loading, 87 ms recovery), and excellent stability (R2 = 0.9946). The theoretical simulations and experiments confirm that the sensor provides exceptional signal feedback, which can easily acquire accurate gait data when fitted to shoe soles. By integrating high-quality gait data with a custom-built deep learning model, the system can detect and infer human motion states in real time (the recognition accuracy reaches 94.7%). To further validate the sensor's application in real life, we constructed a flexible wearable recognition system with human-computer interaction interface and a simple operation process for long-term and continuous tracking of athletes' gait, potentially aiding personalized health management, early detection of disease, and remote medical care.

Reconstruction of Human-Computer Interactive PE Classroom Instructional Quality Assessment System Based AI-powered CAD Technology

Reconstruction of Human-Computer Interactive PE Classroom Instructional Quality Assessment System Based AI-powered CAD Technology

Wide-response-range and high-sensitivity piezoresistive sensors with triple periodic minimal surface (TPMS) structures for wearable human-computer interaction systems

High-performance pressure sensors are garnering interest in human-computer interaction technology, wearable devices, and bionic electronic skin development. However, highly sensitive sensors frequently have a limited response range. In this work, we developed composites with outstanding conductive network structures through the synergistic effect of transition metal carbides (MXene) and multi-walled carbon nanotubes (MWCNTs). Additionally, pressure sensors with various TPMS structures were prepared using innovative parametric design and Fused Deposition Molding (FDM) printing. Due to the stable synergistic conductive network and distinctive curved surface structure, the sensors exhibit exceptional sensing performance. This includes high sensitivity ranging from 4.67 MPa−1 to 7.03 MPa−1 (within the range of 0–0.1 MPa), a broad operating range (maximum 10 MPa), rapid response and recovery times (326 ms/193.4 ms), and long-term fatigue resistance (over 10,000 s cycles). By integrating mechanical properties, sensing properties, and finite element simulations, we analyzed the mechanism underlying the impact of various TPMS pore structures on the sensitivity and response range of the pressure sensor. In addition, the sensors were arrayed as 4 × 4 modules to successfully recognize a wide range of foot movements from different volunteers. These findings illuminate potential applications in human motion detection, healthcare rehabilitation, and artificial intelligence.

CENN: Capsule-enhanced neural network with innovative metrics for robust speech emotion recognition

Speech emotion recognition (SER) plays a pivotal role in enhancing Human-computer interaction (HCI) systems. This paper introduces a groundbreaking Capsule-enhanced neural network (CENN) that significantly advances the state of SER through a robust and reproducible deep learning framework. The CENN architecture seamlessly integrates advanced components, including Multi-head attention (MHA), residual module, and capsule module, which collectively enhance the model's capacity to capture both global and local features essential for precise emotion classification. A key contribution of this work is the development of a comprehensive reproducibility framework, featuring novel metrics: General learning reproducibility (GLR) and Correct learning reproducibility (CLR). These metrics, alongside their fractional and perfect variants, offer a multi-dimensional evaluation of the model's consistency and correctness across multiple executions, thereby ensuring the reliability and credibility of the results. To tackle the persistent challenge of overfitting in deep learning models, we propose an innovative overfitting metric that considers the intricate relationship between training and testing errors, model complexity, and data complexity. This metric, in conjunction with the newly introduced generalization and robustness metrics, provides a holistic assessment of the model's performance, guiding the application of regularization techniques to maintain generalizability and resilience. Extensive experiments conducted on benchmark SER datasets demonstrate that the CENN model not only surpasses existing approaches in terms of accuracy but also sets a new benchmark in reproducibility. This work establishes a new paradigm for deep learning model development in SER, underscoring the vital importance of reproducibility and offering a rigorous framework for future research.

Speech recognition and intelligent translation under multimodal human–computer interaction system

Abstract The traditional translation robot is limited to the translation of single-mode text images and text videos, which has the problem of low translation accuracy. Therefore, speech recognition and intelligent translation in multimodal human–computer interaction (HCI) system are proposed. First, the network structure of speech recognition model in multi-channel HCI system is established, and the multi-head self-attention mechanism is constructed. Then, the artificial intelligence voice wake-up function is designed, and a multimodal machine translation model is constructed. On this basis, selective attention is added to obtain visual recognition of perceived text, and the decoder is used for multimodal gating fusion to realize the output of encoder translation results. Experimental results show that this method has high BLUE value and high translation accuracy.