Human-computer Interaction Systems Research Articles

Three-dimensional numerical calculation and case application of IP five-directional well logging

Abstract Conventional well logging can effectively detect the vertical distribution range of ore bodies near the well by placing detectors in the well. However, its detection capability for the horizontal extension of ore bodies is limited. Therefore, starting from the differential equations satisfied by the point source in three-dimensional structures, this paper deduced the variational problem satisfied by the total potential under three-dimensional complex conditions. By focusing on well logging, tetrahedral cross-grid partitioning technology was employed to achieve high-precision three-dimensional finite element forward modeling. Simulations were carried out for different forms of ore bodies using the induced polarization five-direction observation method, considering the presence of ore bodies near the well and the case where the measurement well passes through the ore body. The paper analyzed and summarized the induced polarization anomaly patterns formed by ore bodies with low resistance and high polarization in different horizontal extension directions under induced polarization five-direction logging with power supply measurements from different directions. Based on this, a human-computer interaction system was constructed to perform forward fitting and inversion interpretation of the measured induced polarization data, resulting in a good application effect.

Vehicle Trajectory Prediction Using Residual Diffusion Model Based on Image Information

In automatic driving, accurate prediction of vehicle trajectory is the key to achieve automatic driving, and multi-vehicle joint trajectory prediction has become an important part of modern human-computer interaction systems such as automatic driving. In order to better predict vehicle trajectories, we propose a new residual diffusion model to infer the joint distribution of future multi-vehicle trajectories. This approach has several major advantages. First, the model is able to learn multiple probability distributions from trajectory data to obtain potential outcomes for vehicles to multiple future trajectories. Secondly, in order to integrate the motion characteristics of multiple vehicles in the same scene, we use the method of combining the reference denoising and multiple residual denoising to improve the model performance and prediction speed. Finally, on this basis, a general trajectory constraint function is introduced, so that the generated trajectories of multiple vehicles will not collide with each other. We perform a rich experimental comparison of various existing methods on the NGSIM dataset and demonstrate that the proposed algorithm achieves a 26% improvement on mAP.

Re-examining the chatBot Usability Scale (BUS-11) to assess user experience with customer relationship management chatbots

AbstractIntelligent systems, such as chatbots, are likely to strike new qualities of UX that are not covered by instruments validated for legacy human–computer interaction systems. A new validated tool to evaluate the interaction quality of chatbots is the chatBot Usability Scale (BUS) composed of 11 items in five subscales. The BUS-11 was developed mainly from a psychometric perspective, focusing on ranking people by their responses and also by comparing designs’ properties (designometric). In this article, 3186 observations (BUS-11) on 44 chatbots are used to re-evaluate the inventory looking at its factorial structure, and reliability from the psychometric and designometric perspectives. We were able to identify a simpler factor structure of the scale, as previously thought. With the new structure, the psychometric and the designometric perspectives coincide, with good to excellent reliability. Moreover, we provided standardized scores to interpret the outcomes of the scale. We conclude that BUS-11 is a reliable and universal scale, meaning that it can be used to rank people and designs, whatever the purpose of the research.

Three-dimensional dynamic gesture recognition method based on convolutional neural network

With the rapid advancement of virtual reality, dynamic gesture recognition technology has become an indispensable and critical technique for users to achieve human–computer interaction in virtual environments. The recognition of dynamic gestures is a challenging task due to the high degree of freedom and the influence of individual differences and the change of gesture space. To solve the problem of low recognition accuracy of existing networks, an improved dynamic gesture recognition algorithm based on ResNeXt architecture is proposed. The algorithm employs three-dimensional convolution techniques to effectively capture the spatiotemporal features intrinsic to dynamic gestures. Additionally, to enhance the model’s focus and improve its accuracy in identifying dynamic gestures, a lightweight convolutional attention mechanism is introduced. This mechanism not only augments the model’s precision but also facilitates faster convergence during the training phase. In order to further optimize the performance of the model, a deep attention submodule is added to the convolutional attention mechanism module to strengthen the network’s capability in temporal feature extraction. Empirical evaluations on EgoGesture and NvGesture datasets show that the accuracy of the proposed model in dynamic gesture recognition reaches 95.03% and 86.21%, respectively. When operating in RGB mode, the accuracy reached 93.49% and 80.22%, respectively. These results underscore the effectiveness of the proposed algorithm in recognizing dynamic gestures with high accuracy, showcasing its potential for applications in advanced human–computer interaction systems.

Enhanced facial expression recognition using transfer learning and M-CLAHE

In recent years, Facial Expression Recognition (FER) techniques have gained substantial attention within the realm of biometric technology due to their wide range of applications, including emotion analysis, human-computer interaction, and surveillance systems. This paper presents a robust and efficient FER system composed of three key steps. First, precise face detection is performed using the Viola-Jones algorithm, a well-established method for detecting facial features in real-time. Second, the detected images are enhanced using a Modified Contrast Limited Adaptive Histogram Equalization (M-CLAHE) technique to improve contrast and visibility. Finally, feature extraction and classification are carried out using three powerful Convolutional Neural Network (CNN) architectures: VGG16, ResNet50, and Inception-v3, all benefiting from Transfer Learning to boost performance. Experiments were conducted on two widely-used datasets, JAFEE and CK+, with Inception-v3 achieving remarkable accuracy, reaching 98.43% and 99.93% on the respective databases. These results underline the effectiveness and robustness of our approach, particularly through the use of Transfer Learning, which significantly enhances the overall performance of the model compared to existing methods.

Design of a Capacitive Tactile Sensor Array System for Human-Computer Interaction.

This paper introduces a novel capacitive sensor array designed for tactile perception applications. Utilizing an all-in-one inkjet deposition printing process, the sensor array exhibited exceptional flexibility and accuracy. With a resolution of up to 32.7 dpi, the sensor array was capable of capturing the fine details of touch inputs, making it suitable for applications requiring high spatial resolution. The design incorporates two multiplexers to achieve a scanning rate of 100 Hz, ensuring the rapid and responsive data acquisition that is essential for real-time feedback in interactive applications, such as gesture recognition and haptic interfaces. To evaluate the performance of the capacitive sensor array, an experiment that involved handwritten number recognition was conducted. The results demonstrated that the sensor accurately captured fingertip inputs with a high precision. When combined with an Auxiliary Classifier Generative Adversarial Network (ACGAN) algorithm, the sensor system achieved a recognition accuracy of 98% for various handwritten numbers from "0" to "9". These results show the potential of the capacitive sensor array for advanced human-computer interaction applications.

Working Memory Capacity for Mid-Air Gestures in Human–Computer Interaction

Mid-air gesture interactions have emerged as one of the predominant modalities used in human-computer interaction (HCI). However, extant research on gestures predominantly concentrates on the design of gestures for singular functionalities, which requires a comprehensive investigation into gesture interaction design from an integrative HCI system perspective. Working memory capacity plays a pivotal role in constraining the number of gestures accommodated within an HCI system. This study adopts the change detection paradigm to conduct an empirical investigation into the working memory capacities associated with mid-air gestures, aiming to elucidate the working memory capacity for such gestures. The experimental design included two independent variables: five distinct categories of mid-air gestures (physical, symbolic, metaphorical, abstract, and mixed) and memory set sizes ranging from two to nine items. The findings revealed that working memory capacities for different types of gestures fluctuate between three and five items, with symbolic gestures presenting the most negligible cognitive load for recall and abstract gestures proving to be the most challenging. The outcomes of this research have significant implications for both the design of individual gestures and the overarching design of gesture interaction systems, serving as a valuable reference for future endeavors in HCI design optimization.

EEG Dataset for the Recognition of Different Emotions Induced in Voice-User Interaction

Electroencephalography (EEG)-based open-access datasets are available for emotion recognition studies, where external auditory/visual stimuli are used to artificially evoke pre-defined emotions. In this study, we provide a novel EEG dataset containing the emotional information induced during a realistic human-computer interaction (HCI) using a voice user interface system that mimics natural human-to-human communication. To validate our dataset via neurophysiological investigation and binary emotion classification, we applied a series of signal processing and machine learning methods to the EEG data. The maximum classification accuracy ranged from 43.3% to 90.8% over 38 subjects and classification features could be interpreted neurophysiologically. Our EEG data could be used to develop a reliable HCI system because they were acquired in a natural HCI environment. In addition, auxiliary physiological data measured simultaneously with the EEG data also showed plausible results, i.e., electrocardiogram, photoplethysmogram, galvanic skin response, and facial images, which could be utilized for automatic emotion discrimination independently from, as well as together with the EEG data via the fusion of multi-modal physiological datasets.

Research on click enhancement strategy of hand-eye dual-channel human-computer interaction system: Trade-off between sensing area and area cursor

This study aims to explore the application of click enhancement strategies in a “Sight Line Localization + Hand Triggered” eye-control human–computer interaction system, proposes a click enhancement strategy to solve two critical problems in eye-control human–computer interaction: Midas touch and low spatial accuracy. By conducting ergonomics experiments, we verify that the proposed click enhancement strategy can effectively improve the operational performance of hand-eye dual-channel HCI systems. The experimental results show that the accuracy of the operation can be significantly improved by using a cursor size equal to the diameter of the interaction control and a sensing area size of 1.8 times the diameter of the control. Based on the comprehensive consideration of operation efficiency and comfort, 0.75 times the control diameter of the cursor and 1.8 times the control diameter of the sensing area are the optimal parameter configurations. The results not only solve the problems of Midas touch and low spatial accuracy but also significantly reduce visual fatigue, thus improving the ease of use and robustness of the hand-eye dual-channel human–computer interaction system.

Automobile Intelligent Vehicle-machine and Human-computer Interaction System based on Big Data

This paper measures the accelerator, brake pedal, clutch, transmission device and steering wheel under different driving conditions in real-time and accurately on the simulation experiment platform. The goal is to conduct human-machine-road system interaction in a virtual simulation of human-vehicle-road systems. Fitting test data establishes the mathematical model of traffic control parameters. In terms of hardware, the distributed architecture of upper and lower computers is utilized. The system communicates point-to-point with the host computer through the RS-232 serial port. The system adopts multi-thread technology and serial communication technology. The simulation system of driving operation is designed with the visual central controller. The system takes 89S52 as the core. The slave program is written in C language. Then, the system establishes a multi-target coordinated obstacle avoidance method based on a multi-sensor information vehicle cooperation collision avoidance method. The multi-vehicle cooperation obstacle avoidance problem is transformed into an optimal control problem under multiple constraints. Simulation analysis shows that the velocity and displacement obtained by the multi-robot collaborative collision avoidance method are in good agreement with the measured values. Compared with the time series algorithm, the output accuracy of the proposed collaborative collision avoidance algorithm is significantly reduced, and the changes in velocity and displacement in the time domain are more stable.

RETRACTED ARTICLEResearch Progress on 2D Human Pose Estimation Based on Deep Learning

Human pose estimation (HPE) is a significant area of study within computer vision and artificial intelligence, which locates the position and pose of human joints through images or videos. HPE has advanced progress in computer vision and deep learning fields at the theoretical level, improved the efficiency and accuracy of neural networks, and supported the development of natural and efficient human-computer interaction systems. At the application level, HPE has extensive value in many fields, including security monitoring, motion analysis and health management, VR&AR, robotics technology, film and television production and animation, intelligent driving and assisted driving. This paper divides 2D HPE tasks into three categories: two-stage (top-down and bottom-up), single-stage, and other improvement strategies (Transformer-based methods and post-processing optimization methods). Although deep learning-based pose estimation algorithms have achieved remarkable results, they still face challenges, such as prediction in complex scenes, crowd and joint occlusion problems, and lightweight design of models. Future work should pay more attention to these challenges and extend pose estimation technology from 2D to 3D contexts.

Multifunctional Human-Computer Interaction System Based on Deep Learning-Assisted Strain Sensing Array.

Continuous and reliable monitoring of gait is crucial for health monitoring, such as postoperative recovery of bone joint surgery and early diagnosis of disease. However, existing gait analysis systems often suffer from large volumes and the requirement of special space for setting motion capture systems, limiting their application in daily life. Here, we develop an intelligent gait monitoring and analysis prediction system based on flexible piezoelectric sensors and deep learning neural networks with high sensitivity (241.29 mV/N), quick response (66 ms loading, 87 ms recovery), and excellent stability (R2 = 0.9946). The theoretical simulations and experiments confirm that the sensor provides exceptional signal feedback, which can easily acquire accurate gait data when fitted to shoe soles. By integrating high-quality gait data with a custom-built deep learning model, the system can detect and infer human motion states in real time (the recognition accuracy reaches 94.7%). To further validate the sensor's application in real life, we constructed a flexible wearable recognition system with human-computer interaction interface and a simple operation process for long-term and continuous tracking of athletes' gait, potentially aiding personalized health management, early detection of disease, and remote medical care.

Reconstruction of Human-Computer Interactive PE Classroom Instructional Quality Assessment System Based AI-powered CAD Technology

Reconstruction of Human-Computer Interactive PE Classroom Instructional Quality Assessment System Based AI-powered CAD Technology

Wide-response-range and high-sensitivity piezoresistive sensors with triple periodic minimal surface (TPMS) structures for wearable human-computer interaction systems

High-performance pressure sensors are garnering interest in human-computer interaction technology, wearable devices, and bionic electronic skin development. However, highly sensitive sensors frequently have a limited response range. In this work, we developed composites with outstanding conductive network structures through the synergistic effect of transition metal carbides (MXene) and multi-walled carbon nanotubes (MWCNTs). Additionally, pressure sensors with various TPMS structures were prepared using innovative parametric design and Fused Deposition Molding (FDM) printing. Due to the stable synergistic conductive network and distinctive curved surface structure, the sensors exhibit exceptional sensing performance. This includes high sensitivity ranging from 4.67 MPa−1 to 7.03 MPa−1 (within the range of 0–0.1 MPa), a broad operating range (maximum 10 MPa), rapid response and recovery times (326 ms/193.4 ms), and long-term fatigue resistance (over 10,000 s cycles). By integrating mechanical properties, sensing properties, and finite element simulations, we analyzed the mechanism underlying the impact of various TPMS pore structures on the sensitivity and response range of the pressure sensor. In addition, the sensors were arrayed as 4 × 4 modules to successfully recognize a wide range of foot movements from different volunteers. These findings illuminate potential applications in human motion detection, healthcare rehabilitation, and artificial intelligence.

CENN: Capsule-enhanced neural network with innovative metrics for robust speech emotion recognition

Speech emotion recognition (SER) plays a pivotal role in enhancing Human-computer interaction (HCI) systems. This paper introduces a groundbreaking Capsule-enhanced neural network (CENN) that significantly advances the state of SER through a robust and reproducible deep learning framework. The CENN architecture seamlessly integrates advanced components, including Multi-head attention (MHA), residual module, and capsule module, which collectively enhance the model's capacity to capture both global and local features essential for precise emotion classification. A key contribution of this work is the development of a comprehensive reproducibility framework, featuring novel metrics: General learning reproducibility (GLR) and Correct learning reproducibility (CLR). These metrics, alongside their fractional and perfect variants, offer a multi-dimensional evaluation of the model's consistency and correctness across multiple executions, thereby ensuring the reliability and credibility of the results. To tackle the persistent challenge of overfitting in deep learning models, we propose an innovative overfitting metric that considers the intricate relationship between training and testing errors, model complexity, and data complexity. This metric, in conjunction with the newly introduced generalization and robustness metrics, provides a holistic assessment of the model's performance, guiding the application of regularization techniques to maintain generalizability and resilience. Extensive experiments conducted on benchmark SER datasets demonstrate that the CENN model not only surpasses existing approaches in terms of accuracy but also sets a new benchmark in reproducibility. This work establishes a new paradigm for deep learning model development in SER, underscoring the vital importance of reproducibility and offering a rigorous framework for future research.

Speech recognition and intelligent translation under multimodal human–computer interaction system

Abstract The traditional translation robot is limited to the translation of single-mode text images and text videos, which has the problem of low translation accuracy. Therefore, speech recognition and intelligent translation in multimodal human–computer interaction (HCI) system are proposed. First, the network structure of speech recognition model in multi-channel HCI system is established, and the multi-head self-attention mechanism is constructed. Then, the artificial intelligence voice wake-up function is designed, and a multimodal machine translation model is constructed. On this basis, selective attention is added to obtain visual recognition of perceived text, and the decoder is used for multimodal gating fusion to realize the output of encoder translation results. Experimental results show that this method has high BLUE value and high translation accuracy.

Cognitive evaluation based on regression and eye-tracking for layout on human–computer multi-interface

ABSTRACT The human–computer cooperation process guided by natural interaction, intelligent interaction, and human–computer integration is gradually becoming a new trend in human–computer interaction. Cooperative scenarios of human–computer interaction systems often contain multi-interface and multi-device results in edges often interrupt the cognitive ergonomics of interface layout. This research takes typical areas as an example to establish a stepwise regression model to predict reaction time at an arbitrary position on the left interface. It uses a foveal region to position the starting point of attention and a parafoveal region to calculate the radius of each objective area, and design 10 similar tasks to analyze eye-tracking indexes through physiological assessment. Unlike fixed thinking such as spatial proximity on multi-interfaces, this research summarises cognitive features of layout based on the positive and negative effects of edge impact through eye-tracking analysis. It analyzes cognition including input, process, and output in human–computer cooperation from human intelligence and artificial intelligence respectively, and visualises the mapping relationship between these indexes and specific stages of cognition. Besides, the quantitative evaluation of the regression equation and qualitative analysis of the eye-tracking indexes provide a reference for other interfaces around the front interface.

Carbon Quantum Dot-Functionalized Dermis-Derived Transparent Electronic Skin for Multimodal Human Motion Signal Monitoring and Construction of Self-Powered Triboelectric Nanogenerator.

Electronic skin (e-skin) is considered as a highly promising interface for human-computer interaction systems and wearable electronic devices. Through elaborate design and assembly of various materials, it possesses multiple characteristics similar to human skin, including remarkable flexibility, stretchability, sensitivity to temperature and humidity, biocompatibility, and efficient interfacial ion/electron transport capabilities. Here, we innovatively integrate multifunctional carbon quantum dots (CQDs), which exhibit conductivity, antibacterial properties, ultraviolet absorption, and fluorescence emission, with poly(acrylic acid) and glycerin (Gly) into a three-dimensional network structure of natural goatskin collagen fibers. Through a top-down design strategy enhanced by hydrogen bond reconstruction, we successfully fabricated a novel transparent e-skin (PAC-eSkin). This e-skin exhibited significant tensile properties (4.94 MPa of tensile strength and 263.42% of a maximum breaking elongation), while also possessing Young's modulus similar to human skin (2.32 MPa). It is noteworthy that the functionalized CQDs used was derived from discarded goat hair, and the addition of Gly gave PAC-eSkin excellent antifreezing and moisturizing properties. Due to the presence of ultrasmall CQDs, which creates efficient ion/electron transport channels within PAC-eSkin, it could rapidly sense human motion and physiological signals (with a gauge factor (GF) of 1.88). Furthermore, PAC-eSkin had the potential to replace traditional electrode patches for real-time monitoring of electrocardiogram, electromyogram, and electrooculogram signals, with a higher SNR (signal-to-noise ratio) of 25.1 dB. Additionally, the customizable size and shape of PAC-eSkin offer vast possibilities for the construction of single-electrode triboelectric nanogenerator systems. We have reason to believe that the design and development of this transparent e-skin based on CQDs-functionalized dermal collagen matrices can pave a new way for innovations in human-computer interaction interfaces and their sensing application in diverse scenarios.

Emotion recognition using hierarchical spatial–temporal learning transformer from regional to global brain

Emotion recognition is an essential but challenging task in human–computer interaction systems due to the distinctive spatial structures and dynamic temporal dependencies associated with each emotion. However, current approaches fail to accurately capture the intricate effects of electroencephalogram (EEG) signals across different brain regions on emotion recognition. Therefore, this paper designs a transformer-based method, denoted by R2G-STLT, which relies on a spatial–temporal transformer encoder with regional to global hierarchical learning that learns the representative spatiotemporal features from the electrode level to the brain-region level. The regional spatial–temporal transformer (RST-Trans) encoder is designed to obtain spatial information and context dependence at the electrode level aiming to learn the regional spatiotemporal features. Then, the global spatial–temporal transformer (GST-Trans) encoder is utilized to extract reliable global spatiotemporal features, reflecting the impact of various brain regions on emotion recognition tasks. Moreover, the multi-head attention mechanism is placed into the GST-Trans encoder to empower it to capture the long-range spatial–temporal information among the brain regions. Finally, subject-independent experiments are conducted on each frequency band of the DEAP, SEED, and SEED-IV datasets to assess the performance of the proposed model. Results indicate that the R2G-STLT model surpasses several state-of-the-art approaches.

A training and assessment system for human-computer interaction combining fNIRS and eye-tracking data

In the domain of engineering interactive product design, the optimization of Human-Computer Interaction (HCI) is central to enhancing user experience and system efficiency. Notably, the application of cognitive load theory holds significant prominence in this field. Efficient evaluation of cognitive load not only facilitates the optimization of resource management and task allocation, but is also the key to creating a fluent interactive experience. With the advancement of Artificial Intelligence (AI) and physiological measurement technologies, new methodologies have emerged to measure and understand cognitive load. This research introduces a system for HCI training and assessment. The system, through the development of digital memory paradigms, induces and verifies varying levels of cognitive load with precision, thereby providing data-driven insights for HCI design. Specifically, this study constructs an AI-assisted multimodal cognitive load assessment framework, integrating physiological data collected from Functional Near-Infrared Spectroscopy (fNIRS) and eye-tracking technologies. By extracting and analyzing 29 physiological features, including channel features, graph-theoretic features, and features pertaining to eyelid and iris movements, we propose an innovative multimodal recognition approach to classify cognitive load across individuals. The experimental results not only verified the validity of the method, but also revealed changes in physiological patterns at different levels of cognitive load. In particular, the significant potential of the fNIRS feature in band I. This discovery suggests that we can monitor and predict cognitive load with greater precision through physiological signals, hence optimizing the design of HCI systems to alleviate user strain, prevent information overload, and enhance the naturalness and intuitiveness of interaction.