Human-computer Interaction Technology Research Articles

Application of a Dual Upper Limb Task-Oriented Robotic System for the Functional Recovery of the Upper Limb in Stroke Patients.

Highly repetitive and task-oriented training has been shown to promote the recovery of limb function in stroke patients. Additionally, bilateral arm training can help stroke survivors regain their upper limb function and improve their daily activities. The dual upper limb task-oriented robotic system is designed to assist the healthy side of the stroke patient in driving the affected side to perform bilateral arm training through the use of a robotic device. It can also guide the patient in carrying out dual upper limb coordinated movements and engage them in a task-oriented virtual game using force feedback and human-computer interaction technology. This study aimed to assess the efficacy of the system in enhancing upper limb function and activities of daily living in stroke patients. The assessment methods used included motor evoked potential (MEP), functional test for the hemiplegic upper extremity-Hong Kong (FTHUE-HK), Fugl-Meyer Assessment Upper Extremity Scale (FMA-UE), and modified Barthel index (MBI). The results of the study indicate that the dual upper limb task-oriented robotic system can significantly improve the corticospinal pathway, upper limb function, and activities of daily living in stroke patients after 6 weeks of treatment. This system can serve as an effective adjunct to upper limb functional rehabilitation in stroke survivors, reducing the dependence on rehabilitation therapists. In conclusion, the dual upper limb task-oriented robotic system provides a new strategy for post-stroke limb functional rehabilitation and holds great potential for application, as it offers certain social and financial benefits.

Research on the Impact of Land Transfer on Farmer Household Consumption through CAD Technology in Human-Computer Interaction

Research on the Impact of Land Transfer on Farmer Household Consumption through CAD Technology in Human-Computer Interaction

Wide-response-range and high-sensitivity piezoresistive sensors with triple periodic minimal surface (TPMS) structures for wearable human-computer interaction systems

High-performance pressure sensors are garnering interest in human-computer interaction technology, wearable devices, and bionic electronic skin development. However, highly sensitive sensors frequently have a limited response range. In this work, we developed composites with outstanding conductive network structures through the synergistic effect of transition metal carbides (MXene) and multi-walled carbon nanotubes (MWCNTs). Additionally, pressure sensors with various TPMS structures were prepared using innovative parametric design and Fused Deposition Molding (FDM) printing. Due to the stable synergistic conductive network and distinctive curved surface structure, the sensors exhibit exceptional sensing performance. This includes high sensitivity ranging from 4.67 MPa−1 to 7.03 MPa−1 (within the range of 0–0.1 MPa), a broad operating range (maximum 10 MPa), rapid response and recovery times (326 ms/193.4 ms), and long-term fatigue resistance (over 10,000 s cycles). By integrating mechanical properties, sensing properties, and finite element simulations, we analyzed the mechanism underlying the impact of various TPMS pore structures on the sensitivity and response range of the pressure sensor. In addition, the sensors were arrayed as 4 × 4 modules to successfully recognize a wide range of foot movements from different volunteers. These findings illuminate potential applications in human motion detection, healthcare rehabilitation, and artificial intelligence.

Real-Time Hand Gesture Recognition: A Comprehensive Review of Techniques, Applications, and Challenges

Abstract Real-time Hand Gesture Recognition (HGR) has emerged as a vital technology in human-computer interaction, offering intuitive and natural ways for users to interact with computer-vision systems. This comprehensive review explores the advancements, challenges, and future directions in real-time HGR. Various HGR-related technologies have also been investigated, including sensors and vision technologies, which are utilized as a preliminary step in acquiring data in HGR systems. This paper discusses different recognition approaches, from traditional handcrafted feature methods to state-of-the-art deep learning techniques. Learning paradigms have been analyzed such as supervised, unsupervised, transfer, and adaptive learning in the context of HGR. A wide range of applications has been covered, from sign language recognition to healthcare and security systems. Despite significant developments in the computer vision domain, challenges remain in areas such as environmental robustness, gesture complexity, computational efficiency, and user adaptability. Lastly, this paper concludes by highlighting potential solutions and future research directions trying to develop more robust, efficient, and user-friendly real-time HGR systems.

Research on the mechanism of motor muscle control based on optical EEG images

The study of motor muscle control mechanisms can improve rehabilitation therapy and human-computer interaction technology. The limitations of traditional electroencephalography (EEG) limit the comprehensive understanding of motor muscle control mechanisms. Therefore, this study aims to explore the mechanism of motor muscle control based on optical EEG images, in order to expand the understanding of the process of motor control. The study selected optical EEG imaging technology as the main data acquisition tool. Optical EEG images have higher spatiotemporal resolution and can provide more detailed neural activity information. This technology combines optical imaging with EEG images to obtain spatiotemporal information of brain activity in a short period of time. The device is composed of multiple optical sensors and can measure blood oxygen concentration and neuronal activity in the cerebral cortex. Preprocess EEG image data using image processing and signal processing techniques, then use computational methods and algorithms to detect activated regions, and evaluate their relationships using correlation analysis and statistical methods. By comparing EEG image data and motor muscle activity data under different motor tasks. The research results show that optical EEG imaging technology can provide more detailed information on brain neural activity and accurately capture the activity patterns of different motor muscles. These results provide new perspectives and methods for further studying the control mechanisms of motor muscles.

Optimization measures for students’ autonomous learning based on deep learning and human-computer interaction technology

Optimization measures for students’ autonomous learning based on deep learning and human-computer interaction technology

Enhanced Hand Gesture Recognition with Surface Electromyogram and Machine Learning.

This study delves into decoding hand gestures using surface electromyography (EMG) signals collected via a precision Myo-armband sensor, leveraging machine learning algorithms. The research entails rigorous data preprocessing to extract features and labels from raw EMG data. Following partitioning into training and testing sets, four traditional machine learning models are scrutinized for their efficacy in classifying finger movements across seven distinct gestures. The analysis includes meticulous parameter optimization and five-fold cross-validation to evaluate model performance. Among the models assessed, the Random Forest emerges as the top performer, consistently delivering superior precision, recall, and F1-score values across gesture classes, with ROC-AUC scores surpassing 99%. These findings underscore the Random Forest model as the optimal classifier for our EMG dataset, promising significant advancements in healthcare rehabilitation engineering and enhancing human-computer interaction technologies.

Flexible and self-healing tannic acid-Fe3O4@MXene-composited dual network hydrogel multifunctional strain sensor

In light of the exponential growth of flexible human-computer interactions and telecommunications, it is becoming increasingly important to develop a high-sensitivity strain sensor to monitor human movement. Herein, the biocompatible, adhesive, and flexible Tannic acid-Fe3O4@MXene-poly(methacryloxyethyl sulfobetaine/acrylamide) (TFM) hydrogel has been prepared to realize the dual-functional applications in strain sensitive sensor and electromagnetic interference (EMI) shielding. The polymer network and 2D conductive filler MXene together construct the conductive network inside the hydrogel. The interpenetrating network and hydrogen bond between poly(acrylamide) and poly(methacryloxyethyl sulfobetaine) chain enhance the mechanical property of hydrogel, which is also improved by the complexation between Fe3O4 and the catechol groups of tannic acid. Benefiting from the excellent electrical conductivity (∼1.33 S/m) and mechanical property of the TFM hydrogels, the strain sensor displays compressive strain (∼77.6 %) and elongation (∼308.5 %), adhesion (∼16 kPa), compression sensitivity (∼42.6 Pa−1), gauge factor (GF = 2.38), and cyclic stability (∼50 cycles). TFM hydrogels can monitor strain distribution sensing, pulse beating and human movement detection. Moreover, the TFM hydrogel exhibits an absorption-predominant EMI shielding performance with an effective value of ∼25.2 dB. This work provides a new inspiration for the development of multifunctional hydrogels in the fields of artificial intelligence, human-computer interaction, and EMI shielding technology.

Multimedia Human-Computer Interaction Technology in Preschool Education

With the increasing of computing and network technology, there has been a rise in children's educational and entertainment software. However, these software applications depend on mouse and keyboard input, which may be tedious and challenging for children. Moreover, these applications fail to capture the interaction between users and computers. This study proposes design principles for children's human-computer interaction based on children's cognitive, psychological, and physiological attributes, and outlines a framework accordingly. The framework integrates natural human-computer interaction technologies, such as physical operation interface, gesture interaction, low-level semantics-based human-computer interaction, and high-level semantics-based human-computer interaction, with the aim of facilitating seamless and intuitive interaction between children and computers. Finally, the results demonstrating the feasibility and effectiveness of the proposed human-computer interaction technology as a child-centered virtual reality application tool.

Feature-Enhanced Multi-Task Learning for Speech Emotion Recognition Using Decision Trees and LSTM

Speech emotion recognition (SER) plays an important role in human-computer interaction (HCI) technology and has a wide range of application scenarios in medical medicine, psychotherapy, and other applications. In recent years, with the development of deep learning, many researchers have combined feature extraction technology with deep learning technology to extract more discriminative emotional information. However, a single speech emotion classification task makes it difficult to effectively utilize feature information, resulting in feature redundancy. Therefore, this paper uses speech feature enhancement (SFE) as an auxiliary task to provide additional information for the SER task. This paper combines Long Short-Term Memory Networks (LSTM) with soft decision trees and proposes a multi-task learning framework based on a decision tree structure. Specifically, it trains the LSTM network by computing the distances of features at different leaf nodes in the soft decision tree, thereby achieving enhanced speech feature representation. The results show that the algorithm achieves 85.6% accuracy on the EMO-DB dataset and 81.3% accuracy on the CASIA dataset. This represents an improvement of 11.8% over the baseline on the EMO-DB dataset and 14.9% on the CASIA dataset, proving the effectiveness of the method. Additionally, we conducted cross-database experiments, real-time performance analysis, and noise environment analysis to validate the robustness and practicality of our method. The additional analyses further demonstrate that our approach performs reliably across different databases, maintains real-time processing capabilities, and is robust to noisy environments.

Hand Gesture Recognition System using Deep Learning

This paper presents a comprehensive exploration of a hand gesture recognition system centered on the YOLOv5 object detection algorithm. The system is meticulously designed to fulfill specific objectives, including real-time gesture detection, high accuracy, scalability, robustness to environmental variability, and seamless integration with other system components. Drawing upon the strengths of YOLOv5, the proposed system aims to provide a reliable and efficient solution for hand gesture recognition, thereby enhancing human- computer interaction and enabling various applications across different domains. The methodology encompasses several stages, including data acquisition, preprocessing, object detection using YOLOv5, integration with other system modules, evaluation, and validation. Through a series of experiments and extensive testing in real-world scenarios, the effectiveness and usability of the system are thoroughly assessed, demonstrating its potential for practical deployment and widespread adoption. Additionally, this paper discusses the implications of the findings, identifies areas for further research and development, and offers insights into the future directions of hand gesture recognition systems utilizing YOLOv5. Overall, this study contributes to advancing the field of human-computer interaction and gesture recognition technology, paving the way for innovative applications and enhanced user experiences in various domains. Keywords— Hand Gesture Recognition, YOLOv5, Real-time Detection, Human-Computer Interaction, Object Detection, Machine Learning

3D facial modeling, animation, and rendering for digital humans: A survey

With the continuous advancement of 3D and human–computer interaction technologies, digital human systems have been widely implemented in our daily lives, such as 3D games, online education, and virtual assistants. This paper provides a comprehensive literature survey of the technologies vitally important to digital humans in 3D facial modeling, animation, and rendering. Firstly, we combed through the research development of 3D face modeling technology and summarized the 3D facial models. Then, we summed up various methods for animating the 3D face based on data from images, videos, and the like. Lastly, we delved deeply into the techniques for 3D face rendering, comparing traditional rendering with neural network rendering. In conclusion, we provided a forward-looking perspective on the current challenges and future development of 3D digital humans.

Semantic distance of icons: Impact on user cognitive performance and a new model for semantic distance classification

With the continuous development of human-computer interaction technologies and the widespread use of graphical interfaces, icons that represent various objects and functions have a particularly important role. This study investigates the effect of semantic distance of icons on cognitive performance through an eye-movement-based experiment which involves a visual search for icons. The findings show that the semantic distance of icons has a significant effect on cognitive performance. A higher cognitive performance is found with semantically close icons which can better capture user attention. In addition, we use eye-movement indicators that are highly correlated with semantic distance, including mean pupil diameter, mean gaze duration and initial gaze time of an AOI, and analyze the objective relationship between these three eye-movement indicators and the semantic distance of icons to establish a dataset. The dataset is used as input for a Gradient Boosting Decision Tree (GBDT), which is a machine learning-based method for classifying the semantic distance of the icons in this study. The output of the GBDT model is classifying the semantic distance as far and close, and the experimental results show that the accuracy of the model reaches 84.28% after a comparison with other types of classifiers, which is in good agreement with the experimental results. Therefore, the model can address the relevant application requirements and simplify the evaluation process of icons to a certain extent, which has great significance in the field of icon design.

Research on Intelligent Wheelchair Multimode Human–Computer Interaction and Assisted Driving Technology

The traditional wheelchair focuses on the “human-chair” motor function interaction to ensure the elderly and people with disabilities’ basic travel. For people with visual, hearing, physical disabilities, etc., the current wheelchairs show shortcomings in terms of accessibility and independent travel for this group. Therefore, this paper develops an intelligent wheelchair with multimodal human–computer interaction and autonomous navigation technology. Firstly, it researches the multimodal human–computer interaction technology of occupant gesture recognition, speech recognition, and head posture recognition and proposes a wheelchair control method of three-dimensional head posture mapping the two-dimensional plane. After testing, the average accuracy of the gesture, head posture and voice control modes of the motorized wheelchair proposed in this study reaches more than 95 percent. Secondly, the LiDAR-based smart wheelchair indoor autonomous navigation technology is investigated to realize the autonomous navigation of the wheelchair by constructing an environment map, using A* and DWA algorithms for global and local path planning, and adaptive Monte Carlo simulation algorithms for real-time localization. Experiments show that the position error of the wheelchair is within 10 cm, and the heading angle error is less than 5° during the autonomous navigation. The multimode human–computer interaction and assisted driving technology proposed in this study can partially compensate and replace the functional deficiencies of the disabled population and improve the quality of life of the elderly and disabled population.

Research on Human Computer Interaction Technology in Virtual Reality Scenes

This paper discusses the current status, challenges and development trends of human computer interaction technology in virtual reality scenarios. Virtual reality technology has gradually become a frontier field of human computer interaction research. This paper analyzes virtual reality technology and human computer interaction technology. By introducing the commonly used human computer interaction hardware equipment and software technology in current virtual reality scenarios, it reveals the actual application of human computer interaction in virtual reality environments. From the three aspects of technology, user experience, and security and privacy protection, this paper analyzes the challenges faced by human computer interaction technology in virtual reality scenarios. In the future, with the continuous advancement of technological innovation, new human-computer interaction technologies will continue to emerge, and the integration of artificial intelligence technology will further enhance the intelligence level of human-computer interaction, making the user interaction experience more natural and convenient.

Literature Review of “Gesture Navigator: AI Based Virtual Mouse”

Abstract: The mouse is one of the wonderful inventions of Human-Computer Interaction (HCI) technology. Currently, wireless mouse or a Bluetooth mouse still uses devices and is not free of devices completely since it uses a battery for power and a dongle to connect it to the PC. In the proposed AI based virtual mouse system, this limitation can be overcome by employing webcam or a built-in camera for capturing of hand gestures and hand tip detection using computer vision. The algorithm used in the system makes use of the machine learning algorithm. Based on the hand gestures, the computer can be controlled virtually and can perform left click, right click, scrolling functions, and computer cursor function without the use of the physical mouse. The algorithm is based on deep learning for detecting the hands. Hence, the suggested system aims to minimize the risk of communicable diseases like COVID-19 transmission by reducing human contact and eliminating the need for additional devices to control the computer system.

Capturing Initial Gaze Attraction in Branded Spaces Through VR Eye-Tracking Technology

This paper leverages recent strides in virtual reality (VR) technology to dissect user gaze patterns and initial responses in a VR-based in-store environment, with a focus on how visual elements impact attention in Human–Computer Interaction (HCI). Specifically, this exploratory study focuses on brand-centric components such as logos and colors, while also considering individual variables such as gender and design background. Employing a mixed-methods approach, the research combines qualitative and quantitative tools. A VR-integrated eye-tracking system records how participants’ gaze behaves in a virtual retail environment. The study collects data from 50 participants in four scenarios: a control store (Case 1), introduction of the brand’s logo signage (Case 2), infusion of the brand’s primary color (Case 3), and a fusion of logo and color (Case 4). Two key eye-tracking metrics—time to first fixation (TTFF) and first fixation duration (FFD)—are harnessed to scrutinize gaze behavior across elements such as wall color, logo signage, wall display fixtures, and media screens. The research unearths statistically significant effects of visual elements on gaze behavior, with particular emphasis on TTFF and FFD. Gender disparities come to light in Cases 2, 3, and 4, affecting FFD on wall color (AOI 1) and TTFF on brand logo signage (AOI 2) in Case 2. Moreover, disparities in design background surface in Cases 1 and 3. Complementing the eye-tracking data and post-experiment interviews capture users’ initial gaze attraction and preferences, offering valuable insights into their rationale. In essence, this study underscores the transformative potential of VR technology and eye-tracking techniques in HCI and furnishes actionable insights for those seeking to elevate brand recognition and consumer engagement within immersive virtual retail settings.

Innovative Application and User Experience of Virtual Reality Technology in Human-Computer Interaction

Virtual Reality (VR) technology has emerged as a transformative force in the realm of Human-computer interaction (HCI), offering novel applications and redefining user experiences. VR games have changed traditional electronic games, providing a human-computer interactive immersive gaming experience. In social media, VR innovative applications allow users to have an immersive conversation through various interactions, giving users a more immersive and real experience. The paper synthesizes recent research findings, highlighting key trends, challenges, and opportunities in the integration of VR into HCI and illustrates the embodiment of immersive experiences while using VR compared to the traditional way. At the same time, the challenges associated with VR technology such as motion sickness, accessibility concerns, ethical considerations, and medical challenges, are also discussed, offering a balanced perspective on the current state of the field. The future development perspective is also mentioned. As technology continues to evolve, we can expect to see even more immersive and interactive experiences that blur the lines between the physical and virtual worlds, offering users new ways to engage with digital content and each other.

Fusion inception and transformer network for continuous estimation of finger kinematics from surface electromyography.

Decoding surface electromyography (sEMG) to recognize human movement intentions enables us to achieve stable, natural and consistent control in the field of human computer interaction (HCI). In this paper, we present a novel deep learning (DL) model, named fusion inception and transformer network (FIT), which effectively models both local and global information on sequence data by fully leveraging the capabilities of Inception and Transformer networks. In the publicly available Ninapro dataset, we selected surface EMG signals from six typical hand grasping maneuvers in 10 subjects for predicting the values of the 10 most important joint angles in the hand. Our model's performance, assessed through Pearson's correlation coefficient (PCC), root mean square error (RMSE), and R-squared (R2) metrics, was compared with temporal convolutional network (TCN), long short-term memory network (LSTM), and bidirectional encoder representation from transformers model (BERT). Additionally, we also calculate the training time and the inference time of the models. The results show that FIT is the most performant, with excellent estimation accuracy and low computational cost. Our model contributes to the development of HCI technology and has significant practical value.

Face and Speech Emotion Recognition System

Abstract: Emotions serve as the cornerstone of human communication, facilitating the expression of one's inner thoughts and feelings to others. Speech Emotion Recognition (SER) represents a pivotal endeavour aimed at deciphering the emotional nuances embedded within a speaker's voice signal. Universal emotions such as neutrality, anger, happiness, and sadness form the basis of this recognition process, allowing for the identification of fundamental emotional states. To achieve this, spectral and prosodic features are leveraged, each offering unique insights into the emotional content of speech. Spectral features, exemplified by the Mel Frequency Cepstral Coefficient (MFCC), provide a detailed analysis of the frequency distribution within speech signals, while prosodic features encompass elements like fundamental frequency, volume, pitch, speech intensity, and glottal parameters, capturing the rhythmic and tonal variations indicative of different emotional states. Through the integration of these features, SER systems can effectively simulate and classify a diverse range of emotional expressions, paving the way for enhanced human-computer interaction and communication technologies