Human-computer Interaction Systems Research Articles

Optimization Analysis of AI Intelligent English Teaching Strategies Based on Computer Virtual Reality Technology

The Internet of Things and other technology breakthroughs have a big impact on how English is taught (IoTV). The investigation of the actual English teaching process and the identification of student features come first. In order to use the interpolation technique with the student image-based feature detection method, we investigate the IoTV reform. As a result, we find a clever algorithm for IoTV that recognises student features more effectively. Artificial intelligence (AI) is the design technique used to create the intelligence algorithm for pupil feature recognition. A comprehensive multifunctional human-computer interaction system utilizes a variety of input and output streams. In addition to the standard computer keyboard, cursor clicking, and screen touching, the most recent speech and facial recognition technology can be employed for data input. Students learned to orally interact with the robot and act as a guide to various destinations. The Multimodal Interaction System for English Education (MMIEE) is an investigation into the use of network and artificial intelligence (AI) in teaching. Recurrent neural network (Rein RNN)-based Reinforced learning is utilized for the perpectual evaluation model to categorize the questions posed by teachers in terms of their content and kind, and to conduct experimental investigation. The results show that the proposed Rein_RNN model achieves 98.6% of accuracy in 4.3sec of mean evaluation time.

Designable high-performance TPU foam strain sensors towards human-machine interfaces

Flexible, breathable, and stable sensors were important in developing human–computer interaction systems and novel wearable devices. However, major challenges were still faced in two aspects, 1) high-performance strain sensors with a large sensing range, and 2) an interaction mode that is both convenient and efficient. Herein, we developed a thermoplastic polyurethane large-skeleton foam (TPU-LS foam) through a steam-induced method. The strain sensor comprised three layers of breathable foam, with CNTs/rGO TPU-LS foam as the inner layer and pure foam as the protective layer. PDMS/TiO2 was sprayed to create a UV-resistant and self-repairing coating, achieving a 66% melanin inhibition rate and a 91% repair rate. The sensor presented a wide work range over 100%, a high gauge factor of 637.8, and long-term durability. We trained machine learning frameworks to recognize different gestures and designed two types of human–computer interaction applications, allowing users to operate apps with gestures.

Facial expression recognition with computer vision

Facial Expression Recognition (FER) is a specialized field within the domains of computer vision and pattern recognition, which is dedicated to the automated identification and examination of facial expressions. Facial expression recognition (FER) has attracted considerable scholarly interest in recent years owing to its diverse array of applications and its potential ramifications across multiple disciplines, such as psychology, human-computer interaction, marketing, and security systems. The objective of this study is to present a thorough examination of the scholarly progression of FER, elucidating the significant achievements, approaches, and obstacles encountered by researchers in this domain. The study presents a selection of databases that are appropriate for Facial Expression Recognition (FER) and conducts a comparative analysis of these databases. The primary methodologies are examined, and recommendations are provided for each stage. In conclusion, this research presents several suggestions for addressing both obstacles and potential in future research endeavors.

Identification Method of Unmanned Aerial Vehicle Graphical Control Strategy Based on Cloud Server

With the rapid development of unmanned aerial vehicle (UAV) technology, UAV has been widely used in agricultural plant protection, electric power inspection, security patrols, and other fields. However, the control system of the UAV is a complex human–computer interaction system, which requires higher requirements in practical applications. Due to differences in hardware design, software development, and other aspects among different manufacturers, these UAV control systems require high hardware requirements, resulting in a long development cycle. At the same time, in practical applications, due to various reasons, there are equipment failures that are difficult to detect and eliminate in a timely manner. This paper used the UAV graphical control strategy identification method based on cloud server technology, and used the support vector machine (SVM) algorithm to analyze its identification accuracy. The research results showed that when other conditions were the same, the number of researchers and experts who were satisfied with the drone trial effect of the cloud server was 42 and 10, respectively, accounting for 84% and 100%. It indicates that they believe that the cloud server can effectively improve the effectiveness of the drone graphical control strategy recognition method, indicating a positive relationship between the two.

Ululate: A Non-Intrusive, Wearable Tongue Gesture Detection System for Human-Computer Interaction

Ululate: A Non-Intrusive, Wearable Tongue Gesture Detection System for Human-Computer Interaction

3D Motion Gesture Control : Gesture Recognition and Adaptation for Human Computer Interaction

Advancements in human-computer interaction (HCI) have paved the way for more intuitive and immersive interfaces. The first part of the paper delves into the fundamental principles of 3D gesture recognition, including sensor technologies, machine learning algorithms, and computer vision techniques. It discusses the challenges associated with accurate recognition in various environmental conditions and the ways in which these challenges are being addressed by researchers. The second part focuses on the adaptation aspect of the technology. It highlights how 3D gesture recognition can be integrated into adaptive HCI systems, enabling personalized and context-aware interactions. These adaptations can range from adjusting the interface layout to suit the user's preferences to dynamically changing the system's behavior based on the user's gestures. Additionally, the paper discusses the potential applications of 3D gesture recognition in fields such as gaming, virtual reality, healthcare, and beyond. It emphasizes the need for continued research to improve accuracy, robustness, and user-friendliness, ultimately driving the widespread adoption of 3D gesture recognition in HCI.

Hardware and Software Design and Implementation of Surface-EMG-Based Gesture Recognition and Control System

The continuous advancement of electronic technology has led to the gradual integration of automated intelligent devices into various aspects of human life. Motion gesture-based human–computer interaction systems offer abundant information, user-friendly functionalities, and visual cues. Surface electromyography (sEMG) signals enable the decoding of muscle movements, facilitating the realization of corresponding control functions. Considering the inherent instability and minuscule nature of sEMG signals, this thesis proposes the integration of a dynamic time regularization algorithm to enhance gesture recognition detection accuracy and real-time system performance. The application of the dynamic time warping algorithm allows the fusion of three sEMG signals, enabling for the calculation of similarity between the sample and the model. This process facilitates gesture recognition and ensures effective communication between individuals and the 3D printed prosthesis. Utilizing this algorithm, the best feature model was generated by amalgamating six types of gesture classification model. A total of 600 training and evaluation experiments were performed, with each movement recognized 100 times. The experimental tests demonstrate that the accuracy of gesture recognition and prosthetic limb control using the temporal dynamic regularization algorithm achieves an impressive 93.75%, surpassing the performance of the traditional threshold control switch.

Convolutional Neural Networks‐Motivated High‐Performance Multi‐Functional Electronic Skin for Intelligent Human‐Computer Interaction

A multi-functional electronic skin (e-skin) with high sensitivity and wide response range is crucial for intelligent human-computer interaction (HCI) system. Here, a multi-functional hybrid e-skin (TPES) integrated by a piezo-capacitive pressure sensor (PCPS) and a triboelectric nanogenerator (TENG) is reported. An ion-gel electrospinning doped with MXene is combined with a micro-structured ion-gel to enrich the structural variations of the overall device, enabling TPES to achieve a high sensitivity of 616.42 kPa−1, a broad response range of 0.6 Pa–1 MPa, and a fast response time of 5.6 ms. The vast surface area of the mixed cellulose foam layer effectively enhances the electrical output of the TENG, making TPES’s application scenarios become more flexible. By further integrating the different functionalities of the TPES with one-dimensional convolutional neural networks and HCI module, a gesture recognition and material perception HCI system is realized. Additionally, benefiting from the excellent performance of the PCPS and TENG, a multi-information recognition HCI system is established, capable of simultaneously recognizing the pressure and surface material of the balls and enabling real-time interaction with a virtual character. In summary, the proposed work offers a novel avenue for practical intelligent HCI system in virtual reality field.

Research on the visualization of information of Chinese traditional music with human-computer interaction from the perspective of metaverse

<p>In the metaverse environment, establish an immersive human-computer interaction system for Chinese traditional music based on virtual reality technology. Design the system’s functionality according to the Y model, and construct a four-layered system architecture. Collect high-quality instructional audio and utilize polygon modeling technology to create contextualized scenes of Chinese traditional music, as well as high-fidelity models of characters and instruments. Implement motion capture through inertial sensor technology for performance action data mapping. Utilize a metaverse engine platform to realize interactive functions and conduct performance optimization. The system is capable of eliciting learners’ intrinsic experiences, enabling interactive self-directed learning and creative exploration of Chinese traditional music performance, demonstrating significant practical value.</p>

Humidity Adaptive Antifreeze Hydrogel Sensor for Intelligent Control and Human-Computer Interaction.

Conductive hydrogels have emerged as ideal candidate materials for strain sensors due to their signal transduction capability and tissue-like flexibility, resembling human tissues. However, due to the presence of water molecules, hydrogels can experience dehydration and low-temperature freezing, which greatly limits the application scope as sensors. In this study, an ionic co-hybrid hydrogel called PBLL is proposed, which utilizes the amphoteric ion betaine hydrochloride (BH) in conjunction with hydrated lithium chloride (LiCl) thereby achieving the function of humidity adaptive. PBLL hydrogel retains water at low humidity (<50%) and absorbs water from air at high humidity (>50%) over the 17 days of testing. Remarkably, the PBLL hydrogel also exhibits strong anti-freezing properties (-80°C), high conductivity (8.18 Sm-1 at room temperature, 1.9 Sm-1 at -80°C), high gauge factor (GF approaching 5.1). Additionally, PBLL hydrogels exhibit strong inhibitory effects against Escherichia coli (E. coli) and Staphylococcus aureus (S. aureus), as well as biocompatibility. By synergistically integrating PBLL hydrogel with wireless transmission and Internet of Things (IoT) technologies, this study has accomplished real-time human-computer interaction systems for sports training and rehabilitation evaluation. PBLL hydrogel exhibits significant potential in the fields of medical rehabilitation, artificial intelligence (AI), and the Internet of Things (IoT).

IMetaTown: A Metaverse System with Multiple Interactive Functions Based on Virtual Reality.

This work aims to pioneer the development of a real-time interactive and immersive Metaverse Human-Computer Interaction (HCI) system leveraging Virtual Reality (VR). The system incorporates a three-dimensional (3D) face reconstruction method, grounded in weakly supervised learning, to enhance player-player interactions within the Metaverse. The proposed method, two-dimensional (2D) face images, are effectively employed in a 2D Self-Supervised Learning (2DASL) approach, significantly optimizing 3D model learning outcomes and improving the quality of 3D face alignment in HCI systems. The work outlines the functional modules of the system, encompassing user interactions such as hugs and handshakes and communication through voice and text via blockchain. Solutions for managing multiple simultaneous online users are presented. Performance evaluation of the HCI system in a 3D reconstruction scene indicates that the 2DASL face reconstruction method achieves noteworthy results, enhancing the system's interaction capabilities by aiding 3D face modeling through 2D face images. The experimental system achieves a maximum processing speed of 18 frames of image data on a personal computer, meeting real-time processing requirements. User feedback regarding social acceptance, action interaction usability, emotions, and satisfaction with the VR interactive system reveals consistently high scores. The designed VR HCI system exhibits outstanding performance across diverse applications.

An Intelligent Human-Computer Interaction System Based on Wireless Self-Powered Sensor for Motion Monitoring

With the development of the Internet of Things and information technology, large numbers of self-powered flexible sensors are needed to monitor the state of human motions. Triboelectric nanogenerators (TENGs) have attracted attention in sports monitoring due to its low cost and materials versatility. An Eco-flex-PTFE TENG (PE-TENG) system has been designed to real-time monitor and analyze the position and force changes of soccer players when kicking the ball on the outside, inside, and front of the foot, and to evaluate the technical motions. The PE-TENG system includes a PE-TENG sensing unit, Bluetooth transmission end, and an upper computer digital signal receiving end. The friction layer of the PE-TENG sensing unit is composed of Eco-flex and PTFE. PE-TENG system has the advantages of small size, light weight, stable working performance, and does not require external power supply. On the basis of the PE-TENG system, designed a human-computer interaction system to achieve the combination of real and virtual kicking skills for analyzing soccer players’ kicking motions. The system provides a new direction for scientific monitoring of soccer training, and entertainment of soccer players, while also accelerating the broad application prospects of self-powered wearable sports sensors in the field of sports monitoring.

An in-sensor humidity computing system for contactless human-computer interaction.

Being capable of processing large amounts of redundant data and decreasing power consumption, in-sensor computing approaches play significant roles in neuromorphic computing and are attracting increasing interest in perceptual information processing. Herein, we proposed a high performance humidity-sensitive memristor based on a Ti/graphene oxide (GO)/HfOx/Pt structure and verified its potential for application in remote health management and contactless human-machine interfaces. Since GO possesses abundant hydrophilic groups (carbonyl, epoxide, and hydroxyl), the memristor shows a high humidity sensitivity, fast response, and wide response range. By utilizing the proton-modulated redox reaction, humidity exposure to the memristor induces a dynamic change in the switching between high and low resistance states, ensuring essential synaptic learning functions, such as paired-pulse facilitation, spike number-dependent plasticity, and spike amplitude-dependent plasticity. More importantly, based on the humidity-induced salient features originating from the abundant hydrophilic functional groups in GO, we have implemented a noncontact human-machine interface utilizing the respiratory mode in humans, demonstrating the potential of promoting health monitoring applications and effectively blocking virus transmission. In addition, the high recognition accuracy of contactless handwriting in a 5 × 5 array artificial neural network was successfully achieved, which is attributed to the excellent emulated synaptic behaviors. This study provides a feasible method to develop an excellent humidity-sensitive memristor for constructing efficient in-sensor computing for application in health management and contactless human-computer interaction.

DV Emotion Net: An Integrated Multimodal Approach for Emotion Recognition

This study introduces a novel approach to emotion recognition by amalgamating information from heterogeneous modalities, specifically audio and video. We employed techniques such as energy, zero crossing rate, and Mel-Frequency Cepstral Coefficients (MFCC) for audio feature extraction, which showed promising results. For video feature extraction, spatialtemporal Gaussian kernels were used to organize video frames within a linear scale space, followed by the application of a Gaussian-weighted function to the second momentum matrix for further feature extraction. The Multimodal Feature Aggregation (MFA) fusion method was employed to unify audio and video features, resulting in a comprehensive dataset. Evaluation through the Fusion of Emotion Recognition Convolutional Neural Network (FERCNN) model, supported by the "TPU VM v3-8" accelerator TPU is a Tensor Processing Unit, showcased notable performance improvements. Using the RAVDESS and CREMAD datasets, accuracies of 94.66%, 95.82%, and 94.36% in the RAVDESS dataset and 79.45%, 96.62%, and 70.14% in the CREMAD dataset for audio, video, and multimodal modalities, respectively, were achieved. These outcomes surpass the capabilities of existing multimodal systems, underscoring the efficacy of our proposed approach. Emotion recognition, particularly through multimodal means, plays a critical role in various domains, including human-computer interfaces, healthcare, legal proceedings, and entertainment. Fusing Audio and Video Modalities to Elevate Human-Computer Interaction and Intelligent System Performance is essential for enhancing communication within these domains. The proposed model is termed "DualVision EmotionNet: DV EmotionNet".

GraphCFC: A Directed Graph Based Cross-Modal Feature Complementation Approach for Multimodal Conversational Emotion Recognition

Emotion Recognition in Conversation (ERC) plays a significant part in Human-Computer Interaction (HCI) systems since it can provide empathetic services. Multimodal ERC can mitigate the drawbacks of uni-modal approaches. Recently, Graph Neural Networks (GNNs) have been widely used in a variety of fields due to their superior performance in relation modeling. In multimodal ERC, GNNs are capable of extracting both long-distance contextual information and inter-modal interactive information. Unfortunately, since existing methods such as MMGCN directly fuse multiple modalities, redundant information may be generated and diverse information may be lost. In this work, we present a directed Graph based Cross-modal Feature Complementation (GraphCFC) module that can efficiently model contextual and interactive information. GraphCFC alleviates the problem of heterogeneity gap in multimodal fusion by utilizing multiple subspace extractors and Pair-wise Cross-modal Complementary (PairCC) strategy. We extract various types of edges from the constructed graph for encoding, thus enabling GNNs to extract crucial contextual and interactive information more accurately when performing message passing. Furthermore, we design a GNN structure called GAT-MLP, which can provide a new unified network framework for multimodal learning. The experimental results on two benchmark datasets show that our GraphCFC outperforms the state-of-the-art (SOTA) approaches.

Preventing the Risks of Inadequate Human-Computer Interaction (HCI) Systems in Manned Spacecraft Missions: Toward an International Public Policy Framework Proposal

Preventing the Risks of Inadequate Human-Computer Interaction (HCI) Systems in Manned Spacecraft Missions: Toward an International Public Policy Framework Proposal

Across Sessions and Subjects Domain Adaptation for Building Robust Myoelectric Interface.

Gesture interaction via surface electromyography (sEMG) signal is a promising approach for advanced human-computer interaction systems. However, improving the performance of the myoelectric interface is challenging due to the domain shift caused by the signal's inherent variability. To enhance the interface's robustness, we propose a novel adaptive information fusion neural network (AIFNN) framework, which could effectively reduce the effects of multiple scenarios. Specifically, domain adversarial training is established to inhibit the shared network's weights from exploiting domain-specific representation, thus allowing for the extraction of domain-invariant features. Effectively, classification loss, domain diversence loss and domain discrimination loss are employed, which improve classification performance while reduce distribution mismatches between the two domains. To simulate the application of myoelectric interface, experiments were carried out involving three scenarios (intra-session, inter-session and inter-subject scenarios). Ten non-disabled subjects were recruited to perform sixteen gestures for ten consecutive days. The experimental results indicated that the performance of AIFNN was better than two other state-of-the-art transfer learning approaches, namely fine-tuning (FT) and domain adversarial network (DANN). This study demonstrates the capability of AIFNN to maintain robustness over time and generalize across users in practical myoelectric interface implementations. These findings could serve as a foundation for future deployments.

MCD-LightGBM System for Intelligent Analyzing Heterogeneous Clinical Drug Therapeutic Effects.

Causal effect estimation of individual heterogeneity is a core issue in the field of causal inference, and its application in medicine poses an active and challenging problem. In high-risk decision-making domain such as healthcare, inappropriate treatments can have serious negative impacts on patients. Recently, machine learning-based methods have been proposed to improve the accuracy of causal effect estimation results. However, many of these methods concentrate on estimating causal effects of continuous outcome variables under binary intervention conditions, and give less consideration to multivariate intervention conditions or discrete outcome variables, thus limiting their scope of application. To tackle this issue, we combine the double machine learning framework with Light Gradient Boosting Machine (LightGBM) and propose a double LightGBM model. This model can estimate binary causal effects more accurately and in less time. Two cyclic structures were added to the model. Data correction method was introduced and improved to transform discrete outcome variables into continuous outcome variables. Multivariate Cyclic Double LightGBM model (MCD-LightGBM) was proposed to intelligently estimate multivariate treatment effects. A visual human-computer interaction system for heterogeneous causal effect estimation was designed, which can be applied to different types of data. This paper reports that the system improved the Logarithm of the Minimum Angle of Resolution (LogMAR) of visual acuity change after Vascular Endothelial Growth Factor (anti-VEGF) treatment in patients with diabetic macular degeneration. The improvement was observed in two clinical problems, from 0.05 to 0.33, and the readmission rate of diabetic patients after cure was reduced from 48.4% to 10.5%. The results above demonstrate the potential of the proposed system in predicting heterogeneous clinical drug treatment effects.

Multimedia Interactive Technology for Improving Language Perception in English Teaching and Learning

Abstract Based on Direct Show software, this paper acquires English teaching videos and stores them in the corresponding files. As the position of the teaching projection screen may be different in height, it will lead to the phenomenon that the captured image will be tilted, and it is necessary to use the affine transformation and perspective transformation methods to realize the correction of the teaching projection image. In order to promote the general application of multimedia interactive technology in the teaching process, dynamic gesture tracking with Kalzmann filtering is utilized to construct a non-contact human-computer interaction system, which is applied to the process of English teaching, thus designing an English teaching model incorporating multimedia interactive technology and analyzing the effect of the model. It was found that the mean score of the experimental group’s pre-experimental evaluation was 19.225, and the mean score for the post-experimental assessment was 23.087, t=2.274, P (two-sided)=0.018&lt;0.05 at the 0.05 significance level, which is a significant difference. It shows that compared with traditional English teaching methods, the teaching mode that integrates multimedia interactive technology can effectively improve the English language perception ability of college students in China.

Research on Innovative Design of Interactive Learning Environment for Architectural Education Integrating Artificial Intelligence and Virtual Reality Technology

Abstract To provide new context-creation tools in architectural education to give students a full sense of immersion and presence. This paper discusses the integration of artificial intelligence and virtual reality technology in architectural education, and then the integration of artificial intelligence and virtual reality technology proposed architecture education virtual learning environment human-computer interaction system. Through the experimental study, in the application effect satisfaction analysis, the student satisfaction in the virtual reality learning environment in stimulating students’ learning interest is 80%, and 72% of the students agree to learn architectural expertise in the virtual reality learning environment.