Computer Interaction Technology Research Articles

Application of e-learning and new media teaching platform based on human computer interaction technology in civil and commercial law courses

At present, the traditional teaching mode has been difficult to meet the students' needs for independent learning, interactive learning and diversified learning resources. Therefore, it is of great significance to develop an e-learning platform that can provide rich learning resources and promote student participation and individualized learning to improve the teaching effect. The research adopts the e-learning new media teaching platform based on human–computer interaction technology, and combines the civil and commercial law courses. The e-learning platform provides personalized learning recommendations according to the students' learning conditions, interests and hobbies. It effectively stimulates the enthusiasm and initiative of students to learn. Teachers can monitor and evaluate students' learning situation in real time through the platform, so as to give better guidance and feedback. The e-learning new media teaching platform based on human–computer interaction technology can greatly improve students' learning effect and learning participation. With the help of the platform, students are easier to understand and master the course content, and through online discussion and interaction can better deepen the understanding and application of knowledge. Personalized learning recommendation also enables students to learn according to their own learning needs and improves their learning efficiency.

Multimodal brain-controlled system for rehabilitation training: Combining asynchronous online brain–computer interface and exoskeleton

Background:Traditional therapist-based rehabilitation training for patients with movement impairment is laborious and expensive. In order to reduce the cost and improve the treatment effect of rehabilitation, many methods based on human–computer interaction (HCI) technology have been proposed, such as robot-assisted therapy and functional electrical stimulation (FES). However, due to the lack of active participation of brain, these methods have limited effects on the promotion of damaged nerve remodeling. New method:Based on the neurofeedback training provided by the combination of brain–computer interface (BCI) and exoskeleton, this paper proposes a multimodal brain-controlled active rehabilitation system to help improve limb function. The joint control mode of steady-state visual evoked potential (SSVEP) and motor imagery (MI) is adopted to achieve self-paced control and thus maximize the degree of brain involvement, and a requirement selection function based on SSVEP design is added to facilitate communication with aphasia patients. Comparison with existing methods:In addition, the Transformer is introduced as the MI decoder in the asynchronous online BCI to improve the global perception of electroencephalogram (EEG) signals and maintain the sensitivity and efficiency of the system. Results:In two multi-task online experiments for left hand, right hand, foot and idle states, subject achieves 91.25% and 92.50% best accuracy, respectively. Conclusion:Compared with previous studies, this paper aims to establish a high-performance and low-latency brain-controlled rehabilitation system, and provide an independent and autonomous control mode of the brain, so as to improve the effect of neural remodeling. The performance of the proposed method is evaluated through offline and online experiments.

IHand: Hand Recognition-Based Text Input Method for Wearable Devices

Text input using hand gestures is an essential component of human–computer interaction technology, providing users with a more natural and enriching interaction experience. Nevertheless, the current gesture input methods have a variety of issues, including a high learning cost for users, poor input performance, and reliance on hardware. To solve these problems and better meet the interaction requirements, a hand recognition-based text input method called iHand is proposed in this paper. In iHand, a two-branch hand recognition algorithm combining a landmark model and a lightweight convolutional neural network is used. The landmark model is used as the backbone network to extract hand landmarks, and then an optimized classification head, which can preserve the space structure of landmarks, is designed to classify gestures. When the landmark model fails to extract hand landmarks, a lightweight convolutional neural network is employed for classification. Regarding the way letters are entered, to reduce the learning cost, the sequence of letters is mapped as a two-dimensional layout, and users can type with seven simple hand gestures. Experimental results on the public datasets show that the proposed hand recognition algorithm achieves high robustness compared to state-of-the-art approaches. Furthermore, we tested the performance of users’ initial use of iHand for text input. The results showed that the iHand’s average input speed was 5.6 words per minute, with the average input error rate was only 1.79%.

Phantom sensation: Threshold and quality indicators of a tactile illusion of motion

Utilizing a randomized, blind, controlled experiment, and the ascending method of limits, we determined the minimum amplitude of motion at which individuals perceive a tactile illusion called moving phantom sensation, the perceived level of clarity and continuity of motion. Implementing tactile illusions in virtual/augmented reality, sensory substitution systems, and other human–computer interaction technologies results in interfaces with improved resolution, using two vibrating actuators only. The actuators are attached to the skin in different locations to render a moving phantom sensation. The intensity of vibrations increases in one actuator while decreases in the other according to the envelope of the voltage supply signals. This intensity variation creates the illusion of a vibrating point moving between the actuators. We gradually increased the amplitude of motion until the participant reported perceiving the illusion, for eight values of duration of the stimulus from 0.1 to 6.0 s. Participants perceived the illusion at a minimum amplitude of motion of 20%; being 100% the motion from one actuator to the other. The median level of clarity of the perceived illusion at the minimum amplitude of motion was 2 (not so clear). Finally, we found a positive correlation between duration and continuity of motion.

Inventory of Landslides in the Northern Half of the Taihang Mountain Range, China

The Taihang Mountains are a critical mountain range and geographical boundary in eastern China. Landslide disasters are particularly common in this region and usually cause serious casualties and property damage. However, previous landslide inventories in the region are limited and lack comprehensive landslide cataloguing. To address this gap, the northern half of the Taihang Mountain Range was selected for this study. A landslide database for the area was constructed using multi-temporal high-resolution optical imagery from the Google Earth and human–computer interactive visual interpretation technology. The results indicate that at least 8349 landslides have occurred in the Taihang Mountain Range, with a total landslide area of about 151.61 km2. The size of the landslides varies, averaging about 18,159.23 m2, with the largest landslide covering 2.83 km2 and the smallest landslide only 5.95 m2. The significance of this study lies in its ability to enhance our understanding of the distribution of landslides in the northern half of the Taihang Mountains. Furthermore, it offers valuable data references and supports for landslide assessment, early warning systems, disaster management, and ecological protection efforts.

Intelligent Vehicle Violation Detection System Under Human–Computer Interaction and Computer Vision

In view of the current problems of low detection accuracy, poor stability and slow detection speed of intelligent vehicle violation detection systems, this article will use human–computer interaction and computer vision technology to solve the existing problems. First, the picture data required for the experiment is collected through the Bit Vehicle model dataset, and computer vision technology is used for preprocessing. Then, use Kalman filtering to track and study the vehicle to help better predict the trajectory of the vehicle in the area that needs to be detected; finally, use human–computer interaction technology to build the interactive interface of the system and improve the operability of the system. The violation detection system based on computer vision technology has an accuracy of more than 96.86% for the detection of the eight types of violations extracted, and the average detection is 98%. Through computer vision technology, the system can accurately detect and identify vehicle violations in real time, effectively improving the efficiency and safety of traffic management. In addition, the system also pays special attention to the design of human–computer interaction, provides an intuitive and easy-to-use user interface, and enables traffic managers to easily monitor and manage traffic conditions. This innovative intelligent vehicle violation detection system is expected to help the development of traffic management technology in the future.

A Review of Immersive Technologies, Knowledge Representation, and AI for Human-Centered Digital Experiences

The evolution of digital technologies has resulted in the emergence of diverse interaction technologies. In this paper, we conducted a review of seven domains under a human-centric approach user interface design, human-centered web-based information systems, semantic knowledge representation, X-reality applications, human motion and 3D digitization, serious games, and AI. In this review, we studied these domains concerning their impact on the way we interact with digital interfaces, process information, and engage in immersive experiences. As such, we highlighted the shifts in design paradigms, user-centered principles, and the rise of web-based information systems. The results of such shifts are materialized in modern immersive technologies, semantic knowledge representation, serious games, and the facilitation of artificial intelligence for interactions. Through this exploration, we aimed to assist our understanding of the challenges that lie ahead. The seamless integration of technologies, ethical considerations, accessibility, education for technological literacy, interoperability, user trust, environmental sustainability, and regulatory frameworks are becoming significant. These challenges present opportunities for the future to enrich human experiences while addressing societal needs. This paper lays the groundwork for thoughtful and innovative approaches to the challenges that will define the future of human–computer interaction and information technologies.

Residual networks for text-independent speaker identification: Unleashing the power of residual learning

The human voice, a dynamic signal, conveys valuable information for speaker identification, encompassing gender, age, emotions, and language. In the biometrics industry, identifying voices in real-time amidst diverse accents, tones, and noisy backgrounds is a challenging task. Voice biometry, a complex aspect of speaker identification, is gaining importance in various applications, such as user authentication, attendance systems, forensics, and banking operations, as it eliminates the need for traditional credentials like cards or passwords. Recent advancements in Human–Computer Interaction technology have made conversational tasks technically feasible. Deep Neural Learning approaches, especially Convolutional Deep Neural Networks (CDNN), have emerged as a powerful tool in the field of speech processing, surpassing traditional Speaker Identification methods. This paper introduces a novel approach using 1-Dimensional Convolutional Residual Blocks for audio classification and Speaker Identification, specifically focusing on speaker recognition from spoken Hindi language. The proposed Residual architecture significantly enhances speaker identification, even in low Signal Noise Ratio environments, achieving an impressive accuracy rate of 86.02%. This outperforms traditional Gaussian Mixture Model (GMM) and Feed Forward Back-propagation Network (FFBN) model for the same set of speakers. Future research directions may explore the classification of audio and speaker identification using various acoustic features derived from speech signals.

Roles of computer agents in digital games: effects on learning performance, perceptions, and behaviors

ABSTRACT When encountering difficulties in conventional educational games, learners seldom self-regulate to discover and organize the learning content in the game environment. With the development of the human–computer interaction technology, computer agents are gradually being applied to educational games to provide personalized guidance or support to learners in specific situations. Different roles of computer agents may affect students' learning achievements and feelings in the learning environment. Therefore, to compare the effects of different roles of computer agents on students' learning outcomes in digital games, this study developed a Tutor Agent-based Digital Game (TA-DG), a Peer Agent-based Digital Game (PA-DG) and a Student Agent-based Digital Game (SA-DG). The experimental results showed that the TA-DG and PA-DG learners had better learning achievements than the SA-DG learners. The PA-DG and SA-DG learners had higher motivation than the TA-DG learners. The PA-DG learners had a higher critical thinking tendency than the other two groups. In addition, the SA-DG learners had the lowest cognitive load, followed by the TA-DG and PA-DG learners. This implies that the individual roles of the agents had different impacts during the digital game-based learning process. The findings provide a good reference for developing future digital game-based learning research with personalized support.

Low Resource Language Analysis Using Deep Learning Algorithm for Gender Classification

Voice signals are the essential input source for applications based on human and computer interaction technology. Gender identification through voice signals is one of the most challenging tasks. For voice signal based analysis, deep learning algorithms provide an alternative to traditional and conventional algorithms for classification. To identify the gender through voice signals of female, male and ‘first-time’ transgender, the deep learning algorithm is used to improve the robustness of the identification model with the Mel Frequency Cepstrum Coefficients (MFCC) as a feature of the voice signals. This article presents the identification accuracy of gender with the help of recorded live voice signals. The voice samples of the third gender are recorded in the Hindi language. These Hindi language voice samples of transgender are very low resources and are unavailable at any recognized sources. The simulation results do not depend on the duration of the signals and are text independent. The recurrent neural network – Bidirectional Long Short-term Memory (RNN – BiLSTM) algorithm has been simulated on the recorded voice signals. The simulation outcome is compared with the earlier reported results in the literature. The gender-wise average accuracy of the proposed model is achieved as 91.44%, 94.94%, and 96.11% for males, females, and transgender, respectively, using voice signals. The identification accuracy of transgender is high in comparison to other genders. On the other hand, the average accuracy of the proposed model is obtained as 94.16%.

Research Progress of Human–Computer Interaction Technology Based on Gesture Recognition

Gesture recognition, as a core technology of human–computer interaction, has broad application prospects and brings new technical possibilities for smart homes, medical care, sports training, and other fields. Compared with the traditional human–computer interaction models based on PC use with keyboards and mice, gesture recognition-based human–computer interaction modes can transmit information more naturally, flexibly, and intuitively, which has become a research hotspot in the field of human–computer interaction in recent years. This paper described the current status of gesture recognition technology, summarized the principles and development history of electromagnetic wave sensor recognition, stress sensor recognition, electromyographic sensor recognition, and visual sensor recognition, and summarized the improvement of this technology by researchers in recent years through the direction of sensor structure, selection of characteristic signals, the algorithm of signal processing, etc. By sorting out and comparing the typical cases of the four implementations, the advantages and disadvantages of each implementation and the application scenarios were discussed from the two aspects of dataset size and accuracy. Based on the abovementioned discussion, the problems and challenges of current gesture recognition technology were discussed in terms of the biocompatibility of sensor structures, wearability and adaptability, stability, robustness, and crossover of signal acquisition and analysis algorithms, and the future development directions in this field were proposed.

Semi-Supervised Learning for Robust Emotional Speech Synthesis with Limited Data

Emotional speech synthesis is an important branch of human–computer interaction technology that aims to generate emotionally expressive and comprehensible speech based on the input text. With the rapid development of speech synthesis technology based on deep learning, the research of affective speech synthesis has gradually attracted the attention of scholars. However, due to the lack of quality emotional speech synthesis corpus, emotional speech synthesis research under low-resource conditions is prone to overfitting, exposure error, catastrophic forgetting and other problems leading to unsatisfactory generated speech results. In this paper, we proposed an emotional speech synthesis method that integrates migration learning, semi-supervised training and robust attention mechanism to achieve better adaptation to the emotional style of the speech data during fine-tuning. By adopting an appropriate fine-tuning strategy, trade-off parameter configuration and pseudo-labels in the form of loss functions, we efficiently guided the learning of the regularized synthesis of emotional speech. The proposed SMAL-ET2 method outperforms the baseline methods in both subjective and objective evaluations. It is demonstrated that our training strategy with stepwise monotonic attention and semi-supervised loss method can alleviate the overfitting phenomenon and improve the generalization ability of the text-to-speech model. Our method can also enable the model to successfully synthesize different categories of emotional speech with better naturalness and emotion similarity.

Application of human–computer interaction system based on machine learning algorithm in artistic visual communication

After entering the twenty-first century, with the development of science and technology represented by artificial intelligence, the content of art design and creation is increasingly rich. Its intelligence, interactivity and digital content expression make the relationship between science and technology, art and people more close and also bring new opportunities for the development of interactive art. Among them, human–computer interaction is more and more widely used. With the promotion of this kind of technology, interactive art is no longer based on auditory sensory experience, but on the in-depth study of human nature and comprehensive senses. On this basis, human–computer interaction technology is introduced, thus forming a comprehensive visual art form, which communicates and expresses through strong interaction, initiative and emotion. Based on machine learning technology and human–computer visual interaction technology, this paper explores the art visual communication module based on human–computer interaction system. By analyzing the audience's line of sight and visual differences, we can get the audience satisfaction, so as to improve the expression of the work. Firstly, this paper proposes the design of human–computer interaction visual structure and introduces the basic structure of art visual communication system, the simulation of art visual scene, the process and evaluation index of human–computer interaction recognition, visual recognition effect, etc., based on the needs of visual interaction. Finally, it analyzes the common forms of human–computer interaction in art visual communication, the reconstruction of thinking mode of art visual communication and the development direction of art visual communication. This paper believes that human–computer interaction technology plays a leading role in the development of the art field. It has a variety of functions, displays methods and values. By studying machine learning algorithm and human–computer interaction system, this paper applies it to the field of artistic vision, thus promoting the development of artistic visual communication.

Mixed Reality collaboration environment improves the efficiency of human-centered industrial system: A case study in the mining industry

The emerging information and communication technologies (ICT) promote the intelligence of industrial systems. However, humans still play an important role in industrial systems, and they are required to collaborate in many scenarios to perform complex tasks. Mixed Reality (MR) technology allows the virtual-real fusion and real-time interaction, and has unparalleled advantages in building a collaborative environment. In this study, Theoretical and technical studies have been conducted to provide a universal solution for the industrial applications of MR collaboration. In terms of theory, a framework of the intelligent industrial system is first proposed by combining the human-centered idea and cloud-edge-terminal collaborative architecture. Based on this framework, the MR collaboration environment is then designed. Finally, the MR task scheduling model is developed to illustrate how the MR collaboration environment uses the cloud-edge-terminal collaborative architecture to improve the performance. The technical study is conducted based on the theoretical foundation. The key technologies involved in both co-located and remote collaboration modes, that can be achieved by the MR collaboration environment, are studied. In addition, the human–computer interaction (HCI) technology in MR collaboration is explored. Based on this study, the proposed MR collaboration environment is developed, tested, and evaluated by considering the mining industry as an example. Moreover, user surveys from a human-centered perspective are conducted. The results show that the MR collaboration environment has high applicability in multiple links in industrial scenarios represented by longwall mining. It can improve the quality, increase the efficiency, and reduce the costs. It can also provide a human-centered collaboration experience.

An artificial synapse based on La:BiFeO3 ferroelectric memristor for pain perceptual nociceptor emulation

Recently, memristors with brain-like synaptic behavior have attracted widespread attention, offering unprecedented opportunities for constructing future artificial intelligent bio-inspired electronic systems. Herein, we report a ferroelectric memristor with a Pd/La0.1Bi0.9FeO3(LBFO)/La0.67Sr0.33MnO3/SrTiO3/P+-Si structure, which exhibits the multilevel storage capacity of 12 resistive states, each with a retention time of 2 × 104 s and 200 ns ultra-fast pulse conductance modulation. Moreover, the synaptic properties are successfully mimicked, including paired-pulse facilitation, post-tetanic potentiation, and spike rate-dependent plasticity. In addition, pattern training with high tolerance to fault disturbance and variations is realized in the LBFO memristors array, endowing with highly accurate synaptic weights update capability. More importantly, the device can emulate essential characteristics of pain perception nociceptors with fast operating pulse and low threshold level, and a piezoresistive sensor is further introduced to integrate with LBFO memristor, realistically simulating the artificial pain perception function. Overall, the proposed device provides a promising avenue for the development of electronic skins, neurorobotics, and human–computer interaction technologies.

A VR-based BCI interactive system for UAV swarm control

The traditional Unmanned Aerial Vehicle (UAV) swarm control mainly adopts the ground station method, which is too fixed, and the interaction is difficult to meet the high dynamic task requirements. There is an urgent need for new interaction methods to integrate the advantages of human thinking in dealing with uncertain problems. Nevertheless, brain-computer interface(BCI) technology is directly controlled by thoughts, one of the most promising next-generation human–computer interaction technologies. Therefore, in this study, we innovatively applied the BCI system based on Virtual Reality (VR) to the group UAV and realized a novel and intelligent group control method, which proposes new ideas and paradigms for the control of swarm UAVs in the future. Specifically, this study takes a quadcopter as an example. A modular and extensible multi-quadcopter system was created, and then a visual stimulation 3D VR scene system with a digital twin function was established. On this basis, the BCI system based on the Stable state visual evoked potential (SSVEP) paradigm was adopted for the swarm control of the quadcopter. The experimental results show that the formation control of multi-quadcopter is successfully realized by the subjects using the proposed VR-based BCI interactive system, with an accuracy rate of 90% and a good performance in information transmission rate. In addition, the immersive VR twin system established one-to-one for EEG signal acquisition allows subjects to have a better experience.

Melting the Psychological Boundary: How Interactive and Sensory Affordance Influence Users’ Adoption of Digital Heritage Service

As a result of the post-pandemic situation, enhancing digital heritage services has become one of the key issues for the recovery of tourism. Disruptive innovation in human–computer interaction technology has brought new opportunities for digitalization and intelligent transformation in the contemporary cultural tourism industry. Existing research on the adoption behavior of digital heritage services primarily focuses on users’ assessments of behavior results. There is a considerable gap in research about the interaction and value co-creation between users and digital intelligence services and users’ cognitive construction logic of digital heritage services. Following reciprocal determinism, we propose a conceptual model to deconstruct the detailed transmission path of interactive affordance and sensory affordance to digital heritage adoption. In Study 1, a lab experiment in an AI-assisted smart screen digital heritage service context revealed that interactive affordance and user adoption of digital heritage services were partially mediated by psychological distance. Findings from a between-subject online experiment in Study 2 confirmed that embodied cognition and psychological distance play a parallel intermediary role in the impact of sensory affordance on adoption. In Study 3, a lab experiment in a VR-based digital museum context further verified that information overload moderates the influence of embodied cognition on psychological distance. This research reveals the deep-bounded, rational decision-making logic of digital heritage service adoption and provides significant practical enlightenment for the optimization of the affordance experience.

Spatio-temporal deep forest for emotion recognition based on facial electromyography signals

Emotion recognition is a key component of human–computer interaction technology, for which facial electromyogram (fEMG) is an important physiological modality. Recently, deep-learning-based emotion recognition using fEMG signals has drawn increased attention. However, the ability of effective feature extraction and the demand of large-scale training data are two dominant factors that restrict the performance of emotion recognition. In this paper, a novel spatio-temporal deep forest (STDF) model is proposed to classify three categories of discrete emotions (neutral, sadness, and fear) using multi-channel fEMG signals.The feature extraction module fully extracts effective spatio-temporal features of fEMG signals using a combination of 2D frame sequences and multi-grained scanning. Meanwhile, a cascade forest-based classifier is designed to provide optimal structures for different scales of training data via automatically adjusting the number of cascade layers.The proposed model and five comparison methods were evaluated on our in-house fEMG dataset that included three discrete emotions and three channels of fEMG electrodes with a total of twenty-seven subjects. Experimental results demonstrate that the proposed STDF model achieves the best recognition performance with an average accuracy of 97.41%. Besides, our proposed STDF model can reduced the scale of training data to 50% while the average accuracy of emotion recognition is only reduced by about 5%. Our proposed model offers an effective solution for practical applications of fEMG-based emotion recognition.

2D Camera-Based Air-Writing Recognition Using Hand Pose Estimation and Hybrid Deep Learning Model

Air-writing is a modern human–computer interaction technology that allows participants to write words or letters with finger or hand movements in free space in a simple and intuitive manner. Air-writing recognition is a particular case of gesture recognition in which gestures can be matched to write characters and digits in the air. Air-written characters show extensive variations depending on the various writing styles of participants and their speed of articulation, which presents quite a difficult task for effective character recognition. In order to solve these difficulties, this current work proposes an air-writing system using a web camera. The proposed system consists of two parts: alphabetic recognition and digit recognition. In order to assess our proposed system, two character datasets were used: an alphabetic dataset and a numeric dataset. We collected samples from 17 participants and asked each participant to write alphabetic characters (A to Z) and numeric digits (0 to 9) about 5–10 times. At the same time, we recorded the position of the fingertips using MediaPipe. As a result, we collected 3166 samples for the alphabetic dataset and 1212 samples for the digit dataset. First, we preprocessed the dataset and then created two datasets: image data and padding sequential data. The image data were fed into the convolution neural networks (CNN) model, whereas the sequential data were fed into bidirectional long short-term memory (BiLSTM). After that, we combined these two models and trained again with 5-fold cross-validation in order to increase the character recognition accuracy. In this work, this combined model is referred to as a hybrid deep learning model. Finally, the experimental results showed that our proposed system achieved an alphabet recognition accuracy of 99.3% and a digit recognition accuracy of 99.5%. We also validated our proposed system using another publicly available 6DMG dataset. Our proposed system provided better recognition accuracy compared to the existing system.

Philosophical Research on Enterprise Innovation Ecology Based on a Human–Computer Interaction Mental Model

Enterprise innovation ecology plays a vital role in the operation and development of enterprises. It can enable enterprises to have benign and sustainable progress space in action. At the same time, it can also make the innovation ability of enterprises grow unprecedentedly, which is a crucial factor related to the survival and lifeline of enterprises. However, enterprise managers are often too immersed in concept innovation and structural optimization in the actual innovation and ecological construction of enterprises. Still, they do not pay attention to the problems in actual construction. This would cause the actual operation of the system to be restricted by key influencing factors such as environmental protection policies, making it difficult to achieve effective progress. In this regard, the purpose of this paper is to analyze and discuss the enterprise innovation ecological philosophy based on the human–computer interaction mental model. Therefore, this paper comprehensively considers four key factors that affect the construction of enterprise innovation ecology, namely, the promotion of enterprise technology, the competitiveness of enterprise market, the innovation characteristics of enterprises, and the environmental constraints of innovation ecology, and improves and perfects the innovation ecosystem from these factors. Finally, an efficient ecological model of enterprise innovation was designed, and the scientific construction was carried out through the human–computer interaction mental model combining human–computer interaction technology and cognitive psychology. At the same time, the model is applied to the actual enterprise operation, and the naive Bayesian algorithm is adopted to optimize the relevant performance of the algorithm model. The experimental results showed that the highest sample accuracy of the algorithm reaches 92.9%, which greatly improved the operation efficiency of enterprise innovation ecological construction.