Human-computer Interaction Technology Research Articles

Bridging Communication Gaps: A Comprehensive Survey on Sign Language Hand Gesture Smart Recognition Technology

Abstract: Efficient communication is crucial in human interaction, yet individuals with language and hearing challenges encounter obstacles. This article centers on Nepali Sign Language (NSL) and introduces the Hand Gesture Smart Recognition initiative, with the goal of translating NSL into English. The research explores hand gesture recognition technology, emphasizing the cultural subtleties inherent in both vision-based and glove-based approaches. Recent advancements, such as the integration of compact models and CNNs, signify advancements in gesture recognition. The article offers valuable insights into the development and application of this technology, fostering optimism regarding the future potential of culturally suitable human-computer interaction technologies.

Neural interface technology for human-computer interaction

Brain-Computer Interfaces (BCI) offer a direct communication channel between the brain and external devices, marking a pivotal convergence of neuroscience and technology. Neural activities, essentially electrical impulses, can be captured either invasively, with methods such as Electrocorticography (ECoG) which require surgical implantation, or non-invasively using techniques like Electroencephalography (EEG) that operate externally. Once acquired, raw neural data undergoes processing; external and physiological noises are filtered out, and meaningful patterns or neural fingerprints are extracted. Modern BCIs then employ machine learning, specifically deep learning, to translate these cleaned neural patterns into discernible commands, with continuous feedback loops enhancing system adaptability. These decoded signals can control varied devices, from medical-grade robotic limbs to cursors on screens. BCIs have transformative applications across sectors: theyre pivotal in neurorehabilitation after brain injuries, providing feedback where traditional methods might fail; theyre integrated with virtual reality for immersive feedback; they revolutionize cognitive training and meditation practices; and theyre finding a foothold in high-risk sectors like deep-sea exploration and military operations. Additionally, research on tactile Event-Related Potential (ERP)-based BCIs emphasizes the importance of congruent Control-Display Mapping for efficient user experience. However, with these advancements come ethical concerns, such as the potential invasion of the privacy of ones thoughts, challenges to human identity and autonomy, societal disparities in access, and health implications.

Residual networks for text-independent speaker identification: Unleashing the power of residual learning

The human voice, a dynamic signal, conveys valuable information for speaker identification, encompassing gender, age, emotions, and language. In the biometrics industry, identifying voices in real-time amidst diverse accents, tones, and noisy backgrounds is a challenging task. Voice biometry, a complex aspect of speaker identification, is gaining importance in various applications, such as user authentication, attendance systems, forensics, and banking operations, as it eliminates the need for traditional credentials like cards or passwords. Recent advancements in Human–Computer Interaction technology have made conversational tasks technically feasible. Deep Neural Learning approaches, especially Convolutional Deep Neural Networks (CDNN), have emerged as a powerful tool in the field of speech processing, surpassing traditional Speaker Identification methods. This paper introduces a novel approach using 1-Dimensional Convolutional Residual Blocks for audio classification and Speaker Identification, specifically focusing on speaker recognition from spoken Hindi language. The proposed Residual architecture significantly enhances speaker identification, even in low Signal Noise Ratio environments, achieving an impressive accuracy rate of 86.02%. This outperforms traditional Gaussian Mixture Model (GMM) and Feed Forward Back-propagation Network (FFBN) model for the same set of speakers. Future research directions may explore the classification of audio and speaker identification using various acoustic features derived from speech signals.

Retracted: Multimodal Opera Performance Form Based on Human-Computer Interaction Technology

Retracted: Multimodal Opera Performance Form Based on Human-Computer Interaction Technology

Application of Deep Neural Network and Human-Computer Interaction Technology Based on Multimodal Perception in Art Design

Abstract The essence of art design is the process of emotional interaction with humans. In this paper, emotions are classified using multimodal fusion features, and abstract fusion features are obtained by sampling the multimodal matching tensor using average pooling. The matching fusion matrix in the tensor operator is used to convert from two-modal to multi-modal matching. Virtual interaction model in art design controls the design objectives, and the optimized virtual world is constructed by using virtual reality technology, so as to build an immersive art design model. Finally, a study was conducted to examine the impact of the use of emotion perception and interaction technology in art product design with students from the School of Design and Art, University of G. The results show that the liking degree of three-color matching in the cognitive experiment test is 0.307, which is higher than the liking degree of two-color matching of 0.223, indicating that overall three-color matching samples are more popular in art design. This study provides effective evaluation and guidance for designing art products that are emotional and innovative.

Mouse Control using Hand Gesture

The mouse, that handy device we use to control our computers, has come a long way in Human-Computer Interaction technology. Even with wireless or Bluetooth mice, we still rely on batteries and dongles to connect them to our PCs. But imagine a new AI-powered virtual mouse system. Instead of needing physical devices, it would use your webcam or built-in camera to track your hand movements and gestures. Using clever computer vision and machine learning algorithms, it could interpret these gestures to control your computer—clicking, scrolling, and moving the cursor—all without an actual mouse. This system, based on deep learning to recognize hand movements, could even help reduce the spread of COVID-19 by removing the need for physical contact and devices to control your computer.

A Survey of Motion Data Processing and Classification Techniques Based on Wearable Sensors

The rapid development of wearable technology provides new opportunities for action data processing and classification techniques. Wearable sensors can monitor the physiological and motion signals of the human body in real-time, providing rich data sources for health monitoring, sports analysis, and human-computer interaction. This paper provides a comprehensive review of motion data processing and classification techniques based on wearable sensors, mainly including feature extraction techniques, classification techniques, and future development and challenges. First, this paper introduces the research background of wearable sensors, emphasizing their important applications in health monitoring, sports analysis, and human-computer interaction. Then, it elaborates on the work content of action data processing and classification techniques, including feature extraction, model construction, and activity recognition. In feature extraction techniques, this paper focuses on the content of shallow feature extraction and deep feature extraction; in classification techniques, it mainly studies traditional machine learning models and deep learning models. Finally, this paper points out the current challenges and prospects for future research directions. Through in-depth discussions of feature extraction techniques and classification techniques for sensor time series data in wearable technology, this paper helps promote the application and development of wearable technology in health monitoring, sports analysis, and human-computer interaction.

Research on the Integration of Computer Human-computer Interaction Technology and Multimedia Medical Big Data Interaction System

Research on the Integration of Computer Human-computer Interaction Technology and Multimedia Medical Big Data Interaction System

Active Rehabilitation Gloves Based on Brain-Computer Interfaces and Deep Learning

Cerebral stroke is the second leading cause of death and the third leading cause of death and disability in the world, and more than half of these patients have hand dysfunction, making hand rehabilitation an urgent challenge. In this study, a system for hand rehabilitation therapy for stroke patients was designed using novel human-computer interaction technology. The system combines a brain-computer interface, a deep learning algorithm and a rehabilitation glove, and designs an electroencephalogram (EEG) signal acquisition card and a rehabilitation glove to realise the application of motor imagery therapy to the active rehabilitation of patients' hands. On the brain-computer interface-based motor imagery experiments, the Long Short Term Memory (LSTM) recurrent neural network algorithm designed in this study achieves an average accuracy of 95.78% for the classification accuracy of mental tasks in seven motor imagery modes, which is important for the active rehabilitation of patients with hand function based on motor imagery-driven rehabilitation.

A REVIEW ON IMPLEMTATION OF SIGN LANGUAGE TRANSLATOR

As artificial intelligence technology has advanced, it has become commonplace to employee gesture detection to control virtual objects. The suggested system in this research is a hand gesture- controlled virtual mouse that uses AI algorithms to recognize hand gestures and transform them into mouse movements. People who have trouble using a conventional mouse or keyboard can use the system to provide an alternate interface. The suggested method takes pictures of the user's hand with a camera, which an AI program then utilizes to identify the motions the user is making. Since the development of computer technology, the method for constructing a process of human- computer interaction is advancing. The mouse is one of the best pieces of HCI (Human- Computer Interaction) technology ever created. This article suggests an HCI-based virtual mouse system that makes use of hand movements and computer vision. webcam or built-in camera recordings of gestures that have been subjected to a color segmentation and detection procedure. This system uses a webcam or an integrated camera to capture frames. It then processes the frames to make them trackable, identifies various user motions, and executes mouse functions. In order to make the interaction more effective and dependable, this study suggests a vision-based system to control various mouse behaviors, such as left and right clicking, using hand gestures. This system uses a webcam or an integrated camera to capture frames. It then processes the frames to make them trackable, identifies various user motions, and executes mouse functions. Therefore, the suggested mouse solution removes the need for a device to use a mouse. Therefore, it can be seen that the development of HCI technology is advantageous. Keywords- HCI(Human Computer Interaction), Hand Gesture, Gesture Recognition, OpenCV, Media-pipe, pyAutoGUI.

A IMPLEMENTATION ON IMPLEMTATION OF VISION GLIDE TECHNIQUE FOR SMOOTH NAVIGATION WITH CAMERA BASED VIRTUAL MOUSE

As artificial intelligence technology has advanced, it has become commonplace to employ hand gesture detection to control virtual objects. The suggested system in this research is a hand gesture-controlled virtual mouse that uses AI algorithms to recognize hand gestures and transform them into mouse movements. People who have trouble using a conventional mouse or keyboard can use the system to provide an alternate interface. The suggested method takes pictures of the user's hand with a camera, which an AI program then utilizes to identify the motions the user is making. Since the development of computer technology, the method for constructing a process of human- computer interaction is advancing. The mouse is one of the best pieces of HCI (Human-Computer Interaction) technology ever created. This article suggests an HCI-based virtual mouse system that makes use of hand movements and computer vision. webcam or built-in camera recordings of gestures that have been subjected to a color segmentation and detection procedure. This system uses a webcam or an integrated camera to capture frames. It then processes the frames to make them trackable, identifies various user motions, and executes mouse functions. In order to make the interaction more effective and dependable, this study suggests a vision-based system to control various mouse behaviors, such as left and right clicking, using hand gestures. This system uses a webcam or an integrated camera to capture frames. It then processes the frames to make them trackable, identifies various user motions, and executes mouse functions. Therefore, the suggested mouse solution removes the need for a device to use a mouse. Therefore, it can be seen that the development of HCI technology is advantageous. Keywords- HCI (Human Computer Interaction), Hand Gesture, Gesture Recognition, OpenCV, Media-pipe, pyAutoGUI.

A IMPLEMENTATION ON SIGN LANGUAGUAGE TRANSLATOR

we propose an innovative Human-Computer Interaction (HCI) system: a hand gesture-controlled virtual mouse powered by AI algorithms. Utilizing computer vision through webcam or built-in camera recordings, the system captures and processes hand gestures, translating them into precise mouse movements. This offers an alternative interface, particularly beneficial for individuals facing challenges with conventional input devices. The vision-based approach enhances the system's reliability, enabling diverse mouse behaviors like left and right clicking. By eliminating the need for a physical mouse, this HCI solution marks a significant stride in advancing technology for seamless human-computer interaction. Keywords: HCI, Hand Gesture, Gesture Recognition, OpenCV, Media-pipe, pyAutoGUI.

Application of Human-Computer Interaction and Digital Image Processing Technology in Virtual Simulation Game Design

Abstract This paper first analyzes the design process, image imaging, and view reconstruction of virtual simulation game scenes and discusses the key technologies used in virtual simulation games. Then, the human-computer interaction techniques of gesture, pose, voice, and data interaction are combined to design a multi-touch detection algorithm based on human-computer interaction. Finally, the edge detection algorithm is combined with the edge detection algorithm to detect the edge of the image, and after the displacement conversion and extraction, the processing process of the digital image is completed. After the performance test, the usability of pose interaction, the accuracy of speech recognition and the timeliness of data interaction of the virtual simulation game are between 0.8 and 1.0, and the overall human-computer interaction capability is highly evaluated.

Application of dynamic time warping optimization algorithm in speech recognition of machine translation

Speech recognition is the foundation of human-computer interaction technology and an important aspect of speech signal processing, with broad application prospects. Therefore, it is very necessary to recognize speech. At present, speech recognition has problems such as low recognition rate, slow recognition speed, and severe interference from other factors. This paper studied speech recognition based on dynamic time warping (DTW) algorithm. By introducing speech recognition, the specific steps of speech recognition were understood. Before performing speech recognition, the speech that needs to be recognized needs to be converted into a speech sequence using an acoustic model. Then, the DTW algorithm was used to preprocess speech recognition, mainly by sampling and windowing the speech. After preprocessing, speech feature extraction was carried out. After feature extraction was completed, speech recognition was carried out. Through experiments, it can be found that the recognition rate of speech recognition on the basis of DTW algorithm was very high. In a quiet environment, the recognition rate was above 93.85 %, and the average recognition rate of the 10 selected testers was 95.8 %. In a noisy environment, the recognition rate was above 91.4 %, and the average recognition rate of the 10 selected testers was 93 %. In addition to high recognition rate, DTW based speech recognition also had a very fast speed for vocabulary recognition. Based on the DTW algorithm, speech recognition not only has a high recognition rate, but also has a faster recognition speed.

Deep neural networks for multimodal perception and human-computer interaction technology in art design

Abstract The first part of this paper examines the aesthetic and application advantages of art design using human-computer interaction technology and develops a multimodal perceptual human-computer interaction system for art design. Multimodal data is obtained using multi-scale convolutional kernels for acoustic feature extraction and deep convolutional neural networks for multiple interaction image feature fusion. Finally, a test analysis is conducted to verify the system's effectiveness in this paper. According to the results, the system has an average wake-up success rate of 99.51% and a wake-up response time of 0.3665 seconds. Implementing human-computer interaction technology and deep neural networks in art design is effective and promotes the development of art design.

Retracted: Application of Multimedia Human-Computer Interaction Technology in Preschool Children Drama Education

Retracted: Application of Multimedia Human-Computer Interaction Technology in Preschool Children Drama Education

A REVIEW ON IMPLEMTATION OF VISION GLIDE TECHNIQUE FOR SMOOTH NAVIGATION WITH CAMERA BASED VIRTUAL MOUSE

As artificial intelligence technology has advanced, it has become commonplace to employ hand gesture detection to control virtual objects. The suggested system in this research is a hand gesture-controlled virtual mouse that uses AI algorithms to recognize hand gestures and transform them into mouse movements. People who have trouble using a conventional mouse or keyboard can use the system to provide an alternate interface. The suggested method takes pictures of the user's hand with a camera, which an AI program then utilizes to identify the motions the user is making. Since the development of computer technology, the method for constructing a process of human-computer interaction is advancing. The mouse is one of the best pieces of HCI (Human-Computer Interaction) technology ever created. This article suggests an HCI-based virtual mouse system that makes use of hand movements and computer vision. webcam or built-in camera recordings of gestures that have been subjected to a color segmentation and detection procedure. This system uses a webcam or an integrated camera to capture frames. It then processes the frames to make them trackable, identifies various user motions, and executes mouse functions. In order to make the interaction more effective and dependable, this study suggests a vision-based system to control various mouse behaviors, such as left and right clicking, using hand gestures. This system uses a webcam or an integrated camera to capture frames. It then processes the frames to make them trackable, identifies various user motions, and executes mouse functions. Therefore, the suggested mouse solution removes the need for a device to use a mouse. Therefore, it can be seen that the development of HCI technology is advantageous. Keywords- HCI (Human Computer Interaction), Hand Gesture, Gesture Recognition, OpenCV, Media-pipe, pyAutoGUI.

The Relationship between Users’ Behavior and Their Flow Experience in Gamified Systems

Modeling users' experience in gameful systems is one of the main contemporary challenges in the field of human-computer interaction. One of the most desired and complex experiences to be identified is the flow experience (i.e., challenge-skill balance, action-awareness merging, clear goals, unambiguous feedback, concentration, sense of control, loss of self-consciousness, transformation of time, and autotelic experience). Facing this challenge, we conducted a quantitative study (N = 313) based on structural equation modeling, aiming to model and predict the users' flow experience through their behavior (represented by performance-related, interaction with gamification, as well as the time they take in different actions) in the system. The main results indicate that i) gamification (i.e., doing well in points, badges, and leader-board) was positively related to users' experience of good challenge-skill balance, ii) whereas it was negatively related to users' concentration. Thirdly iii) user performance was positively related to users' concentration. However, overall, the results indicate that while associations between user behavior and flow experience could be established, there remains future work to be done to fully explain user flow experience while using a system. Our study contributes to the fields of human-computer interaction, gamification, and educational technologies, especially through insights related to modeling and predicting flow experiences in gameful systems through behavior data.

Emotional Experience during Human-Computer Interaction: A Survey

As human-computer interaction (HCI) technology becomes more and more integrated into our daily life, increasing attention has been drawn towards the interaction experience in addition to HCI efficiency. In the present study, we conducted a survey to explore context-specific emotional experience in HCI. Four hundred participants were recruited to report the frequency of their emotional experiences on 44 fine-grained emotion items in six representative HCI scenarios. Compared with six matched human-human interaction (HHI) scenarios used as control, the HCI scenarios were in general more frequently associated with negative emotions, and less frequently associated with positive emotions, especially when computer served as a tool for communication with other people. Furthermore, the 44 emotional experience items in HCI were summarized as five factors, representing low-arousal focused, positively engaged, emotionally empathetic, high-arousal negative and frustratingly confused. Our study presents a comprehensive overview of context-specific emotional experience in human-computer interactions and provides a framework for emotion evaluation in HCI applications.

Emotional Classification Based on Facial Expression Recognition Using Convolutional Neural Network Method

In recent years, the development of human-computer interaction technology has reached remarkable levels, particularly in the field of facial expression recognition. This technology utilizes human facial images to identify and classify emotional expressions such as happiness, sadness, fear, and more through computer image processing. Active research in facial expression recognition yields substantial benefits for individual and societal advancement, especially in the context of its application within Smart City environments. This study demonstrates that well- configured Convolutional Neural Network (CNN) models empowered by TensorFlow exhibit higher accuracy compared to models utilizing PyTorch. The TensorFlow model achieves the highest accuracy of 93% in recognizing emotional expressions, whereas the PyTorch model achieves 69% accuracy. The TensorFlow model also displays lower accuracy loss and shorter training times compared to the PyTorch model. In the context of calculating happiness indices within Smart City environments, the appropriate choice of technology significantly influences measurement accuracy and efficiency. Therefore, the TensorFlow platform, proven to deliver superior performance in this study, can be a strategic choice for integrating facial expression detection technology into happiness index measurements in such locations