Field Of Human-computer Interaction Research Articles

Human action recognition with transformer based on convolutional features

As one of the key research directions in the field of computer vision, human action recognition has a wide range of practical application values and prospects. In the fields of video surveillance, human-computer interaction, sports analysis, and healthcare, human action recognition technology shows a broad application prospect and potential. However, the diversity and complexity of human actions bring many challenges, such as handling complex actions, distinguishing similar actions, coping with changes in viewing angle, and overcoming occlusion problems. To address the challenges, this paper proposes an innovative framework for human action recognition. The framework combines the latest pose estimation algorithms, pre-trained CNN models, and a Vision Transformer to build an efficient system. The first step involves utilizing the latest pose estimation algorithm to accurately extract human pose information from real RGB image frames. Then, a pre-trained CNN model is used to perform feature extraction on the extracted pose information. Finally, the Vision Transformer model is applied for fusion and classification operations on the extracted features. Experimental validation is conducted on two benchmark datasets, UCF 50 and UCF 101, to demonstrate the effectiveness and efficiency of the proposed framework. The applicability and limitations of the framework in different scenarios are further explored through quantitative and qualitative experiments, providing valuable insights and inspiration for future research.

Speaker-aware cognitive network with cross-modal attention for multimodal emotion recognition in conversation

Emotion recognition in conversation (ERC) has gained considerable attention owing to its extensive applications in the field of human-computer interaction. However, previous models have had certain limitations in exploring the potential emotional relationships within the conversation due to their inability to fully leverage speaker information. Additionally, information from various modalities such as text, audio, and video can synergistically enhance and supplement the analysis of emotional context within the conversation. Nonetheless, effectively fusing multimodal features to understand the detailed contextual information in the conversation is challenging. This paper proposes a Speaker-Aware Cognitive network with Cross-Modal Attention (SACCMA) for multimodal ERC to effectively leverage multimodal information and speaker information. Our proposed model primarily consists of the modality encoder and the cognitive module. The modality encoder is employed to fuse multimodal feature information from speech, text, and vision using a cross-modal attention mechanism. Subsequently, the fused features and speaker information are separately fed into the cognitive module to enhance the perception of emotions within the dialogue. Compared to seven common baseline methods, our model increased the Accuracy score by 2.71 % and 1.70 % on the IEMOCAP and MELD datasets, respectively. Additionally, the F1 score improved by 2.92 % and 0.70 % for each dataset. Various experiments also demonstrate the effectiveness of our method.

Experimental Study of the Implantation Process for Array Electrodes into Highly Transparent Agarose Gel.

Brain-computer interface (BCI) technology is currently a cutting-edge exploratory problem in the field of human-computer interaction. However, in experiments involving the implantation of electrodes into brain tissue, particularly high-speed or array implants, existing technologies find it challenging to observe the damage in real time. Considering the difficulties in obtaining biological brain tissue and the challenges associated with real-time observation of damage during the implantation process, we have prepared a transparent agarose gel that closely mimics the mechanical properties of biological brain tissue for use in electrode implantation experiments. Subsequently, we developed an experimental setup for synchronized observation of the electrode implantation process, utilizing the Digital Gradient Sensing (DGS) method. In the single electrode implantation experiments, with the increase in implantation speed, the implantation load increases progressively, and the tissue damage region around the electrode tip gradually diminishes. In the array electrode implantation experiments, compared to a single electrode, the degree of tissue indentation is more severe due to the coupling effect between adjacent electrodes. As the array spacing increases, the coupling effect gradually diminishes. The experimental results indicate that appropriately increasing the velocity and array spacing of the electrodes can enhance the likelihood of successful implantation. The research findings of this article provide valuable guidance for the damage assessment and selection of implantation parameters during the process of electrode implantation into real brain tissue.

HANDVIBE: HAND GESTURE VOLUME AND BRIGHTNESS CONTROL

Gesture Recognition has become increasingly relevant in the field of human-computer interaction, as it is a natural way to convey information. We aim to create a system that can identify specific human gestures and utilize them to transmit information for device control. This enables users to operate a computer by performing a specific gesture in front of the camera. The below approach can detect multiple hands simultaneously, that is left and right, each having its own purpose. It is detected by a standard webcam, and requires no extra equipment. The left hand is responsible in controlling the brightness and the right hand will be responsible for controlling the volume. Here we have used different computer vision techniques which includes border detection, and convex-hull detection. The system was able to detect the distance between two points present in the hand, namely fingertip of thumb as well as index finger. Then it calculates the distance between them which is used to apply for volume and brightness, i.e. If the fingers are pinched, the distance between them becomes “zero”, thereby setting the volume or brightness to zero. If it has the maximum distance ,then the volume or brightness is set to maximum that is “100”.The Volume or brightness is decided based upon the hand. The primary aim here is to enable users to adjust the volume as well as the brightness of their system with ease, either by increasing or decreasing it. This offers a promising alternative to touch-based controls as well as voice-based controls. Key Words: Hand Gesture, Volume Control , Brightness Control, Human Control Interaction, Edge Detection, OpenCV, Computer Vision

Light-Adaptive Human Body Key Point Detection Algorithm Based on Multi-Source Information Fusion.

The identification of key points in the human body is vital for sports rehabilitation, medical diagnosis, human-computer interaction, and related fields. Currently, depth cameras provide more precise depth information on these crucial points. However, human motion can lead to variations in the positions of these key points. While the Mediapipe algorithm demonstrates effective anti-shake capabilities for these points, its accuracy can be easily affected by changes in lighting conditions. To address these challenges, this study proposes an illumination-adaptive algorithm for detecting human key points through the fusion of multi-source information. By integrating key point data from the depth camera and Mediapipe, an illumination change model is established to simulate environmental lighting variations. Subsequently, the fitting function of the relationship between lighting conditions and adaptive weights is solved to achieve lighting adaptation for human key point detection. Experimental verification and similarity analysis with benchmark data yielded R2 results of 0.96 and 0.93, and cosine similarity results of 0.92 and 0.90. With a threshold range of 8, the joint accuracy rates for the two rehabilitation actions were found to be 89% and 88%. The experimental results demonstrate the stability of the proposed method in detecting key points in the human body under changing illumination conditions, its anti-shake ability for human movement, and its high detection accuracy. This method shows promise for applications in human-computer interaction, sports rehabilitation, and virtual reality.

IMPLEMENTATION ON AIR-WRITING RECOGNITION SYSTEM

Air writing is a new and innovative technology that allows users to interact with digital devices and manipulate the air by writing or drawing, eliminating the need for all physical contact. This article provides a comprehensive study of the current state of research and development in the field of climate documentation. In this cutting-edge concept, we delve into its history, the motivations that drove its development, and the fundamentals of the technology. We also explore the main tools and equipment required to use the aerial recorder and discuss various applications, such as virtual reality, to improve access for people with disabilities. However, despite its potential, the adoption of writing is not without its challenges and limitations, which we examine in detail. This survey provides a comprehensive review of the latest techniques, algorithms and machine learning techniques used in weather forecasting and provides insight into the methods used to measure performance. We also consider the key elements of user experience and interaction design that are important to the success of these systems. As we look to the future, we identify possible research and development paths that take into account new technologies and changes that will change the country. Overall, this research demonstrates the importance of cloud typing in the field of human-computer interaction and its potential to usher in a new era of contactless, gesture-based interfaces that revolutionize the way we interact with digital technology. " Key Words: Air-Writing, gesture-recognition, CNN, image preprocessingoptics, photonics, light, lasers, templates, journals

Fusion inception and transformer network for continuous estimation of finger kinematics from surface electromyography.

Decoding surface electromyography (sEMG) to recognize human movement intentions enables us to achieve stable, natural and consistent control in the field of human computer interaction (HCI). In this paper, we present a novel deep learning (DL) model, named fusion inception and transformer network (FIT), which effectively models both local and global information on sequence data by fully leveraging the capabilities of Inception and Transformer networks. In the publicly available Ninapro dataset, we selected surface EMG signals from six typical hand grasping maneuvers in 10 subjects for predicting the values of the 10 most important joint angles in the hand. Our model's performance, assessed through Pearson's correlation coefficient (PCC), root mean square error (RMSE), and R-squared (R2) metrics, was compared with temporal convolutional network (TCN), long short-term memory network (LSTM), and bidirectional encoder representation from transformers model (BERT). Additionally, we also calculate the training time and the inference time of the models. The results show that FIT is the most performant, with excellent estimation accuracy and low computational cost. Our model contributes to the development of HCI technology and has significant practical value.

Wireless Sensor System Based on Organohydrogel Ionic Skin for Physiological Activity Monitoring.

Supermolecular hydrogel ionic skin (i-skin) linked with smartphones has attracted widespread attention in physiological activity detection due to its good stability in complex scenarios. However, the low ionic conductivity, inferior mechanical properties, poor contact adhesion, and insufficient freeze resistance of most used hydrogels limit their practical application in flexible electronics. Herein, a novel multifunctional poly(vinyl alcohol)-based conductive organohydrogel (PCEL5.0%) with a supermolecular structure was constructed by innovatively employing sodium carboxymethyl cellulose (CMC-Na) as reinforcement material, ethylene glycol as antifreeze, and lithium chloride as a water retaining agent. Thanks to the synergistic effect of these components, the PCEL5.0% organohydrogel shows excellent performance in terms of ionic conductivity (1.61 S m-1), mechanical properties (tensile strength of 70.38 kPa and elongation at break of 537.84%), interfacial adhesion (1.06 kPa to pig skin), frost resistance (-50.4 °C), water retention (67.1% at 22% relative humidity), and remoldability. The resultant PCEL5.0%-based i-skin delivers satisfactory sensitivity (GF = 1.38) with fast response (348 ms) and high precision under different deformations and low temperature (-25 °C). Significantly, the wireless sensor system based on the PCEL5.0% organohydrogel i-skin can transmit signals from physiological activities and sign language to a smartphone by Bluetooth technology and dynamically displays the status of these movements. The organohydrogel i-skin shows great potential in diverse fields of physiological activity detection, human-computer interaction, and rehabilitation medicine.

A novel mixture model for characterizing human aiming performance data

Fitts’ law is often employed as a predictive model for human movement, especially in the field of human-computer interaction. Models with an assumed Gaussian error structure are usually adequate when applied to data collected from controlled studies. However, observational data (often referred to as data gathered ‘in the wild’) typically display noticeable positive skewness relative to a mean trend as users do not routinely try to minimize their task completion time. As such, the exponentially modified Gaussian (EMG) regression model has been applied to aimed movements data. However, it is also of interest to reasonably characterize those regions where a user likely was not trying to minimize their task completion time. In this article, we propose a novel model with a two-component mixture structure—one Gaussian and one exponential—on the errors to identify such a region. An expectation-conditional-maximization (ECM) algorithm is developed for estimation of such a model and some properties of the algorithm are established. The efficacy of the proposed model, as well as its ability to inform model-based clustering, are addressed in this work through extensive simulations and an insightful analysis of a human aiming performance study.

A wearable strain sensor based on self-healable MXene/PVA hydrogel for bodily motion detection

Developing flexible, stretchable, and self-healing wearable electronic devices with skin-like capabilities is highly desirable for healthcare and human-machine interaction. Hydrogels as a promising sensing material with crosslinked polymer networks have received widespread attention for decades. However, sensors based on hydrogels suffer from low sensitivity and stability due to their poor electrical conductivity or the movement of nanofillers in hydrogel networks. Herein, a stable, sensitive, and self-healing strain sensor is fabricated by the Ti3C2Tx MXene nanosheets/polyvinyl alcohol (PVA) hydrogel (T-hydrogel). The introduction of MXene increases the number of H-bonds in the PVA hydrogel network and enhances the conductivity, resulting in high sensitivity, stability, and self-healing character. The self-healing T-hydrogel-based strain sensor has a performance close to that of the original sensor. In addition, the device is capable of detecting bodily motions, indicating the potential application in the field of human health monitoring and human-computer interaction.

Sustaining Community-Based Research in Computing: Lessons from Two Tech Capacity Building Initiatives for Local Businesses

The field of human-computer interaction (HCI) has traditionally focused on the design of novel technology artifacts. However, ensuring considerations for artifact maintenance and repair is crucial to sustainably supporting the populations they aim to serve over the long term. Drawing on two multi-year programs for tech capacity building in post-industrial U.S. cities, this article presents a comparative analysis to investigate the challenges and strategies for sustained community-based research in computing. In particular, our work detailed three considerations for academic-community partnerships. First, long-term partnerships prioritized transferring trust across academic and community personnel and continually set expectations that responded to evolving community initiatives (i.e., relational sustainability). Second, partnerships used academic support as a way to kickstart community initiatives, and flexibly reframed interventions to stay aligned with evolving community goals (i.e., economic sustainability). Third, partnerships trained personnel to provide technical support alongside interventions and prioritized advice that resisted short-term trends (i.e., technical sustainability). We provide concrete examples of how our two academic-community partnerships carried out such suggestions-such details go unreported in scholarly articles yet are essential for sustainability considerations. We discuss ongoing challenges, such as rethinking when longevity should and should not be the end goal.

Hybrid model for speech emotion recognition of normal and autistic children (SERNAC)

Since the last decade, autism spectrum disorder (ASD) has been used as a general term to describe a wide range of conditions, including autistic syndrome, Asperger's disorder, and pervasive developmental disability. This problem emerges as a decreased ability to share emotions and a greater difficulty understanding others' feelings, leading to increased social communication difficulties. To assist patients with ASD, we proposed a concept that incorporates speech emotion detection technologies, which are widely used in the field of human-computer interaction (particularly youngsters). An algorithm based on a novel method for classifying normal and autistic children's speech emotions is implemented in this article. The training data set is treated to a new approach after all features have been extracted. The technique discussed in this study is the creation of a hybrid algorithm that serves as a classifier for normal and autistic children's speech emotions. Voice emotion recognition can be identified accurately and with a lower error rate. The data collection includes speech samples from 200 normal and 250 autistic groups in four moods (Angry, Happy, Neutral and Sad). As per research findings, the implemented hybrid algorithm for Normal and Autistic Children Speech Emotions (SERNAC) outperformed the existing classifiers by increasing accuracy.

A Comprehensive Review of Vision-Based 3D Reconstruction Methods.

With the rapid development of 3D reconstruction, especially the emergence of algorithms such as NeRF and 3DGS, 3D reconstruction has become a popular research topic in recent years. 3D reconstruction technology provides crucial support for training extensive computer vision models and advancing the development of general artificial intelligence. With the development of deep learning and GPU technology, the demand for high-precision and high-efficiency 3D reconstruction information is increasing, especially in the fields of unmanned systems, human-computer interaction, virtual reality, and medicine. The rapid development of 3D reconstruction is becoming inevitable. This survey categorizes the various methods and technologies used in 3D reconstruction. It explores and classifies them based on three aspects: traditional static, dynamic, and machine learning. Furthermore, it compares and discusses these methods. At the end of the survey, which includes a detailed analysis of the trends and challenges in 3D reconstruction development, we aim to provide a comprehensive introduction for individuals who are currently engaged in or planning to conduct research on 3D reconstruction. Our goal is to help them gain a comprehensive understanding of the relevant knowledge related to 3D reconstruction.

Plastic-Optical-Fiber-Enabled Smart Glove for Machine-Learning-Based Gesture Recognition

Gesture recognition has always been an important research direction in the field of human-computer interaction (HCI). In this paper, a wearable gesture recognition system based on D-shaped plastic optical fiber (POF) curvature sensor was proposed and experimentally studied. A highly bend sensitive D-shaped POF curvature sensor was made and integrated into a five-channel signal acquisition system on a PCB board (8×4.5 cm), which was embedded into an elastic glove to collect fingers' movement data. Thirteen gestures and eleven grasping actions were defined, and the gesture data, the grasping action data and the gesture data mixed with grasping action data were normalized, calibrated and imported into a support vector machine (SVM) classifier based on Gaussian kernel function and feedforward neural networks (FNN) respectively. The recognition accuracy based on SVM of 13 gestures and 11 grasping actions reached 99.8% and 97.7% respectively. The recognition accuracy of 13 kinds of gesture data mixed with 11 kinds of grasping action data based on Gaussian kernel function in SVM classification model and FNN were 98.9% and 99.4% respectively.

Virtual Air Canvas: A Review

This creative method of utilizing open-source frameworks to create gesture-controlled whiteboard devices is impressive. The device tracks and interprets finger movements collected by a camera to allow users to engage with a virtual whiteboard. The implementation includes effective data processing, hand landmark estimate, and real-time hand detection by utilizing computer vision algorithms. By providing a modern form of communication and going beyond regular typing and writing habits, the system seeks to expand beyond traditional writing methods and create new opportunities for connection. The system contributes to the changing field of human-computer interaction by offering an alternate mode of communication, which improves people's quality of life overall. It also demonstrates the versatility of gesture-controlled systems in terms of enhancing communication accessibility. Key Words: Data Processing, Handmark Estimation, Computer Vision algorithms, MediaPipe, OpenCV.

An Analysis of the Authorship and Co-authorship Networks of the Brazilian Human-Computer Interaction Conference

In Brazil, the Brazilian Symposium on Human Factors in Computing Systems (IHC) gathers the scientific community of researchers interested in the field of Human-Computer Interaction since 1998, being the main Brazilian event in this sub-area of Computing. Over twenty-one editions, the IHC received works from researchers from different regions of the country who, over the years, have been building their own co-authorship relationships with the other authors of the Symposium. In this context, this paper analysed the IHC from the perspective of those who helped to consolidate this important national scientific event, as well as in the expansion of the Human-Computer Interaction area in the Brazilian scenario, that is, its researchers-authors. In total, 1,443 authors were identified and analysed in the study presented in this work, which considered 873 publications of three IHC tracks: Full Papers, Short Papers, and Innovative Ideas and Emerging Results. Issues related to the publications and to the co-authorship relationships of these authors over the years and in the different article tracks of the IHC were considered. In order to describe their research trajectories within the IHC itself, the study presents, in different scales of time, how these authors evolved in relation to their contributions over time. In addition, this paper analyses how the authors contributed with each other and originated the complex collaboration network of the IHC. For this, co-authorship networks and groups of authors who published together were explored, aiming to clarify the collaborations between these authors, as well as how they evolved until the edition of 2022. In this sense, this work seeks, with each research question, to simplify the presentation of results through different visualizations, which were planned and created to describe information that are not clearly evident when observing the IHC publications in a “disconnected” manner. The results of this study are revealed, described and analysed under different perspectives, as well as discussed in details in this paper.

Application of Poisson’s ratio structures and decoupling algorithm for 3D force sensing

Flexible tactile electronic devices are extensively used in the fields of robotics, medical detection, and human-computer interaction. Monitoring contact parameters, including force magnitude, direction, and contact location, is particularly vital for skin-like tactile sensing devices. Herein, a 3D force sensor is designed based on porous structure with deliberately designed Poisson’s ratios. A genetic algorithm (GA) optimized back propagation neuronal network (BPNN) model is proposed to support the 3D force decoupling, which can greatly improve the decoupling accuracy. The introduction of the GA-BPNN significantly enhances decoupling accuracy compared to the initial neural network. Micro-porous structures with varied Poisson’s ratios are embedded into the sensing unit to achieve better sensibility. Significantly, this study underscores that the decoupling accuracy of the force components along the Z-axis can be further improved by substituting the solid unit with a designed porous structure unit featuring a specific Poisson’s ratio in an arrayed 3D force sensor.

Deep-Learning-Assisted Sensor with Multiple Perception Capabilities for an Intelligent Driver Assistance Monitoring System.

Driver assistance systems can help drivers achieve better control of their vehicles while driving and reduce driver fatigue and errors. However, the current driver assistance devices have a complex structure and severely violate the privacy of drivers, hindering the development of driver assistance technology. To address these limitations, this article proposes an intelligent driver assistance monitoring system (IDAMS), which combines a Kresling origami structure-based triboelectric sensor (KOS-TS) and a convolutional neural network (CNN)-based data analysis. For different driving behaviors, the output signals of the KOS-TSs contain various features, such as a driver's pressing force, pressing time, and sensor triggering sequence. This study develops a multiscale CNN that employs different pooling methods to process KOS-TS data and analyze temporal information. The proposed IDAMS is verified by driver identification experiments, and the results show that the accuracy of the IDAMS in discriminating eight different users is improved from 96.25% to 99.38%. In addition, the results indicate that IDAMS can successfully monitor driving behaviors and can accurately distinguish between different driving behaviors. Finally, the proposed IDAMS has excellent hands-off detection (HOD), identification, and driving behavior monitoring capabilities and shows broad potential for application in the fields of safety warning, personalization, and human-computer interaction.

Human-Computer Interaction in Agricultural User Interfaces

This study explores the intersection of Human-Computer Interaction (HCI) and agricultural user interfaces, aiming to enhance the efficiency and user experience within the agricultural domain. A thorough literature review identifies current trends, challenges, and gaps in HCI research specific to agricultural interfaces. The research objectives encompass the proposal of HCI design principles tailored to the unique characteristics of the agricultural environment. Methodologically, the study employs user surveys, interviews, and usability testing to gather insights into user preferences and challenges. The paper presents case studies highlighting successful implementations of HCI principles in agricultural settings, providing tangible examples of improved user engagement and task performance. Challenges and limitations in HCI adoption in agriculture are discussed, along with user feedback and evaluations shaping interface design. The implications of the research extend to HCI practitioners, agricultural technology developers, and policymakers, offering practical insights for future developments. This paper contributes valuable knowledge to the evolving field of HCI in agricultural contexts, emphasizing the importance of user-centered design.

Voice-based age, gender, and language recognition based on ResNet deep model and transfer learning in spectro-temporal domain

In personal identity recognition systems, detecting a person's age, gender, and language using voice signal characteristics is a crucial issue, especially because of the importance of security considerations. Age, gender, and language classification problems are important in signal processing because they are used to analyze and understand human behavior, interactions, and preferences. This can be especially useful in the fields of human-computer interaction, psychology, and social science research. In this paper, a new system for detecting a speaker's age, gender, and language based on deep learning models is presented. Deep learning models have shown great efficacy in various fields of signal processing. For this paper, a range of deep models were tested, including convolutional neural networks (CNNs), recurrent neural network (RNN), and a fine-tuning ResNet34 architecture. Additionally, techniques such as transfer learning were applied to improve the efficiency of the proposed system. The input voice signals are preprocessed by applying the spectro-temporal transform to obtain additional features that can be fed to the ResNet34 model, which is designed specifically for the task of voice signal processing. The dataset used in this paper was sourced from the Mozilla common voice initiative, which is dedicated to advancing speech recognition and language identification technologies. The performance of the proposed algorithm was evaluated in the presence of Gaussian noise to determine its robustness. The experimental results demonstrated that the proposed algorithm significantly outperformed basic algorithms and other deep neural networks in terms of age and gender recognition from voice signals.