Field Of Human-computer Interaction Research Articles

All-in-one, flexible, diversified self-powered sensors through embedded 3D printing

Abstract Triboelectric nanogenerators (TENGs) are characterized by zero power consumption and are often employed as self-powered sensors. However, the complex manufacturing process and expensive equipment limit the further promotion and application of self-powered sensors, which have become urgent challenges in this field. Here, a simple strategy using embedded 3D printing is proposed to enable the fabrication of diverse self-powered sensors in one-step, reducing production costs while increasing design flexibility. Specifically, the designed sensors composed of the silicone as the triboelectric layer and silicone/multi-walled carbon nanotubes as flexible electrodes with excellent all-in-one structures. Meanwhile, diversified self-powered sensors with different complex structures (e.g., planar array sensors and helical icosahedron sensors) were developed to meet the diverse needs of different applications, verifying the capability of the proposed embedded 3D printing method to design and customize sensors with various shapes and structures. In addition, the applications of these functionalized self-powered sensors in cryptographic simulation, pressure position detection, and impact force recognition have been successfully demonstrated. Therefore, this self-powered sensor based on embedded 3D printing has a promising future in human-computer interaction, collision detection and other fields.

Human-computer interactions with farm animals—enhancing welfare through precision livestock farming and artificial intelligence

While user-centered design approaches stemming from the human-computer interaction (HCI) field have notably improved the welfare of companion, service, and zoo animals, their application in farm animal settings remains limited. This shortfall has catalyzed the emergence of animal-computer interaction (ACI), a discipline extending technology’s reach to a multispecies user base involving both animals and humans. Despite significant strides in other sectors, the adaptation of HCI and ACI (collectively HACI) to farm animal welfare—particularly for dairy cows, swine, and poultry—lags behind. Our paper explores the potential of HACI within precision livestock farming (PLF) and artificial intelligence (AI) to enhance individual animal welfare and address the unique challenges within these settings. It underscores the necessity of transitioning from productivity-focused to animal-centered farming methods, advocating for a paradigm shift that emphasizes welfare as integral to sustainable farming practices. Emphasizing the ‘One Welfare’ approach, this discussion highlights how integrating animal-centered technologies not only benefits farm animal health, productivity, and overall well-being but also aligns with broader societal, environmental, and economic benefits, considering the pressures farmers face. This perspective is based on insights from a one-day workshop held on June 24, 2024, which focused on advancing HACI technologies for farm animal welfare.

Optimization Research on Integrating Mental Health Into Ideological and Political Theory Courses Through Deep Learning Network Applications

This study aims to evaluate the effectiveness of incorporating mental health education into ideological and political theory classrooms using speech emotion recognition technology from the field of human-computer interaction. Mental health education is essential for students' well-being, and this study establishes a theoretical framework for its integration into the curriculum. Utilizing recurrent neural networks (RNNs) augmented with attention mechanisms, the research demonstrates the model's proficiency in identifying subtle emotional cues within speech data. The model achieves an average recognition accuracy of 87.21% on the RAVDESS speech emotion corpus, showing particular strengths in detecting emotions like happiness and boredom. The findings suggest that this technology can be effectively employed to gauge student satisfaction with different teaching methodologies post-integration, offering valuable insights for refining educational practices in ideological and political theory classes through improved mental health education initiatives.

Research on gesture recognition technology based on machine learning

Abstract. Hand Gesture Recognition (HGR), as a significant technological advancement in the field of Human-Computer Interaction (HCI), aims to develop systems capable of accurately recognizing and interpreting human gestures for a diverse range of applications, including device control, virtual reality, gesture passwords, and gesture interaction. With the continuous advancement of machine learning algorithms, especially deep learning techniques, machine learning-based gesture recognition has garnered widespread attention. This paper presents a review of the development of gesture recognition techniques from traditional approaches to the current mainstream deep learning-based methods, and outlines the challenges and technical difficulties encountered. It analyzes several of the most popular classification techniques, including Naive Bayes, K-Nearest Neighbors (KNN), Random Forests, XGBoost, Support Vector Classifiers (SVCs), and Convolutional Neural Networks (CNNs). Furthermore, this paper examines the application of these algorithms in both dynamic and static gesture recognition and compares their performance and suitability in different scenarios. The results demonstrate that the accuracy and robustness of gesture recognition systems can be markedly enhanced through the prudent selection and optimization of the algorithms, which serves as a valuable reference for future research and applications.

Hydrophobic and Elastic Structural Triboelectric Materials Enabled by Template Method toward Real-Time Material Recognition.

Since each material has a unique ability to lose or obtain electrons, specific triboelectric signals are produced when triboelectric materials are in contact with different objects. Triboelectric nanogenerator (TENG) devices show great potential for use as tactile sensors; nevertheless, analyzing the structure-function relationship of functionalized triboelectric sensing interfaces under environmental conditions and improving the sensing stability and accuracy through the design of hydrophobic structure on the triboelectric material surface remain major challenges in the development of intelligent sensing networks. Compared with the traditional rigid micronanostructure, the elastic micronanostructure strategy is applied to achieve both hydrophobicity and stability of triboelectric materials based on the template method in this work. The corresponding surface roughness and contact angle are 89.9 nm and 117.9°, respectively. As expected, the output voltage and charge density are enhanced by almost 65.8 and 33.4%, respectively, with the establishment of an elastic micronanostructure on the triboelectric material surface. More importantly, the triboelectric signal waveforms also present acceptable durability for subsequent recognition after immersion in water or ethanol for 12 days and metal impact for 12 000 cycles. Hence, combined with deep machine learning and triboelectric effect, a material perception system integrated with a moisture-resistant TENG-based sensor after fatigue testing, data processing, and display modules is also developed for real-time monitoring with approximately 100% (mask), 76% (plank), 93% (plastic), and 89% (rubber) identification accuracies in the natural environment. Finally, the proposed hydrophobic and elastic triboelectric materials show broad potential for application in the field of human-computer interaction.

Microstructured Self-Healing Flexible Tactile Sensors Inspired by Bamboo Leaves.

Wearable electronic devices with multifunctions such as flexible, integrated, and self-powered play a crucial role in the fields of health monitoring, motion monitoring, and human-computer interaction. However, their core basic components, flexible pressure sensors, face challenges including poor long-term stability and insufficient real-time sensing accuracy. In order to solve the challenges of long-term, stable, and accurate sensing of the sensor, this paper prepares polydimethylsiloxane (SHPDMS) with intrinsic self-healing property and designs a high-sensitivity self-healing capacitive flexible pressure sensor with dual microstructures (grating microstructured electrodes and microporous dielectric layer) as the substrate based on SHPDMS. Specifically speaking, the self-healing of the sensor under mild conditions was realized by introducing reversible imine bonds with low bonding energy into the polydimethylsiloxane (PDMS) flexible substrate, which solved the problem of the material's long-term service durability. A grating-like microstructure was introduced into the flexible electrode by using a spotted bamboo taro leaf as a template, and a dual microstructure sensor was constructed by combining it with a microporous dielectric layer doped with single-walled carbon nanotubes. This way reduces the elastic modulus of the dielectric layer, improves the dielectric constant of the sensor under loading, and thus significantly improves the sensor's sensitivity and extends the measurement accuracy in a low-stress range. The prepared self-healing flexible sensor achieves a sensitivity of 3.6 kPa-1, a minimum detection limit of 5 Pa, a response recovery time of less than 80 ms, and stability over 5000 cycles, which exceeds most previously reported silicone rubber-based capacitive flexible sensors.

Defining social reward: A systematic review of human and animal studies.

Social rewards are strong drivers of behavior and fundamental to well-being, yet there is a lack of consensus regarding what actually defines a reward as "social." Because a systematic overview of existing social reward operationalizations is currently absent, a review of the literature seems necessary to advance toward a unified framework and to better guide research and theory. To bridge this gap, we preregistered and conducted the first comprehensive systematic review of human and animal experimental studies that used the term "social reward" and charted existing operationalizations, revealing the implicit and explicit definitions used in the field. Stimulus characteristics and measures of social reward were extracted from a total of 384 studies encompassing 42,118 participants and subjects. We provide detailed summaries of these elements, stratified by species (human/animal) and study type (behavioral, brain imaging, pharmacological, and physiological). Two main aspects were found to account for most of the difference in operationalizations: the sensory richness of a stimulus (intimacy) and engagement in social interaction (i.e., the synchronous observation and action between at least two individuals, viz., immediacy). Drawing insights from second-person neuroscience approaches and theoretical models in the field of human-computer interaction, we propose that human and animal research can greatly benefit from considering these properties, as they have important theoretical and practical consequences for human and translational research, with far-reaching implications for neighboring research fields such as those pertaining to social media and the development of artificial intelligence. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

CNTs@fabric/Ni@PU piezoresistive sensor with enhanced interfacial contact resistance variation for motion detection and deep-learning-assisted recognition

Abstract Flexible piezoresistive sensors based on the mechanism of interfacial contact resistance change are receiving increasing attention in the fields of human-computer interaction, health monitoring, and behavior tracking. However, the high cost and complex manufacturing process limit the wide application and development of these flexible piezoresistive sensors. Here, a novel carbon nanotubes@fabric (CNTs@fabric)/Ni@polyurethane (Ni@PU) piezoresistive sensor (CNPS) with low-cost, simple-preparation and high-sensitivity was proposed. The effective contact area is obtained by synergizing the woven micro-convex structure of the fabric, the large specific surface area of the CNTs and the porous three dimensional electrodes. Within the small pressure (0–9.52 kPa) effect, the area of connection with the electrodes to the active layer plays a dominant role, resulting in a sensitivity of up to 6.39 kPa−1 for CNPS. In the high pressure region (9.52–44.92 kPa), where internal mechanism of change in the sensitive material dominants, the CNPS has a response time of 85 ms at a constant pressure of 28.31 kPa. Considering the excellent output electrical performances, a variety of body movements could be detected by fixing the CNPS to different joints. Significantly, the designed intelligent object recognition system implemented by the combination of matrix stress detection module and residual neural network (ResNet) algorithm has a recognition accuracy of 99.26%. Enhancing the interfacial contact resistance change mechanism using a simple fabrication process offers a promising strategy for the rapid development of flexible piezoresistive sensors.

Learning from Users: Everyday Playful Interactions to Support Architectural Spatial Changes

While there's growing interest in eliciting situated playful interactions with technologies in different contexts, how these interactions might shape everyday spaces still needs to be fully explored. To address this gap, this article aims to guide the architectural spatial changes by exploring everyday playful interactions of technology-adopted users in domestic spaces. We present our contributions in a two-fold study: First, through an extensive diary study involving 13 technology-adopted residents in gated communities, where distinct boundaries offer increased opportunities for playful interactions, we identified four playful themes: (1) creating and expanding play-spaces, (2) balancing play and comfort, (3) intertwining imagination with spatial experience, and (4) gamifying household interactions. Secondly, by building on these themes, we outline three design implications to inform architectural design processes, aiming to translate everyday playful interactions into tangible spatial changes. These implications include adapting shape-changing and wearable technologies for playful flexibility, embedding new forms of communications within infrastructures, and turning homes into interactive entities with multi-sensory technologies. While these findings provide a starting point for exploring new architectural design possibilities in similar environments, further research with architects, policymakers, and design researchers in the Human-Computer Interaction (HCI) and Human-Building Interaction (HBI) fields is essential to actualizing these changes.

MXene/rGO piezoresistive sensor based on Longan leaves’ hierarchical microstructure for human-motion detection

Flexible pressure sensors exhibit great application potential in wearable devices, human-computer interaction, medical monitoring and other fields and thereby have attracted much attention. However, a common challenge in piezoresistive sensors is the trade-off between sensing range and sensitivity, which limits their versatility across various applications. By employing Longan leaf as a hierarchical microstructure template, the sensing range and sensitivity can be significantly improved at low cost, based on fractal theory. MXene was used as the sensing material, and a systematic study was conducted concerning the effect of MXene concentration on the sensor's sensitivity and detection range, which revealed that the sensitivity firstly increased and then decreased with MXene concentration ranging from 5 mg/mL to 30 mg/mL. Furthermore, based on the solution method to prepare composite sensing layer, MXene and rGO were mixed in proportion to obtain a sensing network that can coordinate with each other. When the volume ratio was 1:1, and the concentration of MXene was up to 15 mg/mL, the fabricated sensor demonstrated high sensitivity (47.36 kPa−1) and wide detection range (>100 kPa) at the same time. In addition, the sensor was successfully used to monitor various human joint activities and physiological signals, which was also expected to provide a competitive scheme for wearable flexible electronic devices in human-motion detection.

MXene-based self-adhesive, ultrasensitive, highly tough flexible hydrogel pressure sensors for motion monitoring and robotic tactile sensing

With the rapid development of artificial intelligence and human–computer interaction fields, wearable electronic devices represented by flexible pressure sensors have a wider range of application scenarios. In this paper, we used a composite hydrogel made by one-pot polymerization using a combination of acrylamide (AM), sodium carboxymethyl cellulose (CMC), dopamine (DA), and MXene (Ti3C2Tx). The composite hydrogel has the advantages of self-adhesion, high sensitivity, large detection range (0–500 kPa), low detection limit, high toughness (compression > 90 %), stability (more than 6700 s) and other advantages of hydrogel flexible pressure strain sensor. Sodium carboxymethyl cellulose (CMC), as a multifunctional thickener and gelling agent, significantly improves rheological properties and viscosity in hydrogels, forms crosslinked structures to enhance mechanical properties and provides reliable toughness for pressure sensors. Dopamine (DA), which is rich in catechol groups, can adhere to the material surface through oxidative polymerization to enhance bond strength. Hydrogel pressure sensors for human motion detection can accurately track joint movement, respond sensitively to changes in knocks and touches, and display specific characters such as Morse code. The technology extends to robotic systems, where pressure-activated electrical signals are used to control robot movements. Therefore, it has a very promising application in the field of human–computer interaction, human-body movement signal detection, etc. PAM/CMC/DA/MXene hydrogel pressure sensors have excellent mechanical, electrical, and adhesion properties, which provide a more reliable solution for the fields of flexible wearable e-skin, human–computer interaction, and tactile sensing.

Machine Learning-Based Gesture Recognition Glove: Design and Implementation.

In the evolving field of human-computer interaction (HCI), gesture recognition has emerged as a critical focus, with smart gloves equipped with sensors playing one of the most important roles. Despite the significance of dynamic gesture recognition, most research on data gloves has concentrated on static gestures, with only a small percentage addressing dynamic gestures or both. This study explores the development of a low-cost smart glove prototype designed to capture and classify dynamic hand gestures for game control and presents a prototype of data gloves equipped with five flex sensors, five force sensors, and one inertial measurement unit (IMU) sensor. To classify dynamic gestures, we developed a neural network-based classifier, utilizing a convolutional neural network (CNN) with three two-dimensional convolutional layers and rectified linear unit (ReLU) activation where its accuracy was 90%. The developed glove effectively captures dynamic gestures for game control, achieving high classification accuracy, precision, and recall, as evidenced by the confusion matrix and training metrics. Despite limitations in the number of gestures and participants, the solution offers a cost-effective and accurate approach to gesture recognition, with potential applications in VR/AR environments.

Double-layer stretchable composite conductive graphene-hydrogel with wide-range linear sensing and thermal-humidity management for health monitoring

Laser-induced graphene (LIG) is widely used in stretchable electrical conductors due to its excellent properties and low manufacturing cost. However, there is usually a mechanical mismatch or poor interface effect between LIG and the flexible substrate, and LIG will inevitably crack when stretched, which leads to LIG-based stretchable conductors usually have poor operating range, linearity and sensitivity stability. In this study, a double-layer stretchable composite conductive (DSCCGH) was prepared based on laser-induced MXene-decorated graphene (LIMG), with LIMG transferred to the hydrogel using a repeated reversible freeze-thaw method and a biomimetic structural design incorporated. The strong mechanical interlocking between LIMG and the hydrogel, along with the graphene-coated MXene derivative TiO2 providing additional electrical pathways when LIG’s conductive networks are disconnected, allowed this DSCCGH to achieve wide-range sensing linearity (0–366%, R2 = 0.9948) and stable sensitivity. This setup enabled stable and reliable muscle movement and pulse monitoring. Furthermore, biomimetic structural design imparts the DSCCGH with perspiration and cooling functions, enhancing comfort and wearability for outdoor exercise. This innovative design provides significant potential for future developments in wearable electronics and human-computer interaction fields.

MelTrans: Mel-Spectrogram Relationship-Learning for Speech Emotion Recognition via Transformers.

Speech emotion recognition (SER) is not only a ubiquitous aspect of everyday communication, but also a central focus in the field of human-computer interaction. However, SER faces several challenges, including difficulties in detecting subtle emotional nuances and the complicated task of recognizing speech emotions in noisy environments. To effectively address these challenges, we introduce a Transformer-based model called MelTrans, which is designed to distill critical clues from speech data by learning core features and long-range dependencies. At the heart of our approach is a dual-stream framework. Using the Transformer architecture as its foundation, MelTrans deciphers broad dependencies within speech mel-spectrograms, facilitating a nuanced understanding of emotional cues embedded in speech signals. Comprehensive experimental evaluations on the EmoDB (92.52%) and IEMOCAP (76.54%) datasets demonstrate the effectiveness of MelTrans. These results highlight MelTrans's ability to capture critical cues and long-range dependencies in speech data, setting a new benchmark within the context of these specific datasets. These results highlight the effectiveness of the proposed model in addressing the complex challenges posed by SER tasks.

Wushu Movement Recognition System Based on DTW Attitude Matching Algorithm

Motion recognition technology is widely used in intelligent video surveillance, human-computer interaction and other fields. With the development of computer vision technology, improving the accuracy and efficiency of motion recognition has become the focus of research. The purpose of this study is to improve the performance of Wushu movement recognition through improved dynamic time warping algorithm and hierarchical model. Firstly, a high-dimensional feature vector is constructed by using the position, velocity and Angle changes of human bone joints. The actions are subdivided by the hierarchical model, and matched and recognized by the max-minimum dynamic time regularization model. Meanwhile, the K-class mean algorithm is combined to optimize the type of tree core, improve the performance of the model, reduce the interference of noise nodes, and effectively classify Wushu actions. Experimental verification was carried out on four public data sets of KTH, Olympic Sports, Hollywood2 and HMDB51. The experimental results showed that the recognition rate of the proposed model in KTH data set was 95.2%, and that in Olympic Sports data set was 91.4%. The Hollywood2 dataset was 66.7%, and the HMDB51 dataset was 61.2%. Comparing the results of different algorithms, the proposed method improved the recognition performance by 10% compared with long short-term memory network and gated cycle unit. Compared with one-dimensional convolutional neural network, the time of the proposed method was 15s longer, but the recognition rate was 1.6% higher. The results showed that the proposed method had significant performance advantages in diverse and complex action recognition tasks. Meanwhile, the results emphasized the factors to be considered in the design of the model, demonstrating its effectiveness in the application of Wushu movement recognition.

Representing scents: An evaluation framework of scent-related experiences through associations between grounded and psychophysiological data

This study introduces an empirical approach for assessing human scent-related experiences within the field of Human-Computer Interaction (HCI). We labeled 43 fragrances based on grounded collective experience, incorporating semantic and impression-based data. Furthermore, we collected comprehensive psychophysiological data, including electroencephalogram (EEG), electrobulbogram (EBG), electrocardiogram (ECG), and facial dynamics captured by a camera, from participants who experienced the scents. By computing scent-wise similarity and correlating both grounded and psychophysiological scent spaces, we identified associations between them, demonstrating the potential of this approach to enhance our understanding of scent-related experiences. Additionally, we propose an iterative evaluation framework to refine the design of smell-based interactions and we conduct a real-life study to validate this framework.

Toward Adaptive Virtual Reality Systems: Understanding Emotional and Physiological Responses in VR Game

Virtual reality (VR) affective computing is expanding to include systems that can identify and respond to human emotions. This study explores emotional responses in VR using the game Pong to monitor these reactions under different gameplay conditions. It investigates how game pace and opponent behavior affect emotions, assessed through physiological metrics like electrocardiogram, galvanic skin response, electromyogram, and player behaviors. Results from three game variations—slow-paced, fast-paced, and lag-induced—showed distinct emotional and physiological responses. Significant differences were found in arousal and valence levels, with varying heart rate and skin conductance among the game conditions. These findings suggest that VR game design can significantly influence emotional states, providing insights that could help develop more engaging and responsive VR systems. Thus, this study demonstrates the potential of VR to evoke and measure specific emotional responses, contributing to the fields of human-computer interaction, and affective computing.

Flexible and self-healing tannic acid-Fe3O4@MXene-composited dual network hydrogel multifunctional strain sensor

In light of the exponential growth of flexible human-computer interactions and telecommunications, it is becoming increasingly important to develop a high-sensitivity strain sensor to monitor human movement. Herein, the biocompatible, adhesive, and flexible Tannic acid-Fe3O4@MXene-poly(methacryloxyethyl sulfobetaine/acrylamide) (TFM) hydrogel has been prepared to realize the dual-functional applications in strain sensitive sensor and electromagnetic interference (EMI) shielding. The polymer network and 2D conductive filler MXene together construct the conductive network inside the hydrogel. The interpenetrating network and hydrogen bond between poly(acrylamide) and poly(methacryloxyethyl sulfobetaine) chain enhance the mechanical property of hydrogel, which is also improved by the complexation between Fe3O4 and the catechol groups of tannic acid. Benefiting from the excellent electrical conductivity (∼1.33 S/m) and mechanical property of the TFM hydrogels, the strain sensor displays compressive strain (∼77.6 %) and elongation (∼308.5 %), adhesion (∼16 kPa), compression sensitivity (∼42.6 Pa−1), gauge factor (GF = 2.38), and cyclic stability (∼50 cycles). TFM hydrogels can monitor strain distribution sensing, pulse beating and human movement detection. Moreover, the TFM hydrogel exhibits an absorption-predominant EMI shielding performance with an effective value of ∼25.2 dB. This work provides a new inspiration for the development of multifunctional hydrogels in the fields of artificial intelligence, human-computer interaction, and EMI shielding technology.

Pollen-Modified Flat Silk Cocoon Pressure Sensors for Wearable Applications.

Microstructures have been proved as crucial factors for the sensing performance of flexible pressure sensors. In this study, polypyrrole (PPy)/sunflower pollen (SFP) (P/SFP) was prepared via the in situ growth of PPy on the surface of degreased SFP with a sea urchin-like microstructure; then, these P/SFP microspheres were sprayed onto a flat silk cocoon (FSC) to prepare a sensing layer P/SFP-FSC. PPy-FSC (P-FSC) was prepared as an electrode layer through the in situ polymerization of PPy on the FSC surface. The sensing layer P/SFP-FSC was placed between two P-FSC electrode layers to assemble a P/SFP-FSC pressure sensor together with a fork finger electrode. With 6 mg/cm2 of optimized sprayed P/SFP microspheres, the prepared flexible pressure sensor has a sensitivity of up to 0.128 KPa-1 in the range of 0-13.18 KPa and up to 0.13 KPa-1 in the range of 13.18-30.65 KPa, a fast response/recovery time (90 ms/80 ms), and a minimum detection limit as low as 40 Pa. This fabricated flexible P/SFP-FSC sensor can monitor human motion and can also be used for the encrypted transmission of important information via Morse code. In conclusion, the developed flexible P/SFP-FSC pressure sensor based on microstructure modification in this study shows good application prospects in the field of human-computer interaction and wearable electronic devices.

Assessing Feature Importance in Eye-Tracking Data within Virtual Reality Using Explainable Artificial Intelligence Techniques

Our research systematically investigates the cognitive and emotional processes revealed through eye movements within the context of virtual reality (VR) environments. We assess the utility of eye-tracking data for predicting emotional states in VR, employing explainable artificial intelligence (XAI) to advance the interpretability and transparency of our findings. Utilizing the VR Eyes: Emotions dataset (VREED) alongside an extra trees classifier enhanced by SHapley Additive ExPlanations (SHAP) and local interpretable model agnostic explanations (LIME), we rigorously evaluate the importance of various eye-tracking metrics. Our results identify significant correlations between metrics such as saccades, micro-saccades, blinks, and fixations and specific emotional states. The application of SHAP and LIME elucidates these relationships, providing deeper insights into the emotional responses triggered by VR. These findings suggest that variations in eye feature patterns serve as indicators of heightened emotional arousal. Not only do these insights advance our understanding of affective computing within VR, but they also highlight the potential for developing more responsive VR systems capable of adapting to user emotions in real-time. This research contributes significantly to the fields of human-computer interaction and psychological research, showcasing how XAI can bridge the gap between complex machine-learning models and practical applications, thereby facilitating the creation of reliable, user-sensitive VR experiences. Future research may explore the integration of multiple physiological signals to enhance emotion detection and interactive dynamics in VR.