Field Of Human-computer Interaction Research Articles

Microstructured Self-Healing Flexible Tactile Sensors Inspired by Bamboo Leaves.

Wearable electronic devices with multifunctions such as flexible, integrated, and self-powered play a crucial role in the fields of health monitoring, motion monitoring, and human-computer interaction. However, their core basic components, flexible pressure sensors, face challenges including poor long-term stability and insufficient real-time sensing accuracy. In order to solve the challenges of long-term, stable, and accurate sensing of the sensor, this paper prepares polydimethylsiloxane (SHPDMS) with intrinsic self-healing property and designs a high-sensitivity self-healing capacitive flexible pressure sensor with dual microstructures (grating microstructured electrodes and microporous dielectric layer) as the substrate based on SHPDMS. Specifically speaking, the self-healing of the sensor under mild conditions was realized by introducing reversible imine bonds with low bonding energy into the polydimethylsiloxane (PDMS) flexible substrate, which solved the problem of the material's long-term service durability. A grating-like microstructure was introduced into the flexible electrode by using a spotted bamboo taro leaf as a template, and a dual microstructure sensor was constructed by combining it with a microporous dielectric layer doped with single-walled carbon nanotubes. This way reduces the elastic modulus of the dielectric layer, improves the dielectric constant of the sensor under loading, and thus significantly improves the sensor's sensitivity and extends the measurement accuracy in a low-stress range. The prepared self-healing flexible sensor achieves a sensitivity of 3.6 kPa-1, a minimum detection limit of 5 Pa, a response recovery time of less than 80 ms, and stability over 5000 cycles, which exceeds most previously reported silicone rubber-based capacitive flexible sensors.

Defining social reward: A systematic review of human and animal studies.

Social rewards are strong drivers of behavior and fundamental to well-being, yet there is a lack of consensus regarding what actually defines a reward as "social." Because a systematic overview of existing social reward operationalizations is currently absent, a review of the literature seems necessary to advance toward a unified framework and to better guide research and theory. To bridge this gap, we preregistered and conducted the first comprehensive systematic review of human and animal experimental studies that used the term "social reward" and charted existing operationalizations, revealing the implicit and explicit definitions used in the field. Stimulus characteristics and measures of social reward were extracted from a total of 384 studies encompassing 42,118 participants and subjects. We provide detailed summaries of these elements, stratified by species (human/animal) and study type (behavioral, brain imaging, pharmacological, and physiological). Two main aspects were found to account for most of the difference in operationalizations: the sensory richness of a stimulus (intimacy) and engagement in social interaction (i.e., the synchronous observation and action between at least two individuals, viz., immediacy). Drawing insights from second-person neuroscience approaches and theoretical models in the field of human-computer interaction, we propose that human and animal research can greatly benefit from considering these properties, as they have important theoretical and practical consequences for human and translational research, with far-reaching implications for neighboring research fields such as those pertaining to social media and the development of artificial intelligence. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

CNTs@fabric/Ni@PU piezoresistive sensor with enhanced interfacial contact resistance variation for motion detection and deep-learning-assisted recognition

Abstract Flexible piezoresistive sensors based on the mechanism of interfacial contact resistance change are receiving increasing attention in the fields of human-computer interaction, health monitoring, and behavior tracking. However, the high cost and complex manufacturing process limit the wide application and development of these flexible piezoresistive sensors. Here, a novel carbon nanotubes@fabric (CNTs@fabric)/Ni@polyurethane (Ni@PU) piezoresistive sensor (CNPS) with low-cost, simple-preparation and high-sensitivity was proposed. The effective contact area is obtained by synergizing the woven micro-convex structure of the fabric, the large specific surface area of the CNTs and the porous three dimensional electrodes. Within the small pressure (0–9.52 kPa) effect, the area of connection with the electrodes to the active layer plays a dominant role, resulting in a sensitivity of up to 6.39 kPa−1 for CNPS. In the high pressure region (9.52–44.92 kPa), where internal mechanism of change in the sensitive material dominants, the CNPS has a response time of 85 ms at a constant pressure of 28.31 kPa. Considering the excellent output electrical performances, a variety of body movements could be detected by fixing the CNPS to different joints. Significantly, the designed intelligent object recognition system implemented by the combination of matrix stress detection module and residual neural network (ResNet) algorithm has a recognition accuracy of 99.26%. Enhancing the interfacial contact resistance change mechanism using a simple fabrication process offers a promising strategy for the rapid development of flexible piezoresistive sensors.

Learning from Users: Everyday Playful Interactions to Support Architectural Spatial Changes

While there's growing interest in eliciting situated playful interactions with technologies in different contexts, how these interactions might shape everyday spaces still needs to be fully explored. To address this gap, this article aims to guide the architectural spatial changes by exploring everyday playful interactions of technology-adopted users in domestic spaces. We present our contributions in a two-fold study: First, through an extensive diary study involving 13 technology-adopted residents in gated communities, where distinct boundaries offer increased opportunities for playful interactions, we identified four playful themes: (1) creating and expanding play-spaces, (2) balancing play and comfort, (3) intertwining imagination with spatial experience, and (4) gamifying household interactions. Secondly, by building on these themes, we outline three design implications to inform architectural design processes, aiming to translate everyday playful interactions into tangible spatial changes. These implications include adapting shape-changing and wearable technologies for playful flexibility, embedding new forms of communications within infrastructures, and turning homes into interactive entities with multi-sensory technologies. While these findings provide a starting point for exploring new architectural design possibilities in similar environments, further research with architects, policymakers, and design researchers in the Human-Computer Interaction (HCI) and Human-Building Interaction (HBI) fields is essential to actualizing these changes.

MXene/rGO piezoresistive sensor based on Longan leaves’ hierarchical microstructure for human-motion detection

Flexible pressure sensors exhibit great application potential in wearable devices, human-computer interaction, medical monitoring and other fields and thereby have attracted much attention. However, a common challenge in piezoresistive sensors is the trade-off between sensing range and sensitivity, which limits their versatility across various applications. By employing Longan leaf as a hierarchical microstructure template, the sensing range and sensitivity can be significantly improved at low cost, based on fractal theory. MXene was used as the sensing material, and a systematic study was conducted concerning the effect of MXene concentration on the sensor's sensitivity and detection range, which revealed that the sensitivity firstly increased and then decreased with MXene concentration ranging from 5 mg/mL to 30 mg/mL. Furthermore, based on the solution method to prepare composite sensing layer, MXene and rGO were mixed in proportion to obtain a sensing network that can coordinate with each other. When the volume ratio was 1:1, and the concentration of MXene was up to 15 mg/mL, the fabricated sensor demonstrated high sensitivity (47.36 kPa−1) and wide detection range (>100 kPa) at the same time. In addition, the sensor was successfully used to monitor various human joint activities and physiological signals, which was also expected to provide a competitive scheme for wearable flexible electronic devices in human-motion detection.

MXene-based self-adhesive, ultrasensitive, highly tough flexible hydrogel pressure sensors for motion monitoring and robotic tactile sensing

With the rapid development of artificial intelligence and human–computer interaction fields, wearable electronic devices represented by flexible pressure sensors have a wider range of application scenarios. In this paper, we used a composite hydrogel made by one-pot polymerization using a combination of acrylamide (AM), sodium carboxymethyl cellulose (CMC), dopamine (DA), and MXene (Ti3C2Tx). The composite hydrogel has the advantages of self-adhesion, high sensitivity, large detection range (0–500 kPa), low detection limit, high toughness (compression > 90 %), stability (more than 6700 s) and other advantages of hydrogel flexible pressure strain sensor. Sodium carboxymethyl cellulose (CMC), as a multifunctional thickener and gelling agent, significantly improves rheological properties and viscosity in hydrogels, forms crosslinked structures to enhance mechanical properties and provides reliable toughness for pressure sensors. Dopamine (DA), which is rich in catechol groups, can adhere to the material surface through oxidative polymerization to enhance bond strength. Hydrogel pressure sensors for human motion detection can accurately track joint movement, respond sensitively to changes in knocks and touches, and display specific characters such as Morse code. The technology extends to robotic systems, where pressure-activated electrical signals are used to control robot movements. Therefore, it has a very promising application in the field of human–computer interaction, human-body movement signal detection, etc. PAM/CMC/DA/MXene hydrogel pressure sensors have excellent mechanical, electrical, and adhesion properties, which provide a more reliable solution for the fields of flexible wearable e-skin, human–computer interaction, and tactile sensing.

Machine Learning-Based Gesture Recognition Glove: Design and Implementation.

In the evolving field of human-computer interaction (HCI), gesture recognition has emerged as a critical focus, with smart gloves equipped with sensors playing one of the most important roles. Despite the significance of dynamic gesture recognition, most research on data gloves has concentrated on static gestures, with only a small percentage addressing dynamic gestures or both. This study explores the development of a low-cost smart glove prototype designed to capture and classify dynamic hand gestures for game control and presents a prototype of data gloves equipped with five flex sensors, five force sensors, and one inertial measurement unit (IMU) sensor. To classify dynamic gestures, we developed a neural network-based classifier, utilizing a convolutional neural network (CNN) with three two-dimensional convolutional layers and rectified linear unit (ReLU) activation where its accuracy was 90%. The developed glove effectively captures dynamic gestures for game control, achieving high classification accuracy, precision, and recall, as evidenced by the confusion matrix and training metrics. Despite limitations in the number of gestures and participants, the solution offers a cost-effective and accurate approach to gesture recognition, with potential applications in VR/AR environments.

Double-layer stretchable composite conductive graphene-hydrogel with wide-range linear sensing and thermal-humidity management for health monitoring

Laser-induced graphene (LIG) is widely used in stretchable electrical conductors due to its excellent properties and low manufacturing cost. However, there is usually a mechanical mismatch or poor interface effect between LIG and the flexible substrate, and LIG will inevitably crack when stretched, which leads to LIG-based stretchable conductors usually have poor operating range, linearity and sensitivity stability. In this study, a double-layer stretchable composite conductive (DSCCGH) was prepared based on laser-induced MXene-decorated graphene (LIMG), with LIMG transferred to the hydrogel using a repeated reversible freeze-thaw method and a biomimetic structural design incorporated. The strong mechanical interlocking between LIMG and the hydrogel, along with the graphene-coated MXene derivative TiO2 providing additional electrical pathways when LIG’s conductive networks are disconnected, allowed this DSCCGH to achieve wide-range sensing linearity (0–366%, R2 = 0.9948) and stable sensitivity. This setup enabled stable and reliable muscle movement and pulse monitoring. Furthermore, biomimetic structural design imparts the DSCCGH with perspiration and cooling functions, enhancing comfort and wearability for outdoor exercise. This innovative design provides significant potential for future developments in wearable electronics and human-computer interaction fields.

MelTrans: Mel-Spectrogram Relationship-Learning for Speech Emotion Recognition via Transformers.

Speech emotion recognition (SER) is not only a ubiquitous aspect of everyday communication, but also a central focus in the field of human-computer interaction. However, SER faces several challenges, including difficulties in detecting subtle emotional nuances and the complicated task of recognizing speech emotions in noisy environments. To effectively address these challenges, we introduce a Transformer-based model called MelTrans, which is designed to distill critical clues from speech data by learning core features and long-range dependencies. At the heart of our approach is a dual-stream framework. Using the Transformer architecture as its foundation, MelTrans deciphers broad dependencies within speech mel-spectrograms, facilitating a nuanced understanding of emotional cues embedded in speech signals. Comprehensive experimental evaluations on the EmoDB (92.52%) and IEMOCAP (76.54%) datasets demonstrate the effectiveness of MelTrans. These results highlight MelTrans's ability to capture critical cues and long-range dependencies in speech data, setting a new benchmark within the context of these specific datasets. These results highlight the effectiveness of the proposed model in addressing the complex challenges posed by SER tasks.

Wushu Movement Recognition System Based on DTW Attitude Matching Algorithm

Motion recognition technology is widely used in intelligent video surveillance, human-computer interaction and other fields. With the development of computer vision technology, improving the accuracy and efficiency of motion recognition has become the focus of research. The purpose of this study is to improve the performance of Wushu movement recognition through improved dynamic time warping algorithm and hierarchical model. Firstly, a high-dimensional feature vector is constructed by using the position, velocity and Angle changes of human bone joints. The actions are subdivided by the hierarchical model, and matched and recognized by the max-minimum dynamic time regularization model. Meanwhile, the K-class mean algorithm is combined to optimize the type of tree core, improve the performance of the model, reduce the interference of noise nodes, and effectively classify Wushu actions. Experimental verification was carried out on four public data sets of KTH, Olympic Sports, Hollywood2 and HMDB51. The experimental results showed that the recognition rate of the proposed model in KTH data set was 95.2%, and that in Olympic Sports data set was 91.4%. The Hollywood2 dataset was 66.7%, and the HMDB51 dataset was 61.2%. Comparing the results of different algorithms, the proposed method improved the recognition performance by 10% compared with long short-term memory network and gated cycle unit. Compared with one-dimensional convolutional neural network, the time of the proposed method was 15s longer, but the recognition rate was 1.6% higher. The results showed that the proposed method had significant performance advantages in diverse and complex action recognition tasks. Meanwhile, the results emphasized the factors to be considered in the design of the model, demonstrating its effectiveness in the application of Wushu movement recognition.

Representing scents: An evaluation framework of scent-related experiences through associations between grounded and psychophysiological data

This study introduces an empirical approach for assessing human scent-related experiences within the field of Human-Computer Interaction (HCI). We labeled 43 fragrances based on grounded collective experience, incorporating semantic and impression-based data. Furthermore, we collected comprehensive psychophysiological data, including electroencephalogram (EEG), electrobulbogram (EBG), electrocardiogram (ECG), and facial dynamics captured by a camera, from participants who experienced the scents. By computing scent-wise similarity and correlating both grounded and psychophysiological scent spaces, we identified associations between them, demonstrating the potential of this approach to enhance our understanding of scent-related experiences. Additionally, we propose an iterative evaluation framework to refine the design of smell-based interactions and we conduct a real-life study to validate this framework.

Toward Adaptive Virtual Reality Systems: Understanding Emotional and Physiological Responses in VR Game

Virtual reality (VR) affective computing is expanding to include systems that can identify and respond to human emotions. This study explores emotional responses in VR using the game Pong to monitor these reactions under different gameplay conditions. It investigates how game pace and opponent behavior affect emotions, assessed through physiological metrics like electrocardiogram, galvanic skin response, electromyogram, and player behaviors. Results from three game variations—slow-paced, fast-paced, and lag-induced—showed distinct emotional and physiological responses. Significant differences were found in arousal and valence levels, with varying heart rate and skin conductance among the game conditions. These findings suggest that VR game design can significantly influence emotional states, providing insights that could help develop more engaging and responsive VR systems. Thus, this study demonstrates the potential of VR to evoke and measure specific emotional responses, contributing to the fields of human-computer interaction, and affective computing.

Flexible and self-healing tannic acid-Fe3O4@MXene-composited dual network hydrogel multifunctional strain sensor

In light of the exponential growth of flexible human-computer interactions and telecommunications, it is becoming increasingly important to develop a high-sensitivity strain sensor to monitor human movement. Herein, the biocompatible, adhesive, and flexible Tannic acid-Fe3O4@MXene-poly(methacryloxyethyl sulfobetaine/acrylamide) (TFM) hydrogel has been prepared to realize the dual-functional applications in strain sensitive sensor and electromagnetic interference (EMI) shielding. The polymer network and 2D conductive filler MXene together construct the conductive network inside the hydrogel. The interpenetrating network and hydrogen bond between poly(acrylamide) and poly(methacryloxyethyl sulfobetaine) chain enhance the mechanical property of hydrogel, which is also improved by the complexation between Fe3O4 and the catechol groups of tannic acid. Benefiting from the excellent electrical conductivity (∼1.33 S/m) and mechanical property of the TFM hydrogels, the strain sensor displays compressive strain (∼77.6 %) and elongation (∼308.5 %), adhesion (∼16 kPa), compression sensitivity (∼42.6 Pa−1), gauge factor (GF = 2.38), and cyclic stability (∼50 cycles). TFM hydrogels can monitor strain distribution sensing, pulse beating and human movement detection. Moreover, the TFM hydrogel exhibits an absorption-predominant EMI shielding performance with an effective value of ∼25.2 dB. This work provides a new inspiration for the development of multifunctional hydrogels in the fields of artificial intelligence, human-computer interaction, and EMI shielding technology.

Pollen-Modified Flat Silk Cocoon Pressure Sensors for Wearable Applications.

Microstructures have been proved as crucial factors for the sensing performance of flexible pressure sensors. In this study, polypyrrole (PPy)/sunflower pollen (SFP) (P/SFP) was prepared via the in situ growth of PPy on the surface of degreased SFP with a sea urchin-like microstructure; then, these P/SFP microspheres were sprayed onto a flat silk cocoon (FSC) to prepare a sensing layer P/SFP-FSC. PPy-FSC (P-FSC) was prepared as an electrode layer through the in situ polymerization of PPy on the FSC surface. The sensing layer P/SFP-FSC was placed between two P-FSC electrode layers to assemble a P/SFP-FSC pressure sensor together with a fork finger electrode. With 6 mg/cm2 of optimized sprayed P/SFP microspheres, the prepared flexible pressure sensor has a sensitivity of up to 0.128 KPa-1 in the range of 0-13.18 KPa and up to 0.13 KPa-1 in the range of 13.18-30.65 KPa, a fast response/recovery time (90 ms/80 ms), and a minimum detection limit as low as 40 Pa. This fabricated flexible P/SFP-FSC sensor can monitor human motion and can also be used for the encrypted transmission of important information via Morse code. In conclusion, the developed flexible P/SFP-FSC pressure sensor based on microstructure modification in this study shows good application prospects in the field of human-computer interaction and wearable electronic devices.

Assessing Feature Importance in Eye-Tracking Data within Virtual Reality Using Explainable Artificial Intelligence Techniques

Our research systematically investigates the cognitive and emotional processes revealed through eye movements within the context of virtual reality (VR) environments. We assess the utility of eye-tracking data for predicting emotional states in VR, employing explainable artificial intelligence (XAI) to advance the interpretability and transparency of our findings. Utilizing the VR Eyes: Emotions dataset (VREED) alongside an extra trees classifier enhanced by SHapley Additive ExPlanations (SHAP) and local interpretable model agnostic explanations (LIME), we rigorously evaluate the importance of various eye-tracking metrics. Our results identify significant correlations between metrics such as saccades, micro-saccades, blinks, and fixations and specific emotional states. The application of SHAP and LIME elucidates these relationships, providing deeper insights into the emotional responses triggered by VR. These findings suggest that variations in eye feature patterns serve as indicators of heightened emotional arousal. Not only do these insights advance our understanding of affective computing within VR, but they also highlight the potential for developing more responsive VR systems capable of adapting to user emotions in real-time. This research contributes significantly to the fields of human-computer interaction and psychological research, showcasing how XAI can bridge the gap between complex machine-learning models and practical applications, thereby facilitating the creation of reliable, user-sensitive VR experiences. Future research may explore the integration of multiple physiological signals to enhance emotion detection and interactive dynamics in VR.

Hand Gesture Recognition System using Deep Learning

This paper presents a comprehensive exploration of a hand gesture recognition system centered on the YOLOv5 object detection algorithm. The system is meticulously designed to fulfill specific objectives, including real-time gesture detection, high accuracy, scalability, robustness to environmental variability, and seamless integration with other system components. Drawing upon the strengths of YOLOv5, the proposed system aims to provide a reliable and efficient solution for hand gesture recognition, thereby enhancing human- computer interaction and enabling various applications across different domains. The methodology encompasses several stages, including data acquisition, preprocessing, object detection using YOLOv5, integration with other system modules, evaluation, and validation. Through a series of experiments and extensive testing in real-world scenarios, the effectiveness and usability of the system are thoroughly assessed, demonstrating its potential for practical deployment and widespread adoption. Additionally, this paper discusses the implications of the findings, identifies areas for further research and development, and offers insights into the future directions of hand gesture recognition systems utilizing YOLOv5. Overall, this study contributes to advancing the field of human-computer interaction and gesture recognition technology, paving the way for innovative applications and enhanced user experiences in various domains. Keywords— Hand Gesture Recognition, YOLOv5, Real-time Detection, Human-Computer Interaction, Object Detection, Machine Learning

Joint low-rank tensor fusion and cross-modal attention for multimodal physiological signals based emotion recognition

Objective. Physiological signals based emotion recognition is a prominent research domain in the field of human-computer interaction. Previous studies predominantly focused on unimodal data, giving limited attention to the interplay among multiple modalities. Within the scope of multimodal emotion recognition, integrating the information from diverse modalities and leveraging the complementary information are the two essential issues to obtain the robust representations. Approach. Thus, we propose a intermediate fusion strategy for combining low-rank tensor fusion with the cross-modal attention to enhance the fusion of electroencephalogram, electrooculogram, electromyography, and galvanic skin response. Firstly, handcrafted features from distinct modalities are individually fed to corresponding feature extractors to obtain latent features. Subsequently, low-rank tensor is fused to integrate the information by the modality interaction representation. Finally, a cross-modal attention module is employed to explore the potential relationships between the distinct latent features and modality interaction representation, and recalibrate the weights of different modalities. And the resultant representation is adopted for emotion recognition. Main results. Furthermore, to validate the effectiveness of the proposed method, we execute subject-independent experiments within the DEAP dataset. The proposed method has achieved the accuracies of 73.82% and 74.55% for valence and arousal classification. Significance. The results of extensive experiments verify the outstanding performance of the proposed method.

Emotion Recognition in Lhasa Tibetan Speech based on Bi-LSTM Graph Convolutional Networks

Speech Emotion Recognition (SER) is a crucial component in the field of Human-Computer Interaction (HCI), with significant research and practical application implications. However, due to the complexity of the Tibetan language and the scarcity of datasets caused by the difficulty in collecting various dialects, there are not many research achievements in Tibetan speech recognition. Based on the foundation of constructing a TBLS1 dataset containing 6,000 Tibetan-language speech samples, an approach was devised for Tibetan speech emotion recognition. This approach leverages MFCC features and incorporates a Bi-directional Long Short-Term Memory (Bi-LSTM) network within a graph convolutional neural network. Finally, by comparing the performance of different models on this dataset, we demonstrated the feasibility of our model for Tibetan speech emotion recognition.

Image Fusion of Multi-Temporal Images of Eyes Feature using Hybrid Approach of PCA and SWT

A wide variety of information is represented by an image feature, which may be applied in various applications such as mage fusion, video processing, medical diagnosis, traffic safety monitoring, visual surveillance, feature matching, image segmentation, pattern matching, person identification, sentiment analysis, human computer interaction and many more fields. This paper focused on multi-temporal feature extraction of eyes feature from human face images captured in two different time slots to reduce the semantic gap from images and to improve the image quality by image fusion using PCA, SWT and hybrid approach of PCA and SWT algorithm. In this paper, comparative study of multi-temporal image fusion using Principle Component Analysis (PCA), Stationary Wavelet Transform (SWT) algorithms and hybrid approach of PCA and SWT are employed and its experimental results are evaluated with its performance analysis. Image fusion performance is compared based on eight quantitative quality measures as SSIM, MSE, NAE, CC , SC , AD, SD and MI.The outcomes of comparison show that employing the hybrid approach of PCA+SWT transform can improve image fusion performance. The applicability of this work approach may have several uses when the utilization of human facial features is feasible.

A Systematic Process to Engineer Dependable Integration of Frame-based Input Devices in a Multimodal Input Chain: Application to Rehabilitation in Healthcare

Designing new input devices and associated interaction techniques is a key contribution in order to increase the bandwidth between users and interactive applications. In the field of Human-Computer Interaction, research and development services in industry and research laboratories in universities have been, since the invention of the mouse and the graphical user interface, proposing multiple contributions including the integration of multiple input devices in multimodal interaction technique. Those contributions, most of the time, are presented as prototypes or demonstrators providing evidence of the bandwidth increase through user studies. Such contributions however, do not provide any support to software developers for integrating them in real-life systems. When done, this integration is performed in a craft manner, outside required software engineering good practice. This paper proposes a systematic process to integrate novel input devices and associated interaction techniques to better support users' work. It exploits most recent interactive systems architectural model and formal model-based approaches for interactive systems supporting verification and validation when required. The paper focusses on Frame-based input devices which support gesture-based interactions and movements recognition but also addresses their multimodal use. This engineering approach is demonstrated on an interactive application in the area of rehabilitation in healthcare where dependability of interactions and applications is as critical as their usability.