Human-computer Interaction Research Articles

Multimodal human computer interaction of wheelchairs supporting lower limb active rehabilitation

Currently, an important challenge in stroke rehabilitation is how to effectively restore motor functions of lower limbs. This paper presents multimodal human computer interaction (HCI) of wheelchairs supporting lower limb active rehabilitation. First, multimodal HCI incorporating motor imagery electroencephalography (EEG), electromyography (EMG) and speech is designed. Second, prototype supporting wheelchair active rehabilitation method is illustrated in details. Third, the preliminary brain-computer interfaces (BCI) and speech recognition task experiments are carried out respectively, and the results are obtained. Finally, discussion is conducted and conclusion is drawn. This study has important practical significance in auxiliary movements and neurorehabilitation for stroke patients.

Deep learning-powered efficient characterization and quantification of microplastics

Characterizing and quantifying microplastics (MPs) are time-consuming and labor-intensive tasks traditionally. This paper presents an artificial intelligence (AI) framework aiming to automate these tasks by integrating computer vision and deep learning techniques. The approach leverages Fourier Transform Infrared (FTIR) spectra and visual images. Primary novelties of this research involve the development of: (1) an AI framework integrating efforts of data processing, analytics, visualization, and human-computer interaction; (2) a method for transforming FTIR data into contour images; (3) data augmentation strategies for resolving data scarcity and imbalance issues; (4) deep learning models for identifying MPs; (5) computer vision algorithms for quantifying MPs; and (6) an engineer-friendly graphic user interface (GUI) for enhancing data accessibility. The AI framework has been applied to polyethylene, polypropylene, polystyrene, polyamide, ethylene-vinyl acetate, and cellulose acetate. Results confirmed the efficacy of the framework, exhibiting high accuracy scores in classification (98 %), segmentation (99 %), and quantification (96 %) tasks. This research advances the capability of automatic assessment of MPs.

Defining social reward: A systematic review of human and animal studies.

Social rewards are strong drivers of behavior and fundamental to well-being, yet there is a lack of consensus regarding what actually defines a reward as "social." Because a systematic overview of existing social reward operationalizations is currently absent, a review of the literature seems necessary to advance toward a unified framework and to better guide research and theory. To bridge this gap, we preregistered and conducted the first comprehensive systematic review of human and animal experimental studies that used the term "social reward" and charted existing operationalizations, revealing the implicit and explicit definitions used in the field. Stimulus characteristics and measures of social reward were extracted from a total of 384 studies encompassing 42,118 participants and subjects. We provide detailed summaries of these elements, stratified by species (human/animal) and study type (behavioral, brain imaging, pharmacological, and physiological). Two main aspects were found to account for most of the difference in operationalizations: the sensory richness of a stimulus (intimacy) and engagement in social interaction (i.e., the synchronous observation and action between at least two individuals, viz., immediacy). Drawing insights from second-person neuroscience approaches and theoretical models in the field of human-computer interaction, we propose that human and animal research can greatly benefit from considering these properties, as they have important theoretical and practical consequences for human and translational research, with far-reaching implications for neighboring research fields such as those pertaining to social media and the development of artificial intelligence. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Trust in artificial intelligence: Producing ontological security through governmental visions

With developments in artificial intelligence (AI) widely framed as security concern in both military and civilian realms, governments have turned their attention to regulating and governing AI. In a study of United States (US), Chinese, and European Union (EU) AI documents, we go beyond instrumental understandings of AI as a technological capability, which serves states’ self-interests and the maintenance of their (supra)national security. Our specific interest lies in how AI policies tap into both problem-solving approaches and affective registers to achieve both physical and ontological securities. We find that in governmental visions, AI is perceived as a capability that enhances societal and geopolitical interests while its risks are framed as manageable. This echoes strands within human–computer interaction that draw on human-centered perceptions of technology and assumptions about human–AI relationships of trust. Despite different cultural and institutional settings, the visions of future AI development are shaped by this (shared) understanding of human–AI interaction, offering common ground in the navigation of innovation policies.

Exploring the Impact of AI Enhancement on the Sports App Community: Analyzing Human-Computer Interaction and Social Factors Using a Hybrid SEM-ANN Approach

The iteration of AI technology has enhanced the industry competitiveness of sports apps and brought a new emotional experience to users. However, the community function of AI-enabled sports apps has not yet received sufficient attention. This study adopts the technology acceptance model framework through the human-computer interaction and social psychology perspectives, incorporating perceived interactivity (human-human interaction, human-information interaction, human-system interaction), user experience (functional experience, content experience, and emotional experience), attachment theory (emotional attachment, emotional loyalty), social identity, social trust, and satisfaction variables to construct a theoretical model. It aims to solve the problem of how AI can enhance users’ emotional interaction, community stickiness, and ultimately social interaction satisfaction in sports apps. Through literature review, theoretical analysis, and case study, the Chinese “KEEP” APP was identified as the research object, and the questionnaire was constructed with an improved test scale. The study used a cross-sectional sampling method to collect data from Chinese online platforms for three months (N = 520) and tested the hypotheses in the theoretical model through structural equation modeling and artificial neural networks. The results of the study show that (1) emotional attachment, emotional loyalty, social identity, and social trust positively and significantly affect user satisfaction; (2) emotional attachment and social identity positively and significantly affect human-human interaction; (3) perceived usefulness, social identity, and social trust significantly construct an influence loop on the internal path. Compared with existing studies, this study proposes a multidimensional innovative framework that reveals the deeper influence mechanism of AI technology in sports APP communities and makes up for the shortcomings of existing studies in terms of users’ emotional interactions and community stickiness. Through this study, sports APP developers and operators, marketers, AI technology developers, as well as sports enthusiasts, and general users can obtain specific practical suggestions to further optimize the design and functionality of sports apps, and enhance the overall user experience and satisfaction.

EIDU-Net: edge-preserved inception DenseGCN U-Net for LiDAR point cloud segmentation

With the development of laser scanners and machine learning, point cloud semantic segmentation plays a significant role in autonomous driving, scene reconstruction, human-computer interaction, and other fields. In recent years, point cloud semantic segmentation based on deep learning has become one of the key research directions in point cloud processing. Due to the limited ability to exploit geometric details and contextual information in point clouds, most methods that adopt encoder-decoder architecture lose local structural information easily, especially detailed features, and extract features insufficiently. To address this issue, the edge-preserving inception DenseGCN U-Net (entitled as EIDU-Net) is proposed. EIDU-Net makes full use of the complementation between geometric details in the original point cloud and high-level features. The edge-preserved graph pooling (EGP) layer, the key module of the EIDU-Net, is designed to retain additional edge feature information from the original point cloud during pooling operations. Accordingly, the edge-preserved graph unpooling (EGU) layer can restore the feature graph more efficiently based on the additionally retained edge features. Extensive experiments demonstrate that our proposed EIDU-Net has remarkable improvements on semantic segmentation tasks under whatever S3DIS or Terracotta Warrior fragments. Our code is publicly available at https://github.com/caoxin918/EIDU-Net.

Grasp with push policy for multi-finger dexterity hand based on deep reinforcement learning

When interacting with the external environment, dexterous hands face various challenges, including interference in cluttered environments and difficulty in accurately locating target objects. We observe that the human hand is often at ease in the face of complex unstructured environments: pushing objects that can rearrange clutter, locating target objects, and creating space for fingers. In addition, the gripping action complements the push by achieving the precise movement of unrelated objects. Taking inspiration from this, to enable manipulators to perform accurate grasping tasks in complex environments, we propose a new data-driven grasping approach for robotic dexterous hands: the push-fit grasping approach. This method combines human grasping with model-free deep reinforcement learning to realize the robot's collaborative grasping ability. An end-to-end conditional converter grasping network is first trained, which takes the visual point cloud input to the action output. We use DQN algorithms for strategy training, further enhanced by fine-tuning methods for real-world learning. The experimental results show that the push strategy we learned significantly improves the grasping performance, and the success rate of cooperative grasping is increased by 8 %. It is worth noting that this fine-tuning method significantly reduces the actual training time, with the real world training taking up only one-fifth of the time in the simulated world. Even in demanding scenarios characterized by confusion and complexity, it facilitates rapid learning. This study provides new insights for effectively addressing complex challenges involving multi-fingered dexterous hands and human-computer interaction. Our code is available in the github repository.

Substantive and Emotional: The Anthropomorphic Evolution of Generative AITaking GPT-4o as an Example

Under the background of the rapid development of artificial intelligence technology, "personal publishing" represented by blog in the past has transformed into "human-computer co-creation", showing a new human-computer interaction relationship, and generative artificial intelligence has become the "collaborator" of human content creation. The flagship multimodal large language model "anthropomorphism" represented by GPT-4o is evolving from "linguistic anthropomorphism" to "emotional anthropomorphism". Based on the anthropomorphic theory, this paper aims to reveal that GPT-4o, while demonstrating its linguistic intelligence in the human-machine co-created dialogue, simulates human emotional response, showing empathy and "resonance", that is, the "anthropomorphic" process at the two levels of "words have things" and "words have feelings". It also reflects on the potential ethical challenges of the anthropomorphic evolution of generative AI to future human society.

High-Performance Flexible PLA/BTO-Based Pressure Sensor for Motion Monitoring and Human-Computer Interaction.

Flexible electronics show wide application prospects in electronic skin, health monitoring, and human-machine interfacing. As an essential part of flexible electronics, flexible pressure sensors have become a compelling subject of academic research. There is an urgent need to develop piezoelectric sensors with high sensitivity and stability. In this work, the high flexibility of polylactic acid (PLA) film and the excellent ferroelectric properties and high dielectric constant of tetragonal barium titanate (BTO) led to their use as filling materials to fabricate flexible piezoelectric composite films by spinning coating. PLA is used to produce flexible binding substrates, and BTO is added to the composite to enhance its electrical output by improving its piezoelectric performance. The peak output voltage of the PLA/BTO tetragonal piezoelectric film is 22.57 V, and the maximum short-circuit current was 3041 nA. Durability tests showed that during 40,000 s of continuous operation, in the range of 15~120 kPa, the linear relationship between pressure and the film was excellent, the sensitivity for the output voltage is 0.176 V/kPa, and the output current is 27.77 nA/kPa. The piezoelectric pressure sensor (PPS) also enables accurate motion detection, and the extensive capabilities of the PENG highlight its potential in advancing motion sensing and human-computer interactions.

Intentional or Designed? The Impact of Stance Attribution on Cognitive Processing of Generative AI Service Failures.

With the rapid expansion of the generative AI market, conducting in-depth research on cognitive conflicts in human-computer interaction is crucial for optimizing user experience and improving the quality of interactions with AI systems. However, existing studies insufficiently explore the role of user cognitive conflicts and the explanation of stance attribution in the design of human-computer interactions. This research, grounded in mental models theory and employing an improved version of the oddball paradigm, utilizes Event-Related Spectral Perturbations (ERSP) and functional connectivity analysis to reveal how task types and stance attribution explanations in generative AI influence users' unconscious cognitive processing mechanisms during service failures. The results indicate that under design stance explanations, the ERSP and Phase Locking Value (PLV) in the theta frequency band were significantly lower for emotional task failures than mechanical task failures. In the case of emotional task failures, the ERSP and PLV in the theta frequency band induced by intentional stance explanations were significantly higher than those induced by design stance explanations. This study found that stance attribution explanations profoundly affect users' mental models of AI, which determine their responses to service failure.

Next-gen strain sensors: Self-healing, ultra-sensitive, lightweight, and durable MWCNT-silicone rubber for advanced human motion tracking

Strain sensors that have flexible wearable forms can be applied in different aspects, such as in soft robotics, human-computer interaction, motion monitoring, and medical treatment process. The main challenge is creating lightweight, flexible sensors with high sensitivity. In this study, we developed flexible strain sensors using a simple sandwich manufacturing method. To achieve this, multi-walled carbon nanotubes (MWCNTs) were chosen for their excellent conductivity, while silicone rubber (SR) provided a flexible and durable substrate. Advanced SEM-EDX and FTIR analyses were conducted to assess the material structure. Furthermore, the electromechanical performance of the sensors was thoroughly evaluated, highlighting their potential in real-world applications. Electromechanical performance evaluation showed that the SR/0.7-MWCNT/SR sensor in the 0–90 % strain range had a sensitivity of 34.47, an increase of 53.35 %, and a linearity of 0.978, an increase of 10 % compared to the SR/0.5-MWCNT/SR sensor. In addition, the sensor has extremely fast response and recovery times of 65 ms and 60 ms, respectively. The sensor is also able to withstand up to 1.200 cycles of stretching, twisting, and bending. The sensor also exhibited good autonomous electromechanical self-healing properties against physical damage. The sensor developed in the present study are proven to be used to detect human movements such as finger bending, wrist, elbow, and knee movements.

Insights from the evaluation of a persuasive intervention for absent-minded smartphone use

ABSTRACT Problematic smartphone use (PSU) is a growing concern that hurts users' daily lives and activities. Previous studies have emphasised the importance of developing mindfulness and self-efficacy concerning smartphone use rather than solely focussing on reducing usage. However, there has been little research developing and evaluating digital interventions specifically targeting absent-minded smartphone use. There is also little knowledge on how best mindfulness practice using digital technologies can be integrated with persuasive designs which have been widely studied and implemented for behaviour change. In this paper, we developed a live wallpaper application for Android lock and home screens as a mindfulness-based intervention for absent-minded smartphone use. The application was evaluated over two weeks with 121 participants. This was followed by a semi-structured interview with 15 participants. The results of our analysis show that the intervention reduced absent-minded smartphone use overall. While participants found the various features (especially the customisation features) of the intervention to be persuasive, we found no correlation between perceived persuasiveness and behaviour change. This work provides valuable insights for advancing human-computer interaction (HCI) research on PSU. Additionally, the results raise important questions for future research, such as the relationship between perceived persuasiveness and behaviour change. Finally, we contribute to the discussion on the applications of mindfulness in persuasive technology, challenges and future research areas.

CNTs@fabric/Ni@PU piezoresistive sensor with enhanced interfacial contact resistance variation for motion detection and deep-learning-assisted recognition

Abstract Flexible piezoresistive sensors based on the mechanism of interfacial contact resistance change are receiving increasing attention in the fields of human-computer interaction, health monitoring, and behavior tracking. However, the high cost and complex manufacturing process limit the wide application and development of these flexible piezoresistive sensors. Here, a novel carbon nanotubes@fabric (CNTs@fabric)/Ni@polyurethane (Ni@PU) piezoresistive sensor (CNPS) with low-cost, simple-preparation and high-sensitivity was proposed. The effective contact area is obtained by synergizing the woven micro-convex structure of the fabric, the large specific surface area of the CNTs and the porous three dimensional electrodes. Within the small pressure (0–9.52 kPa) effect, the area of connection with the electrodes to the active layer plays a dominant role, resulting in a sensitivity of up to 6.39 kPa−1 for CNPS. In the high pressure region (9.52–44.92 kPa), where internal mechanism of change in the sensitive material dominants, the CNPS has a response time of 85 ms at a constant pressure of 28.31 kPa. Considering the excellent output electrical performances, a variety of body movements could be detected by fixing the CNPS to different joints. Significantly, the designed intelligent object recognition system implemented by the combination of matrix stress detection module and residual neural network (ResNet) algorithm has a recognition accuracy of 99.26%. Enhancing the interfacial contact resistance change mechanism using a simple fabrication process offers a promising strategy for the rapid development of flexible piezoresistive sensors.

Simulation Design and Research of Flame Generation in Virtual Fire Scene Based on Unity3D

Since the 21st century, with the continuous development and progress of science and technology, the possibility of fire danger is also rising, the requirements of public fire safety management have also been improved accordingly, and the problems in its fire safety management work make it difficult to meet the management requirements. The selection of firefighters has also become the top priority of management, good psychological quality of firefighters can keep calm in the face of fire explosions and various emergencies, and can make the most correct choice to complete the rescue and fire fighting work. In this paper, a building in a certain location is selected as the location to simulate the scene of fire. Through human-computer interaction to judge the psychological quality of firefighters. First use major modeling software such as 3ds Max, Adobe Fusecc, or find material in the Unity3D store to create models of fire scenes and people and import them into Unity3D. Using Unity3D's own particle system component to build flame models, adding collision body components to all models to detect their collision detection mechanism and the mechanism of flame spreading combustion. A Scenario-based Communication Ability Evaluation System Based on Traditional Cognitive Behavior Measurement and Advanced Eye Movement, EEG and NIR Acquisition Techniques.

Human or Humanoid Animated Pedagogical Avatars in Video Lectures: The Impact of the Knowledge Type on Learning Outcomes

This study examines the impacts of educational videos with human or humanoid animated pedagogical avatars (APAs) on cognitive, affective, and social learning outcomes in declarative and procedural knowledge contexts. A 2 × 2 factorial design was used, considering instructor type (human vs. humanoid APA) and knowledge type (declarative vs. procedural). The study involved 139 university students with no prior knowledge of the video content and was conducted in a human-computer interaction lab using an eye-tracking device. There were no significant differences in learning achievement, perceived lecture engagement, emotion, satisfaction, and social presence between the human and humanoid APA. However, learners showed greater visual interest in the human instructor, which influenced their interest in the learning content. Viewers of declarative knowledge videos showed more visual interest and achieved higher learning scores, while procedural videos increased lecture engagement, emotion, satisfaction, and social presence. The study suggests that humanoid APAs can effectively replace human instructors in educational videos and emphasizes the importance of considering knowledge types in video design to impact learning outcomes.

Cross-Modal Harmony: Low-Rank Fusion for Enhanced Artificial Emotion Recognition

Emotion recognition has become a subject of considerable interest in recent times, owing to its diverse and far-reaching applications in various fields. These applications span from enhancing human-computer interactions to assessing mental health and improving entertainment systems. The proposed study presents a novel approach for emotion recognition by fusing audio and video modalities using low-rank fusion techniques. The proposed methodology leverages the complementary nature of audio and video data in capturing emotional cues. Audio data often encapsulates tone, speech patterns, and vocal nuances, while video data captures facial expressions, body language, and gestures. However, the challenge lies in effectively integrating these two modalities to enhance recognition accuracy. To address the challenge, it employs low-rank fusion, a dimensionality reduction technique that extracts the most informative features from both modalities while minimizing redundancy. Furthermore, it presents the implementation of the chosen low-rank fusion algorithm in a real-world emotion recognition system. The results can contribute to advancing the field of emotion recognition by providing a practical and efficient solution for combining audio and video data to achieve more robust and accurate emotion classification. Keywords: Deep Learning; Emotion Recognition; Human-Computer Interaction; Low-Rank Fusion; Multimodal Fusion.

Ionic Potential Relaxation Effect in a Hydrogel Enabling Synapse-Like Information Processing.

The next-generation brain-like intelligence based on neuromorphic architectures emphasizes learning the ionic language of the brain, aiming for efficient brain-like computation and seamless human-computer interaction. Ionic neuromorphic devices, with ions serving as information carriers, provide possibilities to achieve this goal. Soft and biocompatible ionic conductive hydrogels are an ideal substrate for constructing ionic neuromorphic devices, but it remains a challenge to modulate the ion transport behavior in hydrogels to mimic neuroelectric signals. Here, we describe an ionic potential relaxation effect in a hydrogel device prepared by sandwiching a layer of polycationic hydrogel (CH) between two layers of neutral hydrogel (NH), allowing this device to simulate various electrical signal patterns observed in biological synapses, including short- and long-term plasticity patterns. Theoretical and experimental results show that the selective permeation and hysteretic diffusion of ions caused by the anion selectivity of the CH layer are responsible for potential relaxation. Such an effect allows us with hydrogels to enable synapse-like information processing functions, including tactile perception, learning, memory, and neuromorphic computing. Additionally, the hydrogel device can operate stably even under 180° bending and 50% tensile strain, expanding the pathway for implementing advanced brain-like intelligent systems.

Graded Nanotexturing Architectural Wearable Triboelectric Sensor for Programmable Haptic Exploration.

Emulating biological perception mechanisms to construct intelligent sensing devices and systems represents a paradigm for promoting human-computer interaction in the Internet of Everything era. Nonetheless, developing highly sensitive, real-time sensing and rapidly integrated intelligent interaction units remains a challenging and time-consuming endeavor. This study employs a low-temperature glow discharge technique to rapidly fabricate graded nanotexturing architectural triboelectric nanopaper, upon which wearable triboelectric sensors for real-time tactile detection are designed. The structure enhances the contact area under an external force. Additionally, the Z-stacking structure design enables the sensor to achieve a remarkable sensitivity of 10.3 kPa-1 and a rapid response time of 52 ms. Furthermore, a tactile sensor array was designed to demonstrate the triboelectric sensor's ability to recognize characteristic pressures. With programmable machine learning techniques, the object recognition rate reached 97%. This study supports material structural design across disciplines, laying a solid foundation for the rapid fabrication and integration of transient wearable electronics.

Enhanced facial expression recognition using transfer learning and M-CLAHE

In recent years, Facial Expression Recognition (FER) techniques have gained substantial attention within the realm of biometric technology due to their wide range of applications, including emotion analysis, human-computer interaction, and surveillance systems. This paper presents a robust and efficient FER system composed of three key steps. First, precise face detection is performed using the Viola-Jones algorithm, a well-established method for detecting facial features in real-time. Second, the detected images are enhanced using a Modified Contrast Limited Adaptive Histogram Equalization (M-CLAHE) technique to improve contrast and visibility. Finally, feature extraction and classification are carried out using three powerful Convolutional Neural Network (CNN) architectures: VGG16, ResNet50, and Inception-v3, all benefiting from Transfer Learning to boost performance. Experiments were conducted on two widely-used datasets, JAFEE and CK+, with Inception-v3 achieving remarkable accuracy, reaching 98.43% and 99.93% on the respective databases. These results underline the effectiveness and robustness of our approach, particularly through the use of Transfer Learning, which significantly enhances the overall performance of the model compared to existing methods.

Fabrication of carbon nanotube neuromorphic thin film transistor arrays and their applications for flexible olfactory-visual multisensory synergy recognition

Abstract Artificial multisensory devices play a key role for the human-computer interaction in the field of artificial intelligence (AI). In this work, we have designed and constructed a novel olfactory-visual bimodal neuromorphic carbon nanotube thin film transistor (TFT) arrays for artificial olfactory-visual multisensory synergy recognition with the ultralow single-pulse power consumption of 25 aJ, employing semiconducting single-walled carbon nanotubes (sc-SWCNTs) as channel materials and gas sensitive materials, and poly[[4,8-bis[5-(2-ethylhexyl)-2-thienyl]benzo[1,2-b:4,5-b0]dithiophene-2,6-diyl]-2,5-thiophenediyl-[5,7-bis(2-ethylhexyl)-4,8-dioxo-4H,8H-benzo[1,2-c:4,5-c0]dithio-phene-1,3-diyl]] (PBDB-T) as the photosensitive material. It is noted that it is the first time to realize the simulation of olfactory and visual senses (from 280 nm to 650 nm) with the wide operating temperature range (0-150 °C) in a single SWCNT TFT device and successfully simulate the recovery of olfactory senses after COVID-19 by olfactory-visual synergy. Furthermore, our SWCNT neuromorphic TFT devices with high Ion/Ioff ratio (up to 106) at a low operating voltage (-2-0.5 V) can mimic not only the basic biological synaptic functions of olfaction and vision (such as paired-pulse facilitation, short-term plasticity, and long-term plasticity), but also optical wireless communication by Morse code. The proposed multisensory, broadband light-responsive, low-power synaptic devices provide great potential for developing AI robots to face complex external environments.