Human-computer Interaction Research Articles

Plug-and-play segment anything model improves nnUNet performance.

The automatic segmentation of medical images has widespread applications in modern clinical workflows. The Segment Anything Model (SAM), a recent development of foundational models in computer vision, has become a universal tool for image segmentation without the need for specific domain training. However, SAM's reliance on prompts necessitates human-computer interaction during the inference process. Its performance on specific domains can also be limited without additional adaptation. In contrast, traditional models like nnUNet are designed to perform segmentation tasks automatically during inference and can work well for each specific domain, but they require extensive training on domain-specificdatasets. To leverage the advantages of both foundational and domain-specific models and achieve fully automated segmentation with limited training samples, we propose nnSAM, which combines the robust feature extraction capabilities of SAM with the automatic configuration abilities of nnUNet to enhance the accuracy and robustness of medical image segmentation on smalldatasets. We propose the nnSAM model for small sample medical image segmentation. We made optimizations for this goal via two main approaches: first, we integrated the feature extraction capabilities of SAM with the automatic configuration advantages of nnUNet, which enables robust feature extraction and domain-specific adaptation on small datasets. Second, during the training process, we designed a boundary shape supervision loss based on level set functions and curvature calculations, enabling the model to learn anatomical shape priors from limited annotationdata. We conducted quantitative and qualitative assessments on the performance of our proposed method on four segmentation tasks: brain white matter, liver, lung, and heart segmentation. Our method achieved the best performance across all tasks. Specifically, in brain white matter segmentation using 20 training samples, nnSAM achieved the highest DICE score of 82.77 ( 10.12) % and the lowest average surface distance (ASD) of 1.14 ( 1.03) mm, compared to nnUNet, which had a DICE score of 79.25 ( 17.24) % and an ASD of 1.36 ( 1.63) mm. A sample size study shows that the advantage of nnSAM becomes more prominent under fewer trainingsamples. A comprehensive evaluation of multiple small-sample segmentation tasks demonstrates significant improvements in segmentation performance by nnSAM, highlighting the vast potential of small-samplelearning.

Multimodal human-computer interaction in interventional radiology and surgery: a systematic literature review.

As technology advances, more research dedicated to medical interactive systems emphasizes the integration of touchless and multimodal interaction (MMI). Particularly in surgical and interventional settings, this approach is advantageous because it maintains sterility and promotes a natural interaction. Past reviews have focused on investigating MMI in terms of technology and interaction with robots. However, none has put particular emphasis on analyzing these kind of interactions for surgical and interventional scenarios. Two databases were included in the query to search for relevant publications within the past 10 years. After identification, two screening steps followed which included eligibility criteria. A forward/backward search was added to identify more relevant publications. The analysis incorporated the clustering of references in terms of addressed medical field, input and output modalities, and challenges regarding the development and evaluation. A sample of 31 references was obtained (16 journal articles, 15 conference papers). MMI was predominantly developed for laparoscopy and radiology and interaction with image viewers. The majority implemented two input modalities, with voice-hand interaction being the most common combination-voice for discrete and hand for continuous navigation tasks. The application of gaze, body, and facial control is minimal, primarily because of ergonomic concerns. Feedback was included in 81% publications, of which visual cues were most often applied. This work systematically reviews MMI for surgical and interventional scenarios over the past decade. In future research endeavors, we propose an enhanced focus on conducting in-depth analyses of the considered use cases and the application of standardized evaluation methods. Moreover, insights from various sectors, including but not limited to the gaming sector, should be exploited.

Bioinspired Three-Mode Photosensitive Synaptic LED for Optical Information Processing.

Inspired by human sense organs, AI is advancing toward multimodal perception, with display technology evolving into intelligent human-computer interaction tools. However, hardware networks with multimodal responses connected by different devices bring problems such as delayed information transfer and inefficiency. Thus, an innovative three-mode photosensitive synaptic LED (PSSL) is first proposed by adding a photosensitive layer indacenodithiophene-benzothiadiazole (IDTBT) to the quantum-dot light-emitting diode (QLED), switched by changing the bias voltage. The self-powered PSSL has a photoresponse range from 310 nm to 808 nm (ultraviolet-near-infrared, UV-NIR). The device exhibits a bipolar response under red and UV light at 1 V. When the voltage reaches the turn-on voltage, the PSSL device turns into a neuromorphic LED, exhibiting conductivity enhancement under red-light irradiation and suppression under UV-light irradiation. As a result, the PSSLs are expected to be applied in the field of optical encryption communication and in neuromorphic display.

A Novel Natural Language Processing Model Transfer Strategy Tailored for Deep Learning Platforms

As artificial intelligence technology continues to evolve, numerous advancements enable smoother communication and collaboration between humans and computers. Natural language processing (NLP), as a key direction in the field of computer and artificial intelligence, has been attracting much attention. It focuses on the language used by people in daily life, devoting itself to improving the ability of language generation and understanding, thus making communication between people and computers more fluent and natural, effectively breaking through the problem of human-computer interaction. With the advancement of deep learning technology, the migration of NLP models onto deep learning platforms has emerged as a key trend. Model transfer, a crucial aspect of deep learning, holds significant value in enhancing the performance and efficiency of NLP models. This paper begins by outlining the fundamentals of deep learning platforms and NLP model transfer, followed by a comprehensive examination of current research progress and challenges in this field. Subsequently, we introduce a novel NLP model transfer strategy tailored for deep learning platforms and validate its effectiveness through rigorous experiments. In conclusion, the paper highlights our noteworthy advancements, pointing toward promising future developments.

Human-machine interaction with intelligent machine learning algorithms for smart classrooms

ABSTRACT The modern technology of learning and teaching in education has moved on to smart classes. The introduction of artificial intelligence (AI) in the education sector has essentially taken over traditional classrooms and transformed how education leads to admiration. Various data types, such as videos, images, sound, text, and biometric data like eye-tracking potentially captured through smart classrooms, pose new issues and challenges for artificial intelligence algorithms. Hence, this paper suggests the Machine Learning Assisted Intelligent Human-Machine Interaction (MLIHMI) model for the smart classroom monitoring system. The real-time interactive dimension signifies the capability to support the Smart classroom’s teaching and human-computer interactions, including smooth interaction, convenient operation, and interactive monitoring. This paper used ML and HMI models to introduce a new effective database for online learning and smart classroom environments utilising the learners’ body postures, facial expressions, and hand gestures. The numerical findings show that the suggested system achieves less error rate of 10.1% and enhances the accuracy rate of 96.5%, with the highest classification ratio of 95.6%, prediction rate of 93.6%, the detection rate of 97.3%, and engagement rate of 81.3% in student gesture and postures recognition and the identification of facial expressions with the high performance of 93.3% when compared to other existing models.

Study on the Design of a Non-Contact Interaction System Using Gestures: Framework and Case Study

The interaction patterns are constantly changing as the rapid development in technology. The non-contact interaction system using gestures is becoming important in human-computer interaction nowadays. This paper illustrates a design framework of a non-contact interaction system using gestures, which fully considers the approach to fit non-contact gestures into the system. The corresponding user requirements were investigated by surveys, and the universal design guide of non-contact gestures was summarized by statistical analysis. A case study was illustrated for key points of the developed design framework. The scheme, including interaction architecture, gesture definition, and user interface design, was described. The rapid prototyping was built. Evaluations, including eye-tracking measurements and tests of system usability, were implemented and analyzed by statistical methods to present suggestions of iterative design. The feasibility of the developed framework was verified. By statistical methods, the relationship between the interaction usability and the user’s perception for information was discussed, and the user’s perception for the display of dynamic and static gestures was analyzed. The results provide a clear approach to satisfy user’s preferences and improve the interaction quality for non-contact interaction systems using gestures, and furnish empirical grounding for sustainably devising similar interaction systems.

MNCATM: A Multi-Layer Non-Uniform Coding-Based Adaptive Transmission Method for 360° Video

With the rapid development of multimedia services and smart devices, 360-degree video has enhanced the user viewing experience, ushering in a new era of immersive human–computer interaction. These technologies are increasingly integrating everyday life, including gaming, education, and healthcare. However, the uneven spatiotemporal distribution of wireless resources presents significant challenges for the transmission of ultra-high-definition 360-degree video streaming. To address this issue, this paper proposes a multi-layer non-uniform coding-based adaptive transmission method for 360° video (MNCATM). This method optimizes video caching and transmission by dividing non-uniform tiles and leveraging users’ dynamic field of view (FoV) information and the multi-bitrate characteristics of video content. First, the video transmission process is formalized and modeled, and an adaptive transmission optimization framework for a non-uniform video is proposed. Based on this, the optimization problem required by the paper is summarized, and an algorithm is proposed to solve the problem. Simulation experiments demonstrate that the proposed method, MNCATM, outperforms existing transmission schemes in terms of bandwidth utilization and user quality of experience (QoE). MNCATM can effectively utilize network bandwidth, reduce latency, improve transmission efficiency, and maximize user experience quality.

A Highly Sensitive Strain Sensor with Self-Assembled MXene/Multi-Walled Carbon Nanotube Sliding Networks for Gesture Recognition

Flexible electronics is pursuing a new generation of electronic skin and human–computer interaction. However, effectively detecting large dynamic ranges and highly sensitive human movements remains a challenge. In this study, flexible strain sensors with a self-assembled PDMS/MXene/MWCNT structure are fabricated, in which MXene particles are wrapped and bridged by dense MWCNTs, forming complex sliding conductive networks. Therefore, the strain sensor possesses an impressive sensitivity (gauge factor = 646) and 40% response range. Moreover, a fast response time of 280 ms and detection limit of 0.05% are achieved. The high performance enables good prospects in human detection, like human movement and pulse signals for healthcare. It is also applied to wearable smart data gloves, in which the CNN algorithm is utilized to identify 15 gestures, and the final recognition rate is up to 95%. This comprehensive performance strain sensor is designed for a wide array of human body detection applications and wearable intelligent systems.

Usability in human-robot collaborative workspaces

AbstractThis study explores the usability of human-robot collaboration in the previously under-researched field of forestry and agroforestry. The robotic platforms used were Boston Dynamics Spot and the Agile X Bunker, the latter equipped with a movable arm. The research was conducted in an experimental robotic test park, simulating real-world scenarios relevant to forestry and agriculture. The focus of this study is on the use of these robots as collaborative robots (cobots). Usability, as a central characteristic in human-computer interaction, was evaluated using the well-established System Usability Scale (SUS). The results demonstrate the potential of these robotic systems to enhance productivity and safety, while also underscoring the importance of user-centered design in the development of collaborative tools. A key finding of this work is that successful integration of AI-driven technologies in sectors such as forestry and agriculture requires a focus on human-centered AI which includes good usability, and accessibility, emphasizing the importance of the concept of universal access.

Zooming In: A Review of Designing for Photo Taking in Human-Computer Interaction and Future Prospects

Photography has been pivotal in culture for decades and its importance has increased with the rise of digital technology. However, the exploration of picture taking within the Human-Computer Interaction (HCI) community does not seem to match its cultural and technological significance. Recognizing this discrepancy, we sought to understand areas of interest in photography as a conscious creative process. To address this issue, we performed a systematic literature review using the PRISMA methodology. From our research, we identified 62 pertinent papers spanning from 2005 to 2022. Our examination revealed six primary dimensions, further classified into study type, design goal, photo-taking style, device type, interaction style, and number of users. In-depth analysis showed the dominant role of exploratory and functional research, the balance between qualitative and quantitative methods, and a strong focus on smartphone cameras. Our review has highlighted significant gaps in the existing literature, offering valuable insights for future research on photo taking.

Generative AI insights in tourism and hospitality: A comprehensive review and strategic research roadmap

This study used bibliometric analysis and a systematic literature review (SLR) to examine how the tourism and hospitality industries use generative artificial intelligence (GAI), identifying developed patterns, theoretical frameworks, strengths and limitations, and future research challenges. We conducted a systematic review using the Scopus database, adhering to PRISMA principles. We analyzed a sample of 25 articles published between 2019 and 2023 through narrative synthesis and bibliometric analysis using the VOSviewer software, a tool for visualizing network analysis. The USA, China, India, and Saudi Arabia are the major countries engaged in GAI research in tourism and hospitality. Significant research topics emphasize decision-making, chatbots, deep learning, and sentiment analysis, mainly through the Technology Acceptance Model (TAM), Stimulus-Organism-Response (S-O-R), and Human-Computer Interaction (HCI) frameworks. GAI applications demonstrate strength in improving user experience and operational efficiency, though gaps exist in scope, ethics, technology performance, and collaboration between humans and AI. This study, therefore, provides a fundamental foundation for understanding the current status of GAI research in tourism and hospitality by pointing out some trends and areas that require further investigation to ensure the responsible and effective integration of AI within the industry.

Microstructured Self-Healing Flexible Tactile Sensors Inspired by Bamboo Leaves.

Wearable electronic devices with multifunctions such as flexible, integrated, and self-powered play a crucial role in the fields of health monitoring, motion monitoring, and human-computer interaction. However, their core basic components, flexible pressure sensors, face challenges including poor long-term stability and insufficient real-time sensing accuracy. In order to solve the challenges of long-term, stable, and accurate sensing of the sensor, this paper prepares polydimethylsiloxane (SHPDMS) with intrinsic self-healing property and designs a high-sensitivity self-healing capacitive flexible pressure sensor with dual microstructures (grating microstructured electrodes and microporous dielectric layer) as the substrate based on SHPDMS. Specifically speaking, the self-healing of the sensor under mild conditions was realized by introducing reversible imine bonds with low bonding energy into the polydimethylsiloxane (PDMS) flexible substrate, which solved the problem of the material's long-term service durability. A grating-like microstructure was introduced into the flexible electrode by using a spotted bamboo taro leaf as a template, and a dual microstructure sensor was constructed by combining it with a microporous dielectric layer doped with single-walled carbon nanotubes. This way reduces the elastic modulus of the dielectric layer, improves the dielectric constant of the sensor under loading, and thus significantly improves the sensor's sensitivity and extends the measurement accuracy in a low-stress range. The prepared self-healing flexible sensor achieves a sensitivity of 3.6 kPa-1, a minimum detection limit of 5 Pa, a response recovery time of less than 80 ms, and stability over 5000 cycles, which exceeds most previously reported silicone rubber-based capacitive flexible sensors.

Hybrid deep models for parallel feature extraction and enhanced emotion state classification

Emotions play a vital role in recognizing a person’s thoughts and vary significantly with stress levels. Emotion and stress classification have gained considerable attention in robotics and artificial intelligence applications. While numerous methods based on machine learning techniques provide average classification performance, recent deep learning approaches offer enhanced results. This research presents a hybrid deep learning model that extracts features using AlexNet and DenseNet models, followed by feature fusion and dimensionality reduction via Principal Component Analysis (PCA). The reduced features are then classified using a multi-class Support Vector Machine (SVM) to categorize different types of emotions. The proposed model was evaluated using the DEAP and EEG Brainwave datasets, both well-suited for emotion analysis due to their comprehensive EEG signal recordings and diverse emotional stimuli. The DEAP dataset includes EEG signals from 32 participants who watched 40 one-minute music videos, while the EEG Brainwave dataset categorizes emotions into positive, negative, and neutral based on EEG recordings from participants exposed to six different film clips. The proposed model achieved an accuracy of 95.54% and 97.26% for valence and arousal categories in the DEAP dataset, respectively, and 98.42% for the EEG Brainwave dataset. These results significantly outperform existing methods, demonstrating the model’s superior performance in terms of precision, recall, F1-score, specificity, and Mathew correlation coefficient. The integration of AlexNet and DenseNet, combined with PCA and multi-class SVM, makes this approach particularly effective for capturing the intricate patterns in EEG data, highlighting its potential for applications in human-computer interaction and mental health monitoring, marking a significant advancement over traditional methods.

Bioinspired Photoluminescent "Spider Web" as Ultrafast and Ultrasensitive Airflow-Acoustic Bimodal Sensor for Human-Computer Interaction and Intelligent Recognition.

Nature provides massive biomimetic design inspiration for constructing structural materials with desired performances. Spider webs can perceive vibrations generated by airflow and acoustic waves from prey and transfer the corresponding information to spiders. Herein, by mimicking the perception capability and structure of a spider web, an ultrafast and ultrasensitive airflow-acoustic bimodal sensor (HOF-TCPB@SF) is developed based on the postfunctionalization of hydrogen-bonded organic framework (HOF-TCPB) on silk film (SF) through hydrogen bonds. The "spider web-like" HOF-TCPB@SF possesses light weight and high elasticity, endowing this airflow sensor with superior properties including an ultralow detection limit (DL, 0.0076 m s-1), and excellent repeatability (480 cycles). As an acoustic sensor, HOF-TCPB@SF exhibits ultrahigh sensitivity (105140.77 cps Pa-1 cm-2) and ultralow DL (0.2980 dB), with the greatest response frequency of 375 Hz and the ability to identify multiple sounds. Moreover, both airflow and acoustic sensing processes show an ultrafast response speed (40 ms) and multiangle recognition response (0-180°). The perception mechanisms of airflow and acoustic stimuli are analyzed through finite element simulation. This bimodal sensor also achieves real-time airflow monitoring, speech recognition, and airflow-acoustic interoperability based on human-computer interaction, which holds great promise for applications in health care, tunnel engineering, weather forecasting, and intelligent textiles.

Exploring the mechanism of sustained consumer trust in AI chatbots after service failures: a perspective based on attribution and CASA theories

In recent years, artificial intelligence (AI) technology has been widely employed in brand customer service. However, the inherent limitations of computer-generated natural language content occasionally lead to failures in human-computer interactions, potentially damaging a company’s brand image. Therefore, it is crucial to explore how to maintain consumer trust after AI chatbots fail to provide successful service. This study constructs a model to examine the impact of social interaction cues and anthropomorphic factors on users’ sustained trust by integrating the Computers As Social Actors (CASA) theory with attribution theory. An empirical analysis of 462 survey responses reveals that CASA factors (perceived anthropomorphic characteristics, perceived empathic abilities, and perceived interaction quality) can effectively enhance user trust in AI customer service following interaction failures. This process of sustaining trust is mediated through different attributions of failure. Furthermore, AI anxiety, as a cognitive characteristic of users, not only negatively impacts sustained trust but also significantly moderates the effect of internal attributions on sustained trust. These findings expand the research domain of human-computer interaction and provide insights for the practical development of AI chatbots in communication and customer service fields.

Advanced functional connectivity analysis with integrated EEG signal enhancement and GTW-EEG-GAN for emotion detection

Emotion recognition is crucial in human-computer interaction and psychological research, utilizing modalities such as facial expressions, voice intonations, and EEG signals. This research investigates AI-driven techniques by employing Graph Neural Networks (GNNs) on functional connectivity matrices derived from EEG data to advance emotion recognition. Our approach integrates sophisticated preprocessing methods, including Integrated EEG Signal Enhancement (IESE) and novel augmentation techniques such as Gaussian Time Warping Generative Adversarial Network (GTW-EEG-GAN). This integration aims at enhancing data quality and diversity. Signal decomposition using EMD-Wavelet Hybrid Decomposition further refines feature extraction, enabling robust analysis of EEG signals. Functional connectivity metrics capture critical neural interactions intended for precise emotion characterization. The GNN architecture effectively processes these features, achieving significant accuracy improvements. Evaluations on the DEAP dataset demonstrate promising results, attaining 94.5% accuracy before hyperparameter tuning and further improving to 98.6% after tuning, highlighting the system's efficacy. The methodology showcases the integration of advanced signal processing techniques with AI, offering a comprehensive framework for future studies. The findings advocate potential applications in areas such as personalized mental health monitoring, adaptive learning environments, and responsive gaming experiences, demonstrating the broad impact and versatility of AI-driven EEG-based emotion recognition.

Promises and challenges of generative artificial intelligence for human learning.

Generative artificial intelligence (GenAI) holds the potential to transform the delivery, cultivation and evaluation of human learning. Here the authors examine the integration of GenAI as a tool for human learning, addressing its promises and challenges from a holistic viewpoint that integrates insights from learning sciences, educational technology and human-computer interaction. GenAI promises to enhance learning experiences by scaling personalized support, diversifying learning materials, enabling timely feedback and innovating assessment methods. However, it also presents critical issues such as model imperfections, ethical dilemmas and the disruption of traditional assessments. Thus, cultivating AI literacy and adaptive skills is imperative for facilitating informed engagement with GenAI technologies. Rigorous research across learning contexts is essential to evaluate GenAI's effect on human cognition, metacognition and creativity. Humanity must learn with and about GenAI, ensuring that it becomes a powerful ally in the pursuit of knowledge and innovation, rather than a crutch that undermines our intellectual abilities.

Highly conductive and sensitive alginate hydrogel strain sensors fabricated using near-field electrohydrodynamic direct-writing process

Hydrogel flexible sensors have attracted considerable attention because of their wearability, biocompatibility, and precision signal transmission capability. However, the hydrogel strain sensors fabricated by conventional printing or hand-injection methods have difficulty balancing their mechanical strength and sensing characteristics, limiting the application of hydrogel strain sensors. Herein, polyvinyl alcohol and polyacrylamide were loosely crosslinked with sodium alginate through chemical cross-linking. Subsequently, MXene nanosheets were introduced for doping, the crosslinked hydrogel conductive network was constructed, and the hydrogel strain sensors were fabricated using the electrohydrodynamic (EHD) printing method. The ions in the EHD-printed hydrogel undergo directional movement under an externally enhanced electric field, causing the formation of more uniform and dense porous conductive networks inside the hydrogel, and high electrical conductivity (0.49 S m−1) is obtained. These hydrogel strain sensors have excellent mechanical properties (tensile strength: 0.17 MPa at 787 % strain), high sensitivity (gauge factor: 1.54 at 0–100 % strain), and low detection limits (1 % strain). Furthermore, demonstrations of real-time Morse code tapping information transmission, handwriting recognition during writing, and human physiological behavior monitoring demonstrations using the fabricated sensors indicate that the EHD-printed hydrogel strain sensor method has significant potential for wearable devices and human-computer interaction applications.

The Potential of AI in Information Provision in Energy-Efficient Renovations: A Narrative Review of Literature

<p>Energy-efficient renovation (EER) is a complex process essential for reducing emissions in the built environment. This research identifies homeowners as the main decision-makers, whereas intermediaries and social interactions between peers are highly influential in home renovations. It investigates information and communication barriers encountered during the initial phases of EERs. The study reviews AI tools developed within the EERs domain to assess their capabilities in overcoming these barriers and identifies areas needing improvement. This research examines stakeholders, barriers, and the AI tools in the literature for EERs. The discussion compares the functionalities of these tools against stakeholder needs and the challenges they face. Findings show that tools often overlook methodologies in human–computer interaction and the potential of textual and visual AI methods. Digital tool development also lacks insights from social science and user feedback, potentially limiting the practical impact of these innovations. This article contributes to the EERs literature by proposing an AI-supported framework and outlining potential research areas for future exploration, particularly improving tool effectiveness and stakeholder engagement to scale up the EER practice.</p>

Multimodal human computer interaction of wheelchairs supporting lower limb active rehabilitation

Currently, an important challenge in stroke rehabilitation is how to effectively restore motor functions of lower limbs. This paper presents multimodal human computer interaction (HCI) of wheelchairs supporting lower limb active rehabilitation. First, multimodal HCI incorporating motor imagery electroencephalography (EEG), electromyography (EMG) and speech is designed. Second, prototype supporting wheelchair active rehabilitation method is illustrated in details. Third, the preliminary brain-computer interfaces (BCI) and speech recognition task experiments are carried out respectively, and the results are obtained. Finally, discussion is conducted and conclusion is drawn. This study has important practical significance in auxiliary movements and neurorehabilitation for stroke patients.