Human-computer Interaction Research Articles

Human-Computer Interaction, and Virtual Reality Applications for Memory Enhancement

Human-Computer Interaction (HCI) and Virtual Reality (VR) technologies are redefining the boundaries of cognitive enhancement, particularly in the domain of memory improvement. By immersing users in interactive virtual environments, VR offers unique opportunities to engage multiple sensory modalities and enhance memory encoding, storage, and retrieval processes. This research explores the integration of HCI principles and VR applications to develop innovative solutions for memory enhancement. Key areas of focus include the design of adaptive VR environments tailored to individual cognitive needs, the role of gamification in reinforcing memory, and the application of biofeedback for real-time user monitoring. The study also examines challenges such as user adaptability, accessibility, and the ethical considerations of using immersive technology for cognitive interventions. Through case studies and experimental findings, this research highlights the transformative potential of VR in memory training, particularly for educational purposes and memory rehabilitation. By merging HCI methodologies with VR technology, the study offers a comprehensive framework for enhancing memory, paving the way for innovative tools that benefit diverse populations.

A Systematic Review of Working Memory Applications for Children with Learning Difficulties: Transfer Outcomes and Design Principles

Working memory (WM) is a crucial cognitive function, and a deficit in this function is a critical factor in learning difficulties (LDs). As a result, there is growing interest in exploring different approaches to training WM to support students with LDs. Following the PRISMA 2020 guidelines, this systematic review aims to identify current computer-based WM training applications and their theoretical foundations, explore their effects on improving WM capacity and other cognitive/academic abilities, and extract design principles for creating an effective WM application for children with LDs. The 22 studies selected for this review provide strong evidence that children with LDs have low WM capacity and that their WM functions can be trained. The findings revealed four commercial WM training applications—COGMED, Jungle, BrainWare Safari, and N-back—that were utilized in 16 studies. However, these studies focused on suggesting different types of WM tasks and examining their effects rather than making those tasks user-friendly or providing practical guidelines for the end-user. To address this gap, the principles of the Human–Computer Interaction, with a focus on usability and user experience as well as relevant cognitive theories, and the design recommendations from the selected studies have been reviewed to extract a set of proposed guidelines. A total of 15 guidelines have been extracted that can be utilized to design WM training programs specifically for children with LDs.

All-in-one, flexible, diversified self-powered sensors through embedded 3D printing

Abstract Triboelectric nanogenerators (TENGs) are characterized by zero power consumption and are often employed as self-powered sensors. However, the complex manufacturing process and expensive equipment limit the further promotion and application of self-powered sensors, which have become urgent challenges in this field. Here, a simple strategy using embedded 3D printing is proposed to enable the fabrication of diverse self-powered sensors in one-step, reducing production costs while increasing design flexibility. Specifically, the designed sensors composed of the silicone as the triboelectric layer and silicone/multi-walled carbon nanotubes as flexible electrodes with excellent all-in-one structures. Meanwhile, diversified self-powered sensors with different complex structures (e.g., planar array sensors and helical icosahedron sensors) were developed to meet the diverse needs of different applications, verifying the capability of the proposed embedded 3D printing method to design and customize sensors with various shapes and structures. In addition, the applications of these functionalized self-powered sensors in cryptographic simulation, pressure position detection, and impact force recognition have been successfully demonstrated. Therefore, this self-powered sensor based on embedded 3D printing has a promising future in human-computer interaction, collision detection and other fields.

Harmonizing human-computer interaction: Exploring evolution and integration in media and computing

Abstract. This paper delves into the intricate relationship between Human-Computer Interaction (HCI), media, and computing, examining its historical evolution and contemporary challenges. From the rudimentary interfaces of the 1970s to the immersive digital experiences of the present day, HCI has undergone a profound transformation, driven by advancements in technology and changing user expectations. The convergence of media and computing technologies has blurred traditional boundaries, reshaping user interactions and opening new frontiers for research and innovation. Through quantitative analysis and mathematical modeling, researchers gain insights into user behavior, preferences, and interactions, informing the design of more intuitive and engaging interactive systems. However, this integration also poses challenges related to cross-platform compatibility, ethical considerations, and accessibility. By embracing user-centered design principles, ethical stewardship, and technological innovation, HCI researchers and practitioners can navigate these challenges and shape the future of HCI-Media-Computing integration.

Speech Emotion Recognition Using Feedforward Neural Network

Speech Emotion Recognition (SER) has gained increasing attention due to its application in fields such as human-computer interaction, healthcare, customer service automation, and affective computing. This review focuses on the application of Deep Neural Networks (DNNs), specifically Feedforward Neural Networks (FNNs), in detecting and classifying emotions from speech signals. Recent advancements in deep learning have significantly enhanced the ability of machines to interpret emotional cues from auditory data. This paper discusses the motivation, objectives, and scope of employing DNN FNN architectures for SER. We also address the technical feasibility and potential of this approach for real-time applications. By reviewing existing literature, this study aims to provide insights into the current progress, challenges, and future prospects of SER systems.

Human-Computer Interaction: A Literature Review of Artificial Intelligence and Communication in Healthcare

Human-Computer Interaction: A Literature Review of Artificial Intelligence and Communication in Healthcare

Impact of Virtual Reality on Brain–Computer Interface Performance in IoT Control—Review of Current State of Knowledge

This article examines state-of-the-art research into the impact of virtual reality (VR) on brain–computer interface (BCI) performance: how the use of virtual reality can affect brain activity and neural plasticity in ways that can improve the performance of brain–computer interfaces in IoT control, e.g., for smart home purposes. Integrating BCI with VR improves the performance of brain–computer interfaces in IoT control by providing immersive, adaptive training environments that increase signal accuracy and user control. VR offers real-time feedback and simulations that help users refine their interactions with smart home systems, making the interface more intuitive and responsive. This combination ultimately leads to greater independence, efficiency, and ease of use, especially for users with mobility issues, in managing IoT-connected devices. The integration of BCI and VR shows great potential for transformative applications ranging from neurorehabilitation and human–computer interaction to cognitive assessment and personalized therapeutic interventions for a variety of neurological and cognitive disorders. The literature review highlights the significant advances and multifaceted challenges in this rapidly evolving field. Particularly noteworthy is the emphasis on the importance of adaptive signal processing techniques, which are key to enhancing the overall control and immersion experienced by individuals in virtual environments. The value of multimodal integration, in which BCI technology is combined with complementary biosensors such as gaze tracking and motion capture, is also highlighted. The incorporation of advanced artificial intelligence (AI) techniques will revolutionize the way we approach the diagnosis and treatment of neurodegenerative conditions.

Research on an Eye Control Method Based on the Fusion of Facial Expression and Gaze Intention Recognition

With the deep integration of psychology and artificial intelligence technology and other related technologies, eye control technology has achieved certain results at the practical application level. However, it is found that the accuracy of the current single-modal eye control technology is still not high, which is mainly caused by the inaccurate eye movement detection caused by the high randomness of eye movements in the process of human–computer interaction. Therefore, this study will propose an intent recognition method that fuses facial expressions and eye movement information and expects to complete an eye control method based on the fusion of facial expression and eye movement information based on the multimodal intent recognition dataset, including facial expressions and eye movement information constructed in this study. Based on the self-attention fusion strategy, the fused features are calculated, and the multi-layer perceptron is used to classify the fused features, so as to realize the mutual attention between different features, and improve the accuracy of intention recognition by enhancing the weight of effective features in a targeted manner. In order to solve the problem of inaccurate eye movement detection, an improved YOLOv5 model was proposed, and the accuracy of the model detection was improved by adding two strategies: a small target layer and a CA attention mechanism. At the same time, the corresponding eye movement behavior discrimination algorithm was combined for each eye movement action to realize the output of eye behavior instructions. Finally, the experimental verification of the eye–computer interaction scheme combining the intention recognition model and the eye movement detection model showed that the accuracy of the eye-controlled manipulator to perform various tasks could reach more than 95 percent based on this scheme.

Human-computer interactions with farm animals—enhancing welfare through precision livestock farming and artificial intelligence

While user-centered design approaches stemming from the human-computer interaction (HCI) field have notably improved the welfare of companion, service, and zoo animals, their application in farm animal settings remains limited. This shortfall has catalyzed the emergence of animal-computer interaction (ACI), a discipline extending technology’s reach to a multispecies user base involving both animals and humans. Despite significant strides in other sectors, the adaptation of HCI and ACI (collectively HACI) to farm animal welfare—particularly for dairy cows, swine, and poultry—lags behind. Our paper explores the potential of HACI within precision livestock farming (PLF) and artificial intelligence (AI) to enhance individual animal welfare and address the unique challenges within these settings. It underscores the necessity of transitioning from productivity-focused to animal-centered farming methods, advocating for a paradigm shift that emphasizes welfare as integral to sustainable farming practices. Emphasizing the ‘One Welfare’ approach, this discussion highlights how integrating animal-centered technologies not only benefits farm animal health, productivity, and overall well-being but also aligns with broader societal, environmental, and economic benefits, considering the pressures farmers face. This perspective is based on insights from a one-day workshop held on June 24, 2024, which focused on advancing HACI technologies for farm animal welfare.

Mapping Computer Vision Syndrome: An Engineering Problem in Human–Computer Interaction

Mapping Computer Vision Syndrome: An Engineering Problem in Human–Computer Interaction

Three-dimensional numerical calculation and case application of IP five-directional well logging

Abstract Conventional well logging can effectively detect the vertical distribution range of ore bodies near the well by placing detectors in the well. However, its detection capability for the horizontal extension of ore bodies is limited. Therefore, starting from the differential equations satisfied by the point source in three-dimensional structures, this paper deduced the variational problem satisfied by the total potential under three-dimensional complex conditions. By focusing on well logging, tetrahedral cross-grid partitioning technology was employed to achieve high-precision three-dimensional finite element forward modeling. Simulations were carried out for different forms of ore bodies using the induced polarization five-direction observation method, considering the presence of ore bodies near the well and the case where the measurement well passes through the ore body. The paper analyzed and summarized the induced polarization anomaly patterns formed by ore bodies with low resistance and high polarization in different horizontal extension directions under induced polarization five-direction logging with power supply measurements from different directions. Based on this, a human-computer interaction system was constructed to perform forward fitting and inversion interpretation of the measured induced polarization data, resulting in a good application effect.

Human-computer interaction interface design of flight simulator based on situation awareness.

To meet the comprehensive requirements of drivers for Human-Computer Interaction (HCI) during operation and to enhance situation awareness (SA), this paper proposes a design and evaluation method for flight simulator HCI interfaces grounded in SA. Firstly, the widely adopted Endsley's SA model, which comprehensively covers processes relevant to individual drivers was selected as the foundational framework. The study concentrates on flight, cruise, and landing scenarios within the simulator, analyzing and restructuring the information architecture of the HCI interface. Secondly, a self-assessment scale was developed based on the three-level model of information processing for SA to measure drivers' SA cognitive levels. The core scenarios of the simulation training were analyzed to establish a design-making process. Thirdly, invite participants were invited to engage in simulated driving sessions on the flight simulator, where their SA cognitive levels were assessed. An analysis of detail features was performed on samples with high SA cognitive levels. Subsequently, drivers' requirements for the control layout features of the HCI interface design were gathered through surveys. The design detail features of the HCI interface associated with high SA cognitive levels were identified and synthesized into four distinct dimensions to guide the detailed design process. Finally, participants engaged in simulated driving using the finalized interface design, and their SA cognitive levels were reassessed. The results demonstrated that the HCI interface design derived from this method significantly improved SA cognitive levels compared to the original design.

Efficient Data Processing Pipeline for Event-Based Vision Datasets: Techniques and Insights

Abstract Event-based vision datasets have emerged as a critical asset in advancing the capabilities of real-time perception systems, particularly in fields such as robotics, autonomous vehicles, and human-computer interaction. These datasets enable low-latency processing by capturing asynchronous, pixel-level changes in the scene, providing a distinct advantage over traditional frame-based systems. However, the diverse formats and characteristics of event-based datasets pose significant challenges for efficient processing and analysis, hindering their broader adoption and integration. In this paper, we present a versatile and comprehensive data processing pipeline designed to address these challenges by supporting multiple event-data formats, including newer formats such as EVT2 and EVT3. This pipeline not only converts data into widely supported formats like AEDAT and NPZ, but also ensures that the unique characteristics of event-based data—such as temporal precision and sparse event representation—are preserved throughout the conversion process. By applying this pipeline to several open-source datasets, we establish a standardized, efficient methodology for dataset manipulation that enhances compatibility and reproducibility in event-based vision research. Additionally, we introduce a novel high-resolution event-based action dataset, converted into various formats using our pipeline, which opens new avenues for exploring event-based techniques in action recognition. This dataset and our pipeline serve as valuable resources for the research community, enabling advancements in real-time vision applications and fostering greater collaboration and standardization across studies.

Human errors analysis for remotely controlled ships during collision avoidance

To address human errors in collision avoidance tasks of remotely controlled ships, this study aims to develop a comprehensive framework for human error analysis within the context of autonomous ships. Firstly, the Hierarchical Task Analysis method is utilized to identify crew collision avoidance tasks associated with the traditional ship, and these tasks are then dissected into different operational stages using the Information Decision Action in a Crew cognitive model. Secondly, a combination of the fault hypothesis method and expert opinions are used to identify potential human error that may occur during collision avoidance operations of remotely controlled ships. Thirdly, an integrated approach is proposed to build a quantitative risk assessment model, which combines Failure Mode and Effects Analysis, Evidential Reasoning, and Belief rules-based Bayesian Network. Then, axiomatic analysis is used to verify the robustness and applicability of the risk assessment model. Finally, based on the results of quantitative risk assessment, specific measures are proposed for enhancing the safety of collision avoidance process of remotely controlled ships. The findings show that uncoordinated interactions of human-computer systems during the decision-making stage are a pivotal factor in the collision avoidance process. Therefore, future design efforts for remote-control centre should prioritize improving the clarity of human-computer interaction interfaces.

Enhancing Human-Computer Interaction in Augmented Reality (AR) and Virtual Reality (VR) Environments: The Role of Adaptive Interfaces and Haptic Feedback Systems

Human-Computer Interaction (HCI) in Augmented Reality (AR) and Virtual Reality (VR) environments has emerged as a critical area of research, shaping how users interact with immersive technologies. This study explores the design and development of advanced HCI frameworks in AR/VR environments, with a particular focus on adaptive interfaces and haptic feedback systems. Adaptive interfaces employ real-time data to personalize user experiences, while haptic feedback systems provide tactile sensations to bridge the gap between digital and physical interactions. The integration of these technologies enhances user engagement, immersion, and efficiency across applications such as gaming, training simulations, healthcare, and remote collaboration. This research investigates the challenges in creating intuitive, responsive systems and examines cutting-edge solutions in multimodal interactions, gesture recognition, and real-time feedback processing. By analyzing user performance and satisfaction, this study contributes to the refinement of AR/VR systems, paving the way for next-generation immersive environments.

An analysis of dialogue repair in virtual assistants

Conversational user interfaces have transformed human-computer interaction by providing nearly real-time responses to queries. However, misunderstandings between the user and system persist. This study explores the significance of interactional language in dialogue repair between virtual assistants and users by analyzing interactions with Google Assistant and Siri in both English and Spanish, focusing on the assistants’ utilization and response to the colloquial other-initiated repair strategy “huh?”, which is prevalent as a human-human dialogue repair strategy. Findings revealed ten distinct assistant-generated repair strategies, but an inability to replicate human-like strategies such as “huh?”. Despite slight variations in user acceptability judgments among the two surveyed languages, results indicated an overall hierarchy of preference towards specific dialogue repair strategies, with a notable disparity between the most preferred strategies and those frequently used by the assistants. These findings highlight discrepancies in how interactional language is utilized in human-computer interaction, underscoring the need for further research on the impact of interactional elements among different languages to advance the development of conversational user interfaces across domains, including within human-robot interaction.

Elevating Facial Expression Detection: Empowered by VGG-19 and Weight-Normalized Gradient Boost Algorithm

Facial expression recognition termed as FER, is one of the vital tasks which is able in mimicking the human ability and wellness. Face expression and other gestures delivered via face are some of the important aspects of conveying non-verbal communications which plays a vital role in interpersonal relations. This domain has a higher scope of research due to enhancing computer vision and human-computer interaction. With respect and consideration to these domain interest, DL has gained a reasonable approach in performing FER. Some of the state-of-art approaches encompassing the ML approaches have faced several cases of laybacks such as overfitting, computational complexity, less adaptable to higher and big datasets. Thus, considering these laybacks and in achieving accurate FER in aspects of bringing proximal levels of FER, the current approach deployed DL methods for both feature extraction from the input image and in classifying them. Deep multi-level feature extraction is performed using the VGG-19 model and Weight Normalized gradient boosting algorithm is adapted for classifying these expressions using the FER13 dataset. This dataset constitutes the input images, which are in ranges of 28,709 sample image data and about 3,589 image data for test. These input images are initially pre-processed for obtaining better accuracy rates when performing feature extraction and classifications. The complete model in effective FER is evaluated using the performance metrics comprising Accuracy (98%), Recall (99%), F1-score (98%) and the precision rates (97%). This analysis of the performance will aid in affirming the overall efficacy of the proposed system.

Vehicle Trajectory Prediction Using Residual Diffusion Model Based on Image Information

In automatic driving, accurate prediction of vehicle trajectory is the key to achieve automatic driving, and multi-vehicle joint trajectory prediction has become an important part of modern human-computer interaction systems such as automatic driving. In order to better predict vehicle trajectories, we propose a new residual diffusion model to infer the joint distribution of future multi-vehicle trajectories. This approach has several major advantages. First, the model is able to learn multiple probability distributions from trajectory data to obtain potential outcomes for vehicles to multiple future trajectories. Secondly, in order to integrate the motion characteristics of multiple vehicles in the same scene, we use the method of combining the reference denoising and multiple residual denoising to improve the model performance and prediction speed. Finally, on this basis, a general trajectory constraint function is introduced, so that the generated trajectories of multiple vehicles will not collide with each other. We perform a rich experimental comparison of various existing methods on the NGSIM dataset and demonstrate that the proposed algorithm achieves a 26% improvement on mAP.

HCI Teaching and Learning Model Using Studio-Based Learning

This research explores Studio-based Learning (SBL) within Human-Computer Interaction (HCI) subjects, focusing on the crucial elements required in the design of maker spaces to enhance user experience (UX) design education. As HCI becomes increasingly complex and interdisciplinary, traditional classroom settings often fail to provide the required experiential learning environment for certain subjects. Our study examines various subjects in the discipline, highlighting diverse interactivity requirements and the potential of Studio-based Learning to effectively address students' and staff’s needs. We analyse the impact of studio settings on the learning outcomes and expectations of students, especially in fostering creativity, collaboration, and hands-on engagement. Our research findings indicate that studio-based classrooms significantly improve student learning experiences in interactive and practical subjects, although effective planning around space and time constraints can impede its full implementation.

Exploring the Role of AI in UX Research

Presently, artificial intelligence (AI) is primarily utilized in user experience (UX) research for handling routine tasks like text conversion (e.g., transcription, survey preparation) to manage large volumes of data efficiently. However, its potential as a collaborative partner in generating ideas and insights remains largely untapped, possibly due to the limited AI education among UX researchers. There is a recognized need to enhance AI literacy among Human-Computer Interaction (HCI) students to effectively utilize AI tools in UX research for an AI-driven future. Despite its growing importance, the integration of AI literacy into HCI curricula remains insufficient. This study aims to explore AI’s potential in addressing current challenges in UX research and assess UX researchers’ readiness to adopt AI tools. Through in-depth interviews with five early-career UX researchers, it examined their experiences, challenges, and perspectives on integrating AI. Thematic analysis of the interview data uncovers insights, revealing that AI has significant potential to overcome challenges in collecting, interpreting, and synthesizing qualitative data, as well as in translating user needs into design concepts. The findings are intended to inform strategies for incorporating AI literacy into HCI education, aligning with industry demands for AI-enabled expertise in UX design.