Human-computer Interaction Research Research Articles

Harmonizing human-computer interaction: Exploring evolution and integration in media and computing

Abstract. This paper delves into the intricate relationship between Human-Computer Interaction (HCI), media, and computing, examining its historical evolution and contemporary challenges. From the rudimentary interfaces of the 1970s to the immersive digital experiences of the present day, HCI has undergone a profound transformation, driven by advancements in technology and changing user expectations. The convergence of media and computing technologies has blurred traditional boundaries, reshaping user interactions and opening new frontiers for research and innovation. Through quantitative analysis and mathematical modeling, researchers gain insights into user behavior, preferences, and interactions, informing the design of more intuitive and engaging interactive systems. However, this integration also poses challenges related to cross-platform compatibility, ethical considerations, and accessibility. By embracing user-centered design principles, ethical stewardship, and technological innovation, HCI researchers and practitioners can navigate these challenges and shape the future of HCI-Media-Computing integration.

Advanced functional connectivity analysis with integrated EEG signal enhancement and GTW-EEG-GAN for emotion detection

Emotion recognition is crucial in human-computer interaction and psychological research, utilizing modalities such as facial expressions, voice intonations, and EEG signals. This research investigates AI-driven techniques by employing Graph Neural Networks (GNNs) on functional connectivity matrices derived from EEG data to advance emotion recognition. Our approach integrates sophisticated preprocessing methods, including Integrated EEG Signal Enhancement (IESE) and novel augmentation techniques such as Gaussian Time Warping Generative Adversarial Network (GTW-EEG-GAN). This integration aims at enhancing data quality and diversity. Signal decomposition using EMD-Wavelet Hybrid Decomposition further refines feature extraction, enabling robust analysis of EEG signals. Functional connectivity metrics capture critical neural interactions intended for precise emotion characterization. The GNN architecture effectively processes these features, achieving significant accuracy improvements. Evaluations on the DEAP dataset demonstrate promising results, attaining 94.5% accuracy before hyperparameter tuning and further improving to 98.6% after tuning, highlighting the system's efficacy. The methodology showcases the integration of advanced signal processing techniques with AI, offering a comprehensive framework for future studies. The findings advocate potential applications in areas such as personalized mental health monitoring, adaptive learning environments, and responsive gaming experiences, demonstrating the broad impact and versatility of AI-driven EEG-based emotion recognition.

Insights from the evaluation of a persuasive intervention for absent-minded smartphone use

ABSTRACT Problematic smartphone use (PSU) is a growing concern that hurts users' daily lives and activities. Previous studies have emphasised the importance of developing mindfulness and self-efficacy concerning smartphone use rather than solely focussing on reducing usage. However, there has been little research developing and evaluating digital interventions specifically targeting absent-minded smartphone use. There is also little knowledge on how best mindfulness practice using digital technologies can be integrated with persuasive designs which have been widely studied and implemented for behaviour change. In this paper, we developed a live wallpaper application for Android lock and home screens as a mindfulness-based intervention for absent-minded smartphone use. The application was evaluated over two weeks with 121 participants. This was followed by a semi-structured interview with 15 participants. The results of our analysis show that the intervention reduced absent-minded smartphone use overall. While participants found the various features (especially the customisation features) of the intervention to be persuasive, we found no correlation between perceived persuasiveness and behaviour change. This work provides valuable insights for advancing human-computer interaction (HCI) research on PSU. Additionally, the results raise important questions for future research, such as the relationship between perceived persuasiveness and behaviour change. Finally, we contribute to the discussion on the applications of mindfulness in persuasive technology, challenges and future research areas.

HealthCheck: A method for evaluating persuasive mobile health applications.

Objectives: This paper introduces HealthCheck, a novel evaluation method for persuasive mobile health applications, aiming to fill the critical gap in quick and effective evaluation tools for this domain. Methods: Following Design Science Research, HealthCheck was developed through problem identification, solution design, implementation, evaluation, and iterative refinement. The implementation involved testing with seven experts to assess its applicability and effectiveness. Results: Feedback from the evaluators indicated that while a few heuristics in HealthCheck were considered irrelevant by some, the majority found the heuristics to be both pertinent and beneficial, especially within the caregiving context. This feedback highlights the practical value of HealthCheck and its potential to offer meaningful insights into improving the usability of persuasive eHealth applications. Conclusion: The study shows HealthCheck effectively evaluates persuasive mobile health applications, offering actionable insights to enhance usability. This validates the relevance and robustness of HealthCheck's heuristics, advancing information systems and human-computer interaction research.

A Text-based Intelligently driven Emotion Recognition Framework

Introduction: Emotion recognition from text has gained considerable interest due to its applications in human-computer interaction, emotion analysis, and psychiatric research. Traditional methods have struggled with emotional ambiguity, cultural nuances, and the dynamic nature of language, which affect the reliability of emotion recognition. This paper presents a novel emotion recognition framework named Artificial Gorilla Troops driven Modified Gaussian Support Vector Machine (AGT-MGSVM).Methods: We gathered a publicly available ISEAR dataset containing various textual emotional expressions and applied natural language processing (NLP) techniques for text pre-processing. The suggested AGT-MGSVM approach combines the resilience of the Gaussian assist vector gadget (GSVM) with the ability of AGT, a bio-stimulated optimization method. AGT improves the MGSVM with the aid of dynamically regulating its parameters based on the evolutionary conduct of gorilla troops, optimizing the version to enhance emotion popularity. Results: We examine the performance of the proposed technique against traditional emotion reputation methods using standard metrics inclusive of recall (89.2%), precision (89.5%), F1-score (89.4%), and accuracy (89.9%). Conclusion: The counseled AGT-MGSVM method is a promising improvement in intelligence-driven emotion reputation from the text.

Mimetic Possibilities: Collaboration through movement in multimedia opera

Abstract In the opera Act I: Invocation – The Ghost Trilogy, artists from music composition, Greek tragedy, creative coding, and opera performance collaborated to create an artwork that engages with philosophy and technology. A light sculpture with a novel Internet of Things (IoT) architecture was adapted for the opera’s stage, where both the sculpture and its controlling interface were programmed to visually represent musical processes within the composer’s approach. While filming, the light sculpture contributed to the dramatic context through movement and gestural interaction between the sculpture and the soprano. Drawing upon Human-Computer Interaction (HCI) research and its inquiry into movement for engaging with new technology expressively, movement and gesture, transpiring from mimetic techniques in the opera, were central in this interdisciplinary collaboration.

An Umbrella Review of Reporting Quality in CHI Systematic Reviews: Guiding Questions and Best Practices for HCI

Systematic reviews (SRs) are vital to gathering and structuring knowledge, yet descriptions of their procedures are often inadequate. In human-computer interaction (HCI), SRs are still uncommon but gaining momentum, which prompted us to explore how SRs are reported at CHI—the flagship HCI conference venue. To assess the reporting quality of CHI reviews that aim for a systematic approach, we conducted an umbrella review and applied reporting guidelines for SRs (PRISMA and ENTREQ) to our corpus. We contribute the first exploration of how well SRs at CHI meet guidelines for reporting quality, showcasing strategies for improvement in reporting and conducting SRs especially in the domains of appraisal, synthesis, and documentation (i.e., protocol development). Finally, we present guiding questions for HCI researchers and practitioners for reporting SRs, as well as suggestions for best practices.

DEEP LEARNING BASED EMOTION DETECTION SYSTEM

Emotion detection plays a crucial role in human-computer interaction, affective computing, and psychological research. Traditional methods of emotion detection often rely on manual annotations or subjective assessments, which can be time- consuming and prone to biases. In this paper, we propose a Deep Learning-Based Emotion Detection System that leverages advanced neural network architectures to automatically recognize emotions from facial expressions in real-time. The system utilizes convolutional neural networks (CNNs) for feature extraction from facial images and recurrent neural networks (RNNs) for sequence modeling of temporal dynamics in facial expressions. Our proposed system addresses key challenges in emotion detection, including variations in facial expressions, lighting conditions, and occlusions. By training on large-scale datasets of labeled facial images, the system learns to accurately classify emotions into predefined categories such as happiness, sadness, anger, fear, disgust, and surprise. We evaluate the performance of the system using standard metrics such as accuracy, precision, recall, and F1-score, demonstrating its effectiveness in real- world scenarios. The results indicate that the Deep Learning-Based Emotion Detection System achieves state-of-the-art performance in recognizing emotions from facial expressions, outperforming traditional methods. The system's ability to analyze facial dynamics in real-time opens up opportunities for applications in human-computer interaction, virtual reality, mental health monitoring, and personalized user experiences. Keywords: Deep Learning, Emotion Detection, Facial Expression Recognition, Convolutional Neural Networks, Recurrent Neural Networks, Affective Computing. Keywords: emotion Detection, Machine Learning, Classifier, Natural Language Processing

Assessing Feature Importance in Eye-Tracking Data within Virtual Reality Using Explainable Artificial Intelligence Techniques

Our research systematically investigates the cognitive and emotional processes revealed through eye movements within the context of virtual reality (VR) environments. We assess the utility of eye-tracking data for predicting emotional states in VR, employing explainable artificial intelligence (XAI) to advance the interpretability and transparency of our findings. Utilizing the VR Eyes: Emotions dataset (VREED) alongside an extra trees classifier enhanced by SHapley Additive ExPlanations (SHAP) and local interpretable model agnostic explanations (LIME), we rigorously evaluate the importance of various eye-tracking metrics. Our results identify significant correlations between metrics such as saccades, micro-saccades, blinks, and fixations and specific emotional states. The application of SHAP and LIME elucidates these relationships, providing deeper insights into the emotional responses triggered by VR. These findings suggest that variations in eye feature patterns serve as indicators of heightened emotional arousal. Not only do these insights advance our understanding of affective computing within VR, but they also highlight the potential for developing more responsive VR systems capable of adapting to user emotions in real-time. This research contributes significantly to the fields of human-computer interaction and psychological research, showcasing how XAI can bridge the gap between complex machine-learning models and practical applications, thereby facilitating the creation of reliable, user-sensitive VR experiences. Future research may explore the integration of multiple physiological signals to enhance emotion detection and interactive dynamics in VR.

Research on Human Computer Interaction Technology in Virtual Reality Scenes

This paper discusses the current status, challenges and development trends of human computer interaction technology in virtual reality scenarios. Virtual reality technology has gradually become a frontier field of human computer interaction research. This paper analyzes virtual reality technology and human computer interaction technology. By introducing the commonly used human computer interaction hardware equipment and software technology in current virtual reality scenarios, it reveals the actual application of human computer interaction in virtual reality environments. From the three aspects of technology, user experience, and security and privacy protection, this paper analyzes the challenges faced by human computer interaction technology in virtual reality scenarios. In the future, with the continuous advancement of technological innovation, new human-computer interaction technologies will continue to emerge, and the integration of artificial intelligence technology will further enhance the intelligence level of human-computer interaction, making the user interaction experience more natural and convenient.

Uncovering the theoretical basis of user types: An empirical analysis and critical discussion of user typologies in research on tailored gameful design

Gamification has become one of the main areas in information systems and human–computer interaction research related to users’ motivations and behaviors. Within this context, a significant research gap is the lack of understanding of how users’ characteristics, especially in terms of their preferences for gameful interaction (i.e., user typologies), moderate the effects of gamification and, furthermore, how gamification could be tailored to individual needs. Despite their prominence in classifying users, current typologies and their use in research and practice have received severe criticism regarding validity and reliability, as well as the application and interpretation of their results. Therefore, it is essential to reconsider the relationships and foundations of common user typologies and establish a sound empirical basis to critically discuss their value and limits for personalized gamification. To address this research gap, this study investigated the psychometric properties of the most popular player types within tailored gamification literature (i.e., Bartle’s player types, Yee’s motivations to play, BrainHex, and HEXAD) through a survey study (n=877) using their respective measurement instruments, followed by a correlation analysis to understand their empirical relations and an exploratory factor analysis to identify the underlying factors. The results confirm that user typologies, despite their different origins, show considerable overlap, some being consistent whereas others contradicted theoretically assumed relationships. Furthermore, we show that these four user typologies overall factor into five underlying and fundamental dimensions of Socialization, Escapism, Achievement, Reward Pursuit, and Independence, which could be considered common concepts that may essentially reflect key determinants of user motivation in gamification. Our findings imply that future research and practice in tailored gamification design should shift the focus from developing and applying ever more nuanced typologies to understanding and measuring the key underlying determinants of user motivation in gameful systems. Moreover, given the considerable interrelationships between these determinants, we also argue that researchers should favor continuous representations of users’ motivations in specific situations instead of a dichotomous operationalization of user types as static manifestations of their preferences.

What I Don’t Like about You?: A Systematic Review of Impeding Aspects for the Usage of Conversational Agents

Abstract The application and use cases for conversational agents (CAs) are versatile. Smart speakers such as Alexa and Google Home are used in smart home environments, digital agents are integrated into car systems and chatbots are increasingly used in customer service processes. However, human–computer interaction researchers identify and investigate a wide-ranging variety of aspects impeding the usage of CAs by end-users. In general, impediments differ depending on use case contexts, user group characteristics and the CA’s technological infrastructure. Hence, it is difficult and often ambiguous for designers and developers to generate an appropriate awareness about aspects impeding CA usage. We address this problem, by conducting a systematic review of 65 publications surveying impeding aspects of the usage of CAs.

EEG Emotion Recognition Network Based on Attention and Spatiotemporal Convolution.

Human emotions are complex psychological and physiological responses to external stimuli. Correctly identifying and providing feedback on emotions is an important goal in human-computer interaction research. Compared to facial expressions, speech, or other physiological signals, using electroencephalogram (EEG) signals for the task of emotion recognition has advantages in terms of authenticity, objectivity, and high reliability; thus, it is attracting increasing attention from researchers. However, the current methods have significant room for improvement in terms of the combination of information exchange between different brain regions and time-frequency feature extraction. Therefore, this paper proposes an EEG emotion recognition network, namely, self-organized graph pesudo-3D convolution (SOGPCN), based on attention and spatiotemporal convolution. Unlike previous methods that directly construct graph structures for brain channels, the proposed SOGPCN method considers that the spatial relationships between electrodes in each frequency band differ. First, a self-organizing map is constructed for each channel in each frequency band to obtain the 10 most relevant channels to the current channel, and graph convolution is employed to capture the spatial relationships between all channels in the self-organizing map constructed for each channel in each frequency band. Then, pseudo-three-dimensional convolution combined with partial dot product attention is implemented to extract the temporal features of the EEG sequence. Finally, LSTM is employed to learn the contextual information between adjacent time-series data. Subject-dependent and subject-independent experiments are conducted on the SEED dataset to evaluate the performance of the proposed SOGPCN method, which achieves recognition accuracies of 95.26% and 94.22%, respectively, indicating that the proposed method outperforms several baseline methods.

An Approach Towards to Real Time AI Desktop Voice Assistant

The advent of artificial intelligence (AI) has revolutionized human-computer interaction, making it more natural and intuitive. This research paper presents the development and implementation of an AI Desktop Voice Assistant designed to enhance productivity and accessibility for users. The voice assistant leverages advanced speech recognition, natural language processing (NLP), and machine learning techniques to understand and execute user commands. Key functionalities include voice-activated application control, web searches, personalized reminders, and real-time information retrieval. Our system integrates with widely-used APIs and services, providing a seamless user experience across various tasks. The paper discusses the architecture of the voice assistant, detailing the integration of components such as the speech recognition engine, NLP models, and the dialogue management system. We explore the challenges encountered during development, including accurate speech recognition in noisy environments and handling ambiguous user commands. Solutions implemented to address these challenges are also presented. Furthermore, we conduct a usability study to evaluate the effectiveness and user satisfaction of the voice assistant. The results indicate a high level of user engagement and satisfaction, demonstrating the practical benefits of incorporating AI-driven voice interfaces into desktop environments. Our findings contribute to the ongoing research in human-computer interaction, suggesting pathways for future enhancements in AI voice assistants..

EmoSonics: Emotion Detection via Voice and Speech Recognition

Understanding emotions from speech is like deciphering a rich tapestry of human expression in the realm of human-computer interaction. It's akin to listening to someone's tone and inflection to discern whether they're happy, surprised, or experiencing a range of other feelings. Researchers use a variety of techniques, from analyzing speech patterns to utilizing advanced technologies like fMRI, to decode these emotional cues. Emotions aren't just simple labels; they're complex and nuanced, demanding sophisticated methods for accurate interpretation. Some methods break emotions down into simple categories, while others embrace the intricacies of human emotion, treating them as continuous variables. Ultimately, the goal is to enhance communication between humans and computers by enabling machines to understand and respond appropriately to our emotional states. This pursuit underscores the significance of emotion detection in speech analysis and highlights the need for continually evolving methodologies in human-computer interaction research.

Multimedia Human-Computer Interaction Method in Video Animation Based on Artificial Intelligence Technology

With the development of computer technology innovation, be able to deal with the media comprehensive information and real-time information interaction with the computer multimedia technology arises at the historic moment, it promotes the application fields of computer widen to industrial all aspects of life. As the product of digital technology, animation technology plays an irreplaceable role in the production of multimedia courseware. However, the existing human-computer interaction methods have shortcomings such as incomplete extraction of video features and poor human-computer interaction effect. In this context, this paper designs a multimedia human-computer interaction method for animation works based on CNN model. First of all, the original video data is collected and preprocessed. Then it is input into the HCI framework based on CNN model for feature extraction. Finally, the effectiveness and practicability of the proposed method are proved by simulation experiments, which provides a reference and basis for the research of modern human-computer interaction.

Broadening the mind: how emerging neurotechnology is reshaping HCI and interactive system design

Abstract People are increasingly eager to know more about themselves through technology. To date, technology has primarily provided information on our physiology. Yet, with advances in wearable technology and artificial intelligence, the current advent of consumer neurotechnology will enable users to measure their cognitive activity. We see an opportunity for research in Human-Computer Interaction (HCI) in the development of these devices. Neurotechnology offers new insights into user experiences and facilitates the development of novel methods in HCI. Researchers will be able to create innovative interactive systems based on the ability to measure cognitive activity at scale in real-world settings. In this paper, we contribute a vision of how neurotechnology will transform HCI research and practice. We discuss how neurotechnology prompts a discussion about ethics, privacy, and trust. This trend highlights HCI’s crucial role in ensuring that neurotechnology is developed and utilised in ways that truly benefit people.

Pilot study of the relation between various dynamics of avatar experience and perceptual characteristics

In recent years, due to the prevalence of virtual reality (VR) and human-computer interaction (HCI) research, along with the expectation that understanding the process of establishing sense of ownership, sense of agency, and limb heaviness (in this study, limb heaviness is replaced with comfort level) will contribute to the development of various medical rehabilitation, various studies have been actively conducted in these fields. Previous studies have indicated that each perceptual characteristics decrease in response to positive delay. However, it is still unclear how each perceptual characteristic changes in response to negative delay. Therefore, the purpose of this study was to deduce how changes occur in the perceptual characteristics when certain settings are manipulated using the avatar developed in this study. This study conducted experiments using an avatar system developed for this research that uses electromyography as the interface. Two separate experiments involved twelve participants: a preliminary experiment and a main experiment. As observed in the previous study, it was confirmed that each perceptual characteristics decreased for positive delay. In addition, the range of the preliminary experiment was insufficient for the purpose of this study, which was to confirm the perceptual characteristics for negative delay, thus confirming the validity of conducting this experiment. Meanwhile, the main experiment showed that the sense of ownership, sense of agency, and comfort level decreased gradually as delay time decreased, (i.e., this event is prior to action with intention, which could not be examined in the previous study). This suggests that control by the brain-machine interface is difficult to use when it is too fast. In addition, the distribution of the most strongly perceived settings in human perceptual characteristics was wider in regions with larger delays, suggesting this may lead to the evaluation of an internal model believed to exist in the human cerebellum. The avatar developed for this study may have the potential to create a new experimental paradigm for perceptual characteristics.

User accounts: How technological concepts permeate public law through the EU's AI Act

This article argues that through the EU's technology regulation, technological concepts permeate legal language. Such concepts may function as transplants, even irritants, causing tensions and uncertainties. As technology regulation is increasingly horizontal, i.e. obligating private and public actors alike, these newfound legal concepts remain disconnected from established public law vocabulary and the power constellations it represents and embeds. We approach this evolution of legal language from public law perspective and concentrate on the concepts of ‘user’ and ‘deployer’ in the EU's upcoming Artificial Intelligence Act. We discuss these emerging legal concepts in relation to the rich theorizing on the concepts in human–computer interaction research. Our analysis demonstrates a discrepancy between legal and technology-oriented conceptualizations of the ‘user-deployer’. We draw three conclusions. First, the digital revolution is taking place in conceptual-linguistic practices of law, and not only when translating law into code. Second, when external concepts are appropriated into law, they are uprooted from their established habitat, which may result in unpredictability in future legal interpretation. Third, in public law, adopting the ‘user-deployer’ may have some additional challenges, as it introduces a new agent into the relationship between public authority and private entities. Simultaneously, citizens seem to be mainly excluded from the legal conceptualizing, which risks blurring traditional power constellations.

Conducting Research at the Intersection of HCI and Health: Building and Supporting Teams with Diverse Expertise to Increase Public Health Impact.

Research at the intersection of human-computer interaction (HCI) and health is increasingly done by collaborative cross-disciplinary teams. The need for cross-disciplinary teams arises from the interdisciplinary nature of the work itself-with the need for expertise in a health discipline, experimental design, statistics, and computer science, in addition to HCI. This work can also increase innovation, transfer of knowledge across fields, and have a higher impact on communities. To succeed at a collaborative project, researchers must effectively form and maintain a team that has the right expertise, integrate research perspectives and work practices, align individual and team goals, and secure funding to support the research. However, successfully operating as a team has been challenging for HCI researchers, and can be limited due to a lack of training, shared vocabularies, lack of institutional incentives, support from funding agencies, and more; which significantly inhibits their impact. This workshop aims to draw on the wealth of individual experiences in health project team collaboration across the CHI community and beyond. By bringing together different stakeholders involved in HCI health research, together, we will identify needs experienced during interdisciplinary HCI and health collaborations. We will identify existing practices and success stories for supporting team collaboration and increasing HCI capacity in health research. We aim for participants to leave our workshop with a toolbox of methods to tackle future team challenges, a community of peers who can strive for more effective teamwork, and feeling positioned to make the health impact they wish to see through their work.