Human-computer Interaction Research Research Articles

DEEP LEARNING BASED EMOTION DETECTION SYSTEM

Emotion detection plays a crucial role in human-computer interaction, affective computing, and psychological research. Traditional methods of emotion detection often rely on manual annotations or subjective assessments, which can be time- consuming and prone to biases. In this paper, we propose a Deep Learning-Based Emotion Detection System that leverages advanced neural network architectures to automatically recognize emotions from facial expressions in real-time. The system utilizes convolutional neural networks (CNNs) for feature extraction from facial images and recurrent neural networks (RNNs) for sequence modeling of temporal dynamics in facial expressions. Our proposed system addresses key challenges in emotion detection, including variations in facial expressions, lighting conditions, and occlusions. By training on large-scale datasets of labeled facial images, the system learns to accurately classify emotions into predefined categories such as happiness, sadness, anger, fear, disgust, and surprise. We evaluate the performance of the system using standard metrics such as accuracy, precision, recall, and F1-score, demonstrating its effectiveness in real- world scenarios. The results indicate that the Deep Learning-Based Emotion Detection System achieves state-of-the-art performance in recognizing emotions from facial expressions, outperforming traditional methods. The system's ability to analyze facial dynamics in real-time opens up opportunities for applications in human-computer interaction, virtual reality, mental health monitoring, and personalized user experiences. Keywords: Deep Learning, Emotion Detection, Facial Expression Recognition, Convolutional Neural Networks, Recurrent Neural Networks, Affective Computing. Keywords: emotion Detection, Machine Learning, Classifier, Natural Language Processing

EEG Emotion Recognition Network Based on Attention and Spatiotemporal Convolution.

Human emotions are complex psychological and physiological responses to external stimuli. Correctly identifying and providing feedback on emotions is an important goal in human-computer interaction research. Compared to facial expressions, speech, or other physiological signals, using electroencephalogram (EEG) signals for the task of emotion recognition has advantages in terms of authenticity, objectivity, and high reliability; thus, it is attracting increasing attention from researchers. However, the current methods have significant room for improvement in terms of the combination of information exchange between different brain regions and time-frequency feature extraction. Therefore, this paper proposes an EEG emotion recognition network, namely, self-organized graph pesudo-3D convolution (SOGPCN), based on attention and spatiotemporal convolution. Unlike previous methods that directly construct graph structures for brain channels, the proposed SOGPCN method considers that the spatial relationships between electrodes in each frequency band differ. First, a self-organizing map is constructed for each channel in each frequency band to obtain the 10 most relevant channels to the current channel, and graph convolution is employed to capture the spatial relationships between all channels in the self-organizing map constructed for each channel in each frequency band. Then, pseudo-three-dimensional convolution combined with partial dot product attention is implemented to extract the temporal features of the EEG sequence. Finally, LSTM is employed to learn the contextual information between adjacent time-series data. Subject-dependent and subject-independent experiments are conducted on the SEED dataset to evaluate the performance of the proposed SOGPCN method, which achieves recognition accuracies of 95.26% and 94.22%, respectively, indicating that the proposed method outperforms several baseline methods.

An Approach Towards to Real Time AI Desktop Voice Assistant

The advent of artificial intelligence (AI) has revolutionized human-computer interaction, making it more natural and intuitive. This research paper presents the development and implementation of an AI Desktop Voice Assistant designed to enhance productivity and accessibility for users. The voice assistant leverages advanced speech recognition, natural language processing (NLP), and machine learning techniques to understand and execute user commands. Key functionalities include voice-activated application control, web searches, personalized reminders, and real-time information retrieval. Our system integrates with widely-used APIs and services, providing a seamless user experience across various tasks. The paper discusses the architecture of the voice assistant, detailing the integration of components such as the speech recognition engine, NLP models, and the dialogue management system. We explore the challenges encountered during development, including accurate speech recognition in noisy environments and handling ambiguous user commands. Solutions implemented to address these challenges are also presented. Furthermore, we conduct a usability study to evaluate the effectiveness and user satisfaction of the voice assistant. The results indicate a high level of user engagement and satisfaction, demonstrating the practical benefits of incorporating AI-driven voice interfaces into desktop environments. Our findings contribute to the ongoing research in human-computer interaction, suggesting pathways for future enhancements in AI voice assistants..

EmoSonics: Emotion Detection via Voice and Speech Recognition

Understanding emotions from speech is like deciphering a rich tapestry of human expression in the realm of human-computer interaction. It's akin to listening to someone's tone and inflection to discern whether they're happy, surprised, or experiencing a range of other feelings. Researchers use a variety of techniques, from analyzing speech patterns to utilizing advanced technologies like fMRI, to decode these emotional cues. Emotions aren't just simple labels; they're complex and nuanced, demanding sophisticated methods for accurate interpretation. Some methods break emotions down into simple categories, while others embrace the intricacies of human emotion, treating them as continuous variables. Ultimately, the goal is to enhance communication between humans and computers by enabling machines to understand and respond appropriately to our emotional states. This pursuit underscores the significance of emotion detection in speech analysis and highlights the need for continually evolving methodologies in human-computer interaction research.

Multimedia Human-Computer Interaction Method in Video Animation Based on Artificial Intelligence Technology

With the development of computer technology innovation, be able to deal with the media comprehensive information and real-time information interaction with the computer multimedia technology arises at the historic moment, it promotes the application fields of computer widen to industrial all aspects of life. As the product of digital technology, animation technology plays an irreplaceable role in the production of multimedia courseware. However, the existing human-computer interaction methods have shortcomings such as incomplete extraction of video features and poor human-computer interaction effect. In this context, this paper designs a multimedia human-computer interaction method for animation works based on CNN model. First of all, the original video data is collected and preprocessed. Then it is input into the HCI framework based on CNN model for feature extraction. Finally, the effectiveness and practicability of the proposed method are proved by simulation experiments, which provides a reference and basis for the research of modern human-computer interaction.

Pilot study of the relation between various dynamics of avatar experience and perceptual characteristics

In recent years, due to the prevalence of virtual reality (VR) and human-computer interaction (HCI) research, along with the expectation that understanding the process of establishing sense of ownership, sense of agency, and limb heaviness (in this study, limb heaviness is replaced with comfort level) will contribute to the development of various medical rehabilitation, various studies have been actively conducted in these fields. Previous studies have indicated that each perceptual characteristics decrease in response to positive delay. However, it is still unclear how each perceptual characteristic changes in response to negative delay. Therefore, the purpose of this study was to deduce how changes occur in the perceptual characteristics when certain settings are manipulated using the avatar developed in this study. This study conducted experiments using an avatar system developed for this research that uses electromyography as the interface. Two separate experiments involved twelve participants: a preliminary experiment and a main experiment. As observed in the previous study, it was confirmed that each perceptual characteristics decreased for positive delay. In addition, the range of the preliminary experiment was insufficient for the purpose of this study, which was to confirm the perceptual characteristics for negative delay, thus confirming the validity of conducting this experiment. Meanwhile, the main experiment showed that the sense of ownership, sense of agency, and comfort level decreased gradually as delay time decreased, (i.e., this event is prior to action with intention, which could not be examined in the previous study). This suggests that control by the brain-machine interface is difficult to use when it is too fast. In addition, the distribution of the most strongly perceived settings in human perceptual characteristics was wider in regions with larger delays, suggesting this may lead to the evaluation of an internal model believed to exist in the human cerebellum. The avatar developed for this study may have the potential to create a new experimental paradigm for perceptual characteristics.

User accounts: How technological concepts permeate public law through the EU's AI Act

This article argues that through the EU's technology regulation, technological concepts permeate legal language. Such concepts may function as transplants, even irritants, causing tensions and uncertainties. As technology regulation is increasingly horizontal, i.e. obligating private and public actors alike, these newfound legal concepts remain disconnected from established public law vocabulary and the power constellations it represents and embeds. We approach this evolution of legal language from public law perspective and concentrate on the concepts of ‘user’ and ‘deployer’ in the EU's upcoming Artificial Intelligence Act. We discuss these emerging legal concepts in relation to the rich theorizing on the concepts in human–computer interaction research. Our analysis demonstrates a discrepancy between legal and technology-oriented conceptualizations of the ‘user-deployer’. We draw three conclusions. First, the digital revolution is taking place in conceptual-linguistic practices of law, and not only when translating law into code. Second, when external concepts are appropriated into law, they are uprooted from their established habitat, which may result in unpredictability in future legal interpretation. Third, in public law, adopting the ‘user-deployer’ may have some additional challenges, as it introduces a new agent into the relationship between public authority and private entities. Simultaneously, citizens seem to be mainly excluded from the legal conceptualizing, which risks blurring traditional power constellations.

Conducting Research at the Intersection of HCI and Health: Building and Supporting Teams with Diverse Expertise to Increase Public Health Impact.

Research at the intersection of human-computer interaction (HCI) and health is increasingly done by collaborative cross-disciplinary teams. The need for cross-disciplinary teams arises from the interdisciplinary nature of the work itself-with the need for expertise in a health discipline, experimental design, statistics, and computer science, in addition to HCI. This work can also increase innovation, transfer of knowledge across fields, and have a higher impact on communities. To succeed at a collaborative project, researchers must effectively form and maintain a team that has the right expertise, integrate research perspectives and work practices, align individual and team goals, and secure funding to support the research. However, successfully operating as a team has been challenging for HCI researchers, and can be limited due to a lack of training, shared vocabularies, lack of institutional incentives, support from funding agencies, and more; which significantly inhibits their impact. This workshop aims to draw on the wealth of individual experiences in health project team collaboration across the CHI community and beyond. By bringing together different stakeholders involved in HCI health research, together, we will identify needs experienced during interdisciplinary HCI and health collaborations. We will identify existing practices and success stories for supporting team collaboration and increasing HCI capacity in health research. We aim for participants to leave our workshop with a toolbox of methods to tackle future team challenges, a community of peers who can strive for more effective teamwork, and feeling positioned to make the health impact they wish to see through their work.

Using generative AI in human resource development: an applied research study

ABSTRACT In this article, we report results of an experimental study investigating the use of ChatGPT in the design of an HRD intervention. Our goal was to identify the benefits and drawbacks of using this technology in HRD practice. The theoretical framework for the study was provided by the Activity Theory, an often used approach in Human-Computer Interaction research. The study participants were HRD doctoral students from two large public universities in the USA. The participants used ChatGPT in developing an HRD intervention plan, based on a supplied organisational scenario. Data for the analysis included the intervention plans, prompts used by participants, outputs received from ChatGPT, and participants’ responses to open-ended questions. The study revealed varied participant experiences with ChatGPT, including differences in time investment and interaction rules. Among key findings were the tendency to interact with ChatGPT as if it were a human consultant, and steps to establish division of labour between the tool and the human agent. ChatGPT was valued for its usefulness in initial planning and brainstorming ideas for HRD interventions, yet its limitations were apparent in generic responses and lack of contextual understanding. Across all participants, the crucial role of human expertise in refining ChatGPT’s output was consistently highlighted.

Towards more trusted virtual physicians: the combinative effects of healthcare chatbot design cues and threat perception on health information trust

ABSTRACT Although virtual healthcare assistants have been extensively adopted in the healthcare industry, users still question their reliability. Based on the Computers are Social Actor (CASA) paradigm, the current study conducted an experiment to examine how different chatbot design cues (Chatbot vs. Layperson vs. Doctor) affect users’ trust towards virtual healthcare assistants. Results indicate that the doctor-like design cues and bot-like design cues (vs. layperson-like design cues) elicited significantly greater perceived chatbot expertise, which enhanced users’ trust in health information. This study further found the significant moderating effects of users’ perceived threat on chatbot expertise and privacy concerns. Positive effects of the doctor-like design cue and the bot-like design cue on perceived expertise were found to be significant only for those whose perceived threat was high. Interestingly, the doctor-like design cues led to greater (less) privacy concerns when the perceived threat level was low (high). The findings provide human–computer interaction researchers and chatbot UX/UI designers with important theoretical and practical implications.

Towards identifying the components of students' AI literacy: An exploratory study based on Hungarian higher education students' perceptions

With the advent of the popular use of artificial intelligence (AI), the higher education (HE) sector is now facing a new challenge regarding how to exploit the educational potentials of Human-computer interaction (HCI). Developing students’ AI literacy is now attracting the attention of the HE and HCI research community. This paper aims to explore HE students’ perceptions of efficient and critical use of AI tools, and to systematically map the potential components of AI literacy as a new 21st century skill for HE students. This study applied a qualitative, exploratory approach in the form of semi-structured interviews with HE students. Results indicate that the participants primarily use ChatGPT for tasks such as brainstorming, topic selection, searching for information, and translation. While many find it useful for creating and reformatting texts, some encountered challenges, including generality in responses, outdated information, and issues during exams. Students highlighted its effectiveness in various academic tasks, from writing essays and CVs to language learning and transcription. Instructors’ perspectives on ChatGPT varied, with some advocating for its integration, while others expressed concerns about job security and misinformation. The implications of the study call for a more systematic introduction and discussion around AI literacy in educational settings.

Examining the Integration of a GPS-based Staff Performance Monitoring and Assessment System within Academic Settings

Research evidence of manipulations of appraisal systems of various institutions in Nigeria abound, and little is known about the application of a Geographical positioning system (GPS) enabled system in this context. To cover this gap, this study investigates the integration of a real-time staff performance monitoring and assessment system with GPS technology, within academic settings. With the increasing emphasis on quality education and efficient resource utilization, academic institutions are exploring innovative approaches to monitor staff performance and student outcomes. The integration of real-time performance systems offers a promising avenue to achieve these goals by providing timely feedback, fostering accountability, and facilitating data-driven decision-making processes. This study outlines the key components of such a system, including its technological infrastructure, data collection methods, and performance evaluation metrics. Additionally, it explores the potential benefits and challenges associated with its adoption, considering factors such as staff engagement, privacy concerns, and institutional culture. Through a comprehensive examination of existing literature and case studies, this research aims to provide valuable insights and recommendations for academic institutions seeking to leverage real-time staff performance monitoring for institutional enhancement and educational excellence. While this study highlights the significance of stakeholder collaboration and continuous improvement mechanisms in ensuring the successful integration and sustained effectiveness of the monitoring system, it introduces a nuanced novel GPS technology and highlights some design implications which will guide future human-computer interaction researchers.

HCI Research and Innovation in China: A 10-Year Perspective

In the past years, human computer interaction (HCI) research and innovation have developed substantially, leading to a number of fruitful research topics. In this paper, we surveyed the HCI research and innovation in China from a 10-year perspective. We analyzed the popular research methodology and topics among Chinese researchers, including human modeling, user interface techniques, context awareness, user acceptance and performance, user experience design, human-AI interaction, HCI applications and social influences. We also conducted a bibliography analysis on the published papers in top-tier conferences and journals, which revealed a significant rising trend, and a generally broad distribution of research types. Moreover, we described typical applications and the industry influence of the research outcomes. We concluded with implications and reflections for HCI researchers across the world and shared the future research trends envisioned by Chinese researchers.

Human-Computer Interaction in Smart Living Environments: Challenges and Opportunities

The proliferation of smart technologies in living environments has ushered in an era where humans and computers interact more intimately than ever before. This research paper delves into the complex landscape of Human-Computer Interaction (HCI) within the context of smart living environments, shedding light on the multifaceted challenges and exciting opportunities that lie ahead.
 In this exploration, we first review the fundamental concepts and technologies that underpin smart living environments, emphasizing the centrality of HCI in bridging the gap between humans and the myriad of interconnected devices and systems that constitute these environments. We then identify and analyze key challenges such as user privacy, security, usability, and adaptability, which continue to shape the HCI landscape in smart living.
 The paper also highlights the emerging opportunities for HCI research, including the integration of augmented reality (AR), virtual reality (VR), and natural language processing (NLP) into smart living systems, fostering more intuitive and seamless interactions. Additionally, it discusses the potential for personalized and context-aware HCI solutions that enhance user experiences and promote inclusivity.
 By addressing these challenges and harnessing these opportunities, this research seeks to provide valuable insights for HCI researchers, designers, and policymakers, ultimately contributing to the development of smarter and more user-friendly living environments that enhance our quality of life while respecting our fundamental rights and values.

Bibliometric Analysis of the Research Landscape in Human-Computer Interaction in Ibero-America

Human-Computer Interaction (HCI) is a research discipline that optimizes human-machine interfaces. Ibero-America has become a leading region in HCI advances adapted to its unique socioeconomic environment. However, a global characterization of the research landscape for this discipline in the region still needs to be improved. Therefore, this study aimed to analyze the scientific literature on HCI in Ibero-America using bibliometric techniques, identifying key authors, institutions, conceptual structure, and research trends. The methodology employed consisted of a bibliometric analysis of 9812 documents from 1990-2023 from Scopus and Web of Science. The data were analyzed using Bibliometrix and VOSviewer for statistical and network analysis. The results revealed that Spain, Brazil, Portugal, Mexico, and Colombia are at the forefront of HCI research in the region, with more than 90 % of total publications experiencing an annual growth rate of approximately 14.6 %, driven by strong international collaborations. Key topics are human-computer interaction, virtual reality, augmented reality, and video games, while education is one of the main application areas. The main conclusion is that the HCI research landscape in Ibero-America is characterized by rapid evolution, international integration, and conceptual diversity, indicative of a mature and interdisciplinary field that responds to regional demands. The advancement of the field could be driven by increased private-sector funding. Future system designs in the region should prioritize cultural sensitivity and inclusion.

Capturing the College Experience

Understanding the dynamics of mental health among undergraduate students across the college years is of critical importance, particularly during a global pandemic. In our study, we track two cohorts of first-year students at Dartmouth College for four years, both on and off campus, creating the longest longitudinal mobile sensing study to date. Using passive sensor data, surveys, and interviews, we capture changing behaviors before, during, and after the COVID-19 pandemic subsides. Our findings reveal the pandemic's impact on students' mental health, gender based behavioral differences, impact of changing living conditions and evidence of persistent behavioral patterns as the pandemic subsides. We observe that while some behaviors return to normal, others remain elevated. Tracking over 200 undergraduate students from high school to graduation, our study provides invaluable insights into changing behaviors, resilience and mental health in college life. Conducting a long-term study with frequent phone OS updates poses significant challenges for mobile sensing apps, data completeness and compliance. Our results offer new insights for Human-Computer Interaction researchers, educators and administrators regarding college life pressures. We also detail the public release of the de-identified College Experience Study dataset used in this paper and discuss a number of open research questions that could be studied using the public dataset.

Human-Computer Interaction in Agricultural User Interfaces

This study explores the intersection of Human-Computer Interaction (HCI) and agricultural user interfaces, aiming to enhance the efficiency and user experience within the agricultural domain. A thorough literature review identifies current trends, challenges, and gaps in HCI research specific to agricultural interfaces. The research objectives encompass the proposal of HCI design principles tailored to the unique characteristics of the agricultural environment. Methodologically, the study employs user surveys, interviews, and usability testing to gather insights into user preferences and challenges. The paper presents case studies highlighting successful implementations of HCI principles in agricultural settings, providing tangible examples of improved user engagement and task performance. Challenges and limitations in HCI adoption in agriculture are discussed, along with user feedback and evaluations shaping interface design. The implications of the research extend to HCI practitioners, agricultural technology developers, and policymakers, offering practical insights for future developments. This paper contributes valuable knowledge to the evolving field of HCI in agricultural contexts, emphasizing the importance of user-centered design.

Voice-based age, gender, and language recognition based on ResNet deep model and transfer learning in spectro-temporal domain

In personal identity recognition systems, detecting a person's age, gender, and language using voice signal characteristics is a crucial issue, especially because of the importance of security considerations. Age, gender, and language classification problems are important in signal processing because they are used to analyze and understand human behavior, interactions, and preferences. This can be especially useful in the fields of human-computer interaction, psychology, and social science research. In this paper, a new system for detecting a speaker's age, gender, and language based on deep learning models is presented. Deep learning models have shown great efficacy in various fields of signal processing. For this paper, a range of deep models were tested, including convolutional neural networks (CNNs), recurrent neural network (RNN), and a fine-tuning ResNet34 architecture. Additionally, techniques such as transfer learning were applied to improve the efficiency of the proposed system. The input voice signals are preprocessed by applying the spectro-temporal transform to obtain additional features that can be fed to the ResNet34 model, which is designed specifically for the task of voice signal processing. The dataset used in this paper was sourced from the Mozilla common voice initiative, which is dedicated to advancing speech recognition and language identification technologies. The performance of the proposed algorithm was evaluated in the presence of Gaussian noise to determine its robustness. The experimental results demonstrated that the proposed algorithm significantly outperformed basic algorithms and other deep neural networks in terms of age and gender recognition from voice signals.

Coral Morph: An Artistic Shape-Changing Textile Installation for Mindful Emotion Regulation in the Wild

Mindful human–computer interaction has been increasingly used to induce emotion regulation. The Coral Morph is an artistic shape-changing textile installation designed for mindful emotion regulation in public spaces. The installation incorporates affective soft-robotic material movements, heart rate physicalization, and expressive interactive textile features conceived to enhance sensory engagement. In an exhibition, 55 participants were recruited to interact with Coral Morph, allowing the evaluation of its sensory engagement, somesthetic appreciation, and lifelikeness perception. Participants’ facial expressions during interaction were triangulated with their questionnaire responses, and short interviews were conducted enquiring on the somatic feedback after the interaction. The empirical findings confirmed that participants perceived the installation as emotionally intelligent, animate, likable, safe, and interesting. Participants regarded the experience as pleasant, calm, positive, and relaxed, appreciating the somesthetic aspects, thus highlighting the effectiveness of the design elements. Furthermore, we highlighted the significance of individual differences in shaping the interactive encounter. These findings inform well-being and human–computer interaction researchers on designing future responsive installations for mindfulness and identifying future research avenues, encompassing the nuanced exploration of interactive material behaviors and advancements in evaluation methodologies.

Emotion Detection via Voice and Speech Recognition

In human-computer interaction, spoken emotion detection is essential yet difficult. Researchers extract emotional information from speech sounds using a variety of analytic techniques. Effective communication requires an understanding of human emotions, which range from happiness to astonishment and beyond. Classification, pre-processing, feature extraction, and pattern recognition are all involved in emotion recognition. While some approaches take into account a continuous spectrum, others concentrate on a single-dimensional representation. The study of emotional cognition benefits from the use of technologies such as fMRI, which facilitate the distinction between positive and negative emotions. The intricacy and significance of identifying emotions in speech are emphasised in this abstract, underscoring the necessity of sophisticated methodologies in the field of human-computer interaction research