Emotional State Of User Research Articles

Deep Learning Based Song Recommendation System Using Facial Expression And Heart Rate

In today’s music streaming landscape, personalized song recommendations are crucial for enhancing user experiences. This research study combines heart rate monitoring and facial expression detection to present a novel deep learning method for song suggestion. With a 92% accuracy rate, a Convolutional Neural Network (CNN) model is used to identify emotional states from facial expressions, and heart rate information adds more context to the user's emotional intensity. The large music library is matched to these emotional cues, guaranteeing that suggested songs correspond with the user's present emotional state. This approach provides a really interesting, emotionally stirring, and contextually appropriate music finding experience.

Integrating Diverse Data Streams for Enhanced Emotional Intelligence in Mental Health Care

This research presents a comprehensive system for real-time emotion recognition and analysis using multimodal data, including image, video, audio, and text. The system employs deep learning models to extract features and classify emotions from each modality. By integrating these predictions, we aim to provide a holistic understanding of a user's emotional state and assess potential risks. The system further analyzes the collected emotion data to identify trends, patterns, and indicators of emotional well-being and suicide risk. Visualizations such as time-series plots, distribution charts, and stacked area charts are employed to represent the emotional dynamics over time.Based on the analysis; the system generates personalized recommendations and alerts for users exhibiting signs of distress or emotional instability, including potential suicide risk factors. The goal is to promote proactive well-being management and early intervention. It is crucial to emphasize that this system is a tool for early detection and should be used in conjunction with professional mental health care.

Personalized Mood-Centric Book Recommendation Integrating Machine Learning with Content Based Filtering

This paper introduces a novel personalized book recommendation system aimed at enhancing subjective well-being (SWB). Despite the vast array of books available on the internet, people often struggle to find literature that aligns with their current emotional state. The system dynamically detects users' emotional states and recommends books tailored to their mood. It utilizes a content-based filtering algorithm to suggest top-rated books in real-time based on the user's current emotional state. For users with low mood, uplifting and inspirational books are recommended, while a mix of happy and sad books is suggested for happy users to acknowledge the nuanced interplay of emotions. Neutral-themed books are proposed to users with a neutral mood for balance. The system employs an emotion classification algorithm for accurate mood detection, analyzing book summaries to extract emotional tones. Users' emotional states are evaluated using a Likert scale, adjusting recommendations accordingly. The methodology involves data collection, preprocessing, emotion classification, and mood meter operation. Results demonstrate the system's effectiveness in providing tailored book recommendations, enriching users' reading experiences.

Towards enhanced performance: an integrated framework of emotional valence, arousal, and task demand

Extensive evaluations exist concerning the linkage between objective task demands and subsequent effects on user performance. However, the human user also experiences a range of emotions related to external task demands. Problematically, little is known about the associations between emotional valence, and arousal associated with the task demand–performance axis. In this paper, we advance a theoretical model concerning such interactive influences using three dimensions: (1) emotional valence, (2) arousal, and (3) task demand. The model evaluates the impact of these dimensions on user performance. It also identifies critical emotional user states, particularly those resulting in negative performance effects, as well as non-critical emotional states that can positively impact performance. Finally, we discuss the implications for affect-adaptive systems that can mitigate the impact of critical emotional states while leveraging the benefits of non-critical ones.

Research on Recognition Method of Social Robot Based on T-A-GCNIIT in the Metaverse

Social robots are used in intelligent customer service, intelligent chat, intelligent shopping guides, and more because of emotion recognition studies in cognitive psychology. However, determining the user's purpose quickly and precisely has proved challenging. Domestic researchers proposed the A-GCNII model to address missing feature information; however, it needs a lot of math. This research offers a social robot recognition approach using the T-A-GCNIIT model and cognitive psychology to optimize computing complexity and performance. The T-A-GCNIIT algorithm processes social network data, and the Viola–Jones algorithm improves social robot intelligence to represent social robots in the meta-universe. The model performs well in node classification, link prediction, community discovery, and other tasks, with enhanced accuracy, recall, F1 score value, and other metrics. The model can also better comprehend the user's emotional state using cognitive psychology to better recognize their purpose and propose a fresh notion for enhancing social robots' cognitive psychology.

Song Recommendation System Based on Facial Emotion

This research is aiming to enhance the user experience in music consumption by incorporating real-time facial emotion analysis. Emotions play a fundamental role in shaping individual preferences, and leveraging facial expressions as a means of understanding user’s emotional states can significantly contribute to personalized music recommendations. Our proposed system begins by capturing real-time facial expressions using a webcam or analyzing static images. These facial expressions are then processed through a CNN-based emotion recognition model trained to classify emotions such as happiness, sadness, anger, and more. The CNN model extracts high-level features from facial images, enabling accurate emotion recognition. Using the detected emotional state as input, our system employs a recommendation algorithm tailored to the user's current emotional state to suggest relevant music or videos from YouTube.

EMOTION BASED MUSIC RECOMMENDATION SYSTEM

The daily lives of people are greatly influenced by music. Everyone wants to listen to music that suits their personal tastes and mood. Users are constantly needed to manually browse the music and construct a playlist based on their mood. The suggested project, which creates a music playlist depending on users' current moods, is quite effective. The most effective approach to convey a person's current mood is through their facial expressions. Webcams are used to capture facial expressions, which are then fed into a learning algorithm to determine the most likely emotion. The emotion recognition system is built using the FER 2013 dataset for training, which allows it to identify seven different emotions. Its operation involves capturing live video input from a webcam, processing it through the model, and then providing a prediction of the detected emotion. By employing webcams to capture real-time facial expressions, the system dynamically adapts playlist recommendations to match the user's emotional state. This seamless integration of emotion recognition technology with music selection enhances user experience by ensuring that the music aligns with their current feelings, thereby fostering a more immersive and personalized listening experience.

Una investigación sobre un sistema de recomendación musical basado en expresiones faciales mediante mecanismos de aprendizaje profundo

In this study, we propose a new music recommendation system (MRS) that combines facial expression recognition technology and deep learning algorithms to respond to the changing music industry environment and provide personalized music recommendations based on the user's emotional state. Our approach includes a thorough study of facial expression recognition, emotion-based music recommendation systems, and deep learning engines, as well as a detailed presentation of the MRS design, system architecture, and deep learning engines used. Through extensive experiments, we evaluate MRS's ability to accurately recognize facial expressions, filter music based on emotional states, and effectively recommend music to users. We analyze the results of follow-up experiments to identify the strengths and limitations of MRS compared to existing approaches, and conduct a comparative study with the latest music recommendation systems based on deep learning and emotion. This comparison highlights the originality and potential of the proposed MRS system to improve user experience and promote the development of artificial intelligence-based music recommendation systems. This study demonstrates the problem of accurately determining a user's emotional state from facial expressions, which requires the integration of facial expression recognition systems, deep learning, and music recommendation systems. Using advanced deep learning techniques and a comprehensive experimental setup, the proposed MRS provides a solution to this problem by facilitating accurate emotional state identification and personalized music recommendations. Overall, MRS represents a powerful and innovative response to the growing demand for accurate and reliable music recommendations and shows significant potential for future collaboration and development of AI-based music recommendation systems.

EMOTION BASED MUSIC PLAYER

Recent studies suggest that humans have a strong bond with music and it plays a crucial role in shaping brain function. It is common for individuals to spend around four hours daily immersing themselves in music that aligns with their emotions and preferences. This initiative aims to create an application that utilizes facial expressions to suggest songs that match the user's mood. Facial cues are vital for non-verbal communication. The Emotion-based music player introduces a groundbreaking idea of selecting music for users based on their emotions. The system uses facial recognition technology to understand the user's emotions and selects music that fits their mood. Computer vision is a versatile tool that enables computers to interpret images and videos effectively. By analyzing facial expressions, the system can determine the user's emotional state and suggest a playlist accordingly, saving time and effort. Key Words: Music, Emotions, Facial Recognition, Computer Vision

Literature Review : Emotion-Based Music Recommendation System with Age Detection

This research introduces an innovative Emotion-Based Music Recommendation System coupled with Age Estimation. Leveraging computer vision techniques, specifically Convolutional Neural Networks (CNNs), the proposed system analyzes facial data, obtained either through a camera or OpenCV, to discern both the emotional state and age of an individual. The synergy of emotion and age information serves as the foundation for personalized music recommendations. To compose playlists and propose songs, a third-party API is used, personalizing the musical experience to the user's emotional state and age group. This integration of facial analysis, deep learning, and music recommendation aims to enhance user engagement and satisfaction by providing a more contextually relevant and emotionally resonant music selection. Keywords— Emotion-Based Music Recommendation System, Age Estimation, Convolutional Neural Networks (CNN), OpenCV, Deep Learning, Third-Party API.

WalkingWizard—A Truly Wearable EEG Headset for Everyday Use

Electroencephalography (EEG) provides an opportunity to gain insights to electrocortical activity without the need for invasive technology. While increasingly used in various application areas, EEG headsets tend to be suited only to a laboratory environment due to the long preparation time to don the headset and the need for users to remain stationary. We present our design of a dry, dual-electrodes flexible PCB assembly that realizes accurate sensing in the face of practical motion artifacts. Using it, we present WalkingWizard, our prototype dry-electrode EEG baseball cap that can be used under motion in everyday scenarios. We first evaluated its hardware performance by comparing its electrode-scalp impedance and ability to capture alpha rhythm against both wet EEG and commercially available dry EEG headsets. We then tested WalkingWizard using steady-state visual evoked potential (SSVEP) experiments, achieving high classification accuracy of 87% for walking speeds up to 5.0 km/h, beating state-of-the-art. Expanding on WalkingWizard, we integrated all necessary electronic components into a flexible PCB assembly—realizing WalkingWizard Integrated , in a truly wearable form-factor. Utilizing WalkingWizard Integrated, we demonstrated several applications as proof-of-concept: classification of SSVEP in VR environment while walking, real-time acquisition of emotional state of users while moving around the neighbourhood, and understanding the effect of guided meditation for relaxation.

Music Recommendation System based on Facial Expressions by using CNN

Abstract: In today's world, the integration of emotion detection and personalized music recommendation has become increasingly prevalent across diverse fields. This project endeavors to develop a cutting-edge music recommendation system that tailors playlists to users' emotional states, as discerned from their facial expressions in real-time. Leveraging the Facial Expression Recognition 2013 (FER 2013) dataset, which encompasses a wide array of emotional expressions including happiness, sadness, anger, fear, disgust, neutrality, and surprise, serves as the foundation for training our model. The project employs the Mediapipe algorithm for feature extraction from facial expressions and utilizes a Convolutional Neural Network (CNN) to classify these emotions accurately. For seamless integration and user interaction, streamlit web browser frameworks are harnessed to facilitate music recommendations within an intuitive interface. Furthermore, the proposed system capitalizes on real-time face detection through camera input using OpenCV, ensuring timely and responsive interaction. By automatically generating music playlists based on the user's current emotional state, our system aims to significantly reduce computational time and overall costs compared to existing literature. In essence, this project presents a comprehensive solution that combines real-time facial expression analysis with personalized music recommendation, offering enhanced efficiency and accuracy while minimizing computational overhead.

Exploring factors in Human Cognition that affect Human Performance during Takeover in Autonomous Driving

Driver's mental load is an important influence on the design of the human-machine interface (HMI) for self-driving cars. During the autopilot-forced takeover process, aspects of the user's cognition, decision-making, attention, and emotional state affect the takeover performance. Therefore, the cognitive and psychological states of drivers must be considered in the HMI design of self-driving cars. And a good HMI design will reduce the user's mental burden and improve the takeover performance. In this study, a model affecting drivers' takeover performance was designed through human factors. And we refine the analysis from three aspects of cognitive load, emotional load, and environmental load by making subjective likert-style-questionnaires, derive the main influencing factors through the reliability analysis of the data, factor analysis, and other methods. This work finds that the clarity, comprehensiveness, and level of humanization of takeover messages, as well as the driver's arousal level when receiving the message, have the greatest impact on the driver's workload.

Emotion-oriented recommender system for personalized control of indoor environmental quality

Indoor Environmental Quality (IEQ) significantly influences health, cognitive abilities, and even moods. With the rise of smart homes and the Internet of Things (IoT), the need for personalized IEQ control has become paramount. Traditional systems, primarily focusing on explicit feedback (i.e., preference), often neglect the role of emotions in shaping this feedback. Therefore, this study presents the Emotion-oriented Recommender System for personalized control of IEQ (ERS-IEQ), based on the R-E-C-S ontology, focusing on (i) Recognizing user's continuous emotional states, (ii) Estimating emotional similarity between users, (iii) Collecting user's feedback on IEQ conditions, and (iv) Systemizing an emotion-oriented recommender system using a graph neural network. In order to confirm the predictive performance of the ERS-IEQ and verify its higher level of excellence compared to traditional recommender systems, a private dataset composed of IEQ conditions, users' explicit feedback, and users' emotional states was built via a human participant experiment using a climate chamber. As a result, the ERS-IEQ significantly enhances the recommender system's predictive performance, particularly in thermal preferences. The number of linguistic terms of emotional similarity has a profound effect on the system's predictions, with four terms proving most effective. In the near future, ERS-IEQ will play a role as a personal assistant in smart home automation, offering emotion-based recommendations. It addresses key challenges in traditional recommender systems, such as the cold start problem and rating sparsity, and ensures personalized adjustments to IEQ conditions in both private and public spaces.

Empowering Financial Peace through Chatbot Guidance

Real-time conversations with humans can be held by intelligent chatbots, utilized by businesses to enhance customer service and streamline operations. A gap is perceived in existing chatbots' achievement of human-like conversational abilities, despite their widespread acceptance and implementation across various sectors. This research paper discusses the development of a unique chatbot providing not only financial guidance but also emotional support, thus sparking a blend of technological innovation and human connection. Emotional states of users are deciphered by the chatbot, using natural language processing (NLP) and sentiment analysis. Tailored responses are generated, offering practical financial advice as well as emotional support during times of stress. Aiming to empower users in managing their financial challenges with serenity, the chatbot integrates emotional support, financial education, goal setting and planning, stress management, and continuous support and monitoring. Cutting-edge technologies, including the Python programming language and NLP libraries like spaCy and NLTK, are incorporated in the project. It interfaces with external financial APIs to provide real-time data enabling informed decision-making. Our chatbot's focus lies in providing a secure and positive environment for users to share financial worries, with the unique ability to comprehend and respond to financial inquiries and emotional nuances in users' messages. By documenting the creation of this chatbot, a contribution is made to the expanding field of emotional intelligence in artificial intelligence applications. The expected outcomes of this research project include an enhanced sense of financial well-being for users, bridging the gap between technological solutions and human-centered support

The Role of Coherent Robot Behavior and Embodiment in Emotion Perception and Recognition During Human-Robot Interaction: Experimental Study.

Social robots are becoming increasingly important as companions in our daily lives. Consequently, humans expect to interact with them using the same mental models applied to human-human interactions, including the use of cospeech gestures. Research efforts have been devoted to understanding users' needs and developing robot's behavioral models that can perceive the user state and properly plan a reaction. Despite the efforts made, some challenges regarding the effect of robot embodiment and behavior in the perception of emotions remain open. The aim of this study is dual. First, it aims to assess the role of the robot's cospeech gestures and embodiment in the user's perceived emotions in terms of valence (stimulus pleasantness), arousal (intensity of evoked emotion), and dominance (degree of control exerted by the stimulus). Second, it aims to evaluate the robot's accuracy in identifying positive, negative, and neutral emotions displayed by interacting humans using 3 supervised machine learning algorithms: support vector machine, random forest, and K-nearest neighbor. Pepper robot was used to elicit the 3 emotions in humans using a set of 60 images retrieved from a standardized database. In particular, 2 experimental conditions for emotion elicitation were performed with Pepper robot: with a static behavior or with a robot that expresses coherent (COH) cospeech behavior. Furthermore, to evaluate the role of the robot embodiment, the third elicitation was performed by asking the participant to interact with a PC, where a graphical interface showed the same images. Each participant was requested to undergo only 1 of the 3 experimental conditions. A total of 60 participants were recruited for this study, 20 for each experimental condition for a total of 3600 interactions. The results showed significant differences (P<.05) in valence, arousal, and dominance when stimulated with the Pepper robot behaving COH with respect to the PC condition, thus underlying the importance of the robot's nonverbal communication and embodiment. A higher valence score was obtained for the elicitation of the robot (COH and robot with static behavior) with respect to the PC. For emotion recognition, the K-nearest neighbor classifiers achieved the best accuracy results. In particular, the COH modality achieved the highest level of accuracy (0.97) when compared with the static behavior and PC elicitations (0.88 and 0.94, respectively). The results suggest that the use of multimodal communication channels, such as cospeech and visual channels, as in the COH modality, may improve the recognition accuracy of the user's emotional state and can reinforce the perceived emotion. Future studies should investigate the effect of age, culture, and cognitive profile on the emotion perception and recognition going beyond the limitation of this work.

Automated EEG based Emotion Detection using Bonobo Optimizer with Deep Learning on Human Computer Interaction

Recently, Emotion detection utilizing EEG signals develops popularity in domain of Human-Computer Interaction (HCI). EEG (electroencephalography) is a non-invasive approach, which processes electrical action from the brain through electrodes located in the scalp. An emotion recognition approach could not only be significant for healthy people among them disabled persons for detecting emotional changes and is utilized for different applications. It is significant to realize that emotion recognition in EEG indications is a difficult task owing to difficult and subjective nature of emotions. In recent times, Machine learning (ML) algorithms like Random Forests or Support Vector Machines (SVM) and Deep Learning (DL) systems namely Recurrent Neural Network (RNN) or Convolutional Neural Network (CNN) are trained on EEG feature extracted and connected emotional labels for classifying the user emotional state. This study presents an Automated EEG-based Emotion Detection using Bonobo Optimizer with Deep Learning (AEEGED-BODL) technique on HCI applications. The goal of the study is to analyze the EEG signals for the classification of several kinds of emotions in HCI applications. To achieve this, the AEEGED-BODL technique uses Higuchi fractal dimension (HFD) approach for extracting features in the EEG signals. Besides, the AEEGED-BODL technique makes use of the quasi-recurrent neural network (QRNN) approach for the detection and classification of distinct kinds of emotions. Furthermore, the BO system was demoralized for optimum hyperparameter selection of QRNN model, which helps in attaining an improved detection rate. The simulation validation of AEEGED-BODL algorithm was simulated on EEG signal database. The comprehensive result stated best outcome of the AEEGED-BODL algorithm over other recent approaches

Emotion-performance relationship in safety-critical human-machine systems

Affect-adaptive systems (AAS) monitor the emotional user state, analyze it concerning the current context and apply appropriate adaptation strategies when necessary to support the user. In safety-critical work environments, human-machine systems aim at maintaining high performance and avoid failure. The detection of critical emotional user states requiring adaptation should take individual differences in the emotion-performance relationship into account. Hence, we suggested the Affective Response Categories (ARCs), a classification concept considering these differences. In the present contribution, we investigated if the proposed ARCs are part of a stable trait or states changing over time. Physiological measures, such as pupil diameter and facial expression; as well as performance data from two air space surveillance laboratory experiments differing in context and sample origin (N1 = 50; N2 = 17) were analyzed. Results of two one-sided tests (TOSTs) and intraclass correlation coefficients (ICC) based on data from both experiments provided evidence for stability of the categories over time, indicating that a continuous user state diagnosis based on a single classification of a user's ARC is possible. With these results, the present contribution provides the foundation for developing a diagnostic engine of users' emotional state.

Semantic speech analysis using machine learning and deep learning techniques: a comprehensive review

Human cognitive functions such as perception, attention, learning, memory, reasoning, and problem-solving are all significantly influenced by emotion. Emotion has a particularly potent impact on attention, modifying its selectivity in particular and influencing behavior and action motivation. Artificial Emotional Intelligence (AEI) technologies enable computers to understand a user's emotional state and respond appropriately. These systems enable a realistic dialogue between people and machines. The current generation of adaptive user interference technologies is built on techniques from data analytics and machine learning (ML), namely deep learning (DL) artificial neural networks (ANN) from multimodal data, such as videos of facial expressions, stance, and gesture, voice, and bio-physiological data (such as eye movement, ECG, respiration, EEG, FMRT, EMG, eye tracking). In this study, we reviewed existing literature based on ML and data analytics techniques being used to detect emotions in speech. The efficacy of data analytics and ML techniques in this unique area of multimodal data processing and extracting emotions from speech. This study analyzes how emotional chatbots, facial expressions, images, and social media texts can be effective in detecting emotions. PRISMA methodology is used to review the existing survey. Support Vector Machines (SVM), Naïve Bayes (NB), Random Forests (RF), Recurrent Neural Networks (RNN), Logistic Regression (LR), etc., are commonly used ML techniques for emotion extraction purposes. This study provides a new taxonomy about the application of ML in SER. The result shows that Long-Short Term Memory (LSTM) and Convolutional Neural Networks (CNN) are found to be the most useful methodology for this purpose.

Review on Facial Expression Based Music Recommendation System

The Emotion-Aware Smart Music Recommendation System represents a pioneering venture in the realm of music technology, aiming to revolutionize the way we connect with music. This system employs advanced emotion recognition techniques, including facial expression analysis, voice sentiment analysis, and biometric sensors, to continuously assess the user's emotional state. Complemented by a meticulously curated music database with songs categorized based on emotional attributes, the system's recommendation engine utilizes machine learning and deep learning models to offer music that aligns with the user's current emotional profile. The integration of a feedback loop ensures that the system adapts and refines its recommendations, improving the user's experience over time. Accessible across a variety of platforms, including mobile apps, web interfaces, and smart speakers, this system aspires to offer a profoundly personalized and emotionally resonant listening experience, redefining the relationship between music and emotion.