User Emotions Research Articles

Emotion Recognizing by a Robotic Solution Initiative (EMOTIVE Project).

Background: Emotion recognition skills are predicted to be fundamental features in social robots. Since facial detection and recognition algorithms are compute-intensive operations, it needs to identify methods that can parallelize the algorithmic operations for large-scale information exchange in real time. The study aims were to identify if traditional machine learning algorithms could be used to assess every user emotions separately, to relate emotion recognizing in two robotic modalities: static or motion robot, and to evaluate the acceptability and usability of assistive robot from an end-user point of view. Methods: Twenty-seven hospital employees (M = 12; F = 15) were recruited to perform the experiment showing 60 positive, negative, or neutral images selected in the International Affective Picture System (IAPS) database. The experiment was performed with the Pepper robot. Concerning experimental phase with Pepper in active mode, a concordant mimicry was programmed based on types of images (positive, negative, and neutral). During the experimentation, the images were shown by a tablet on robot chest and a web interface lasting 7 s for each slide. For each image, the participants were asked to perform a subjective assessment of the perceived emotional experience using the Self-Assessment Manikin (SAM). After participants used robotic solution, Almere model questionnaire (AMQ) and system usability scale (SUS) were administered to assess acceptability, usability, and functionality of robotic solution. Analysis wasperformed on video recordings. The evaluation of three types of attitude (positive, negative, andneutral) wasperformed through two classification algorithms of machine learning: k-nearest neighbors (KNN) and random forest (RF). Results: According to the analysis of emotions performed on the recorded videos, RF algorithm performance wasbetter in terms of accuracy (mean ± sd = 0.98 ± 0.01) and execution time (mean ± sd = 5.73 ± 0.86 s) than KNN algorithm. By RF algorithm, all neutral, positive and negative attitudes had an equal and high precision (mean = 0.98) and F-measure (mean = 0.98). Most of the participants confirmed a high level of usability and acceptability of the robotic solution. Conclusions: RF algorithm performance was better in terms of accuracy and execution time than KNN algorithm. The robot was not a disturbing factor in the arousal of emotions.

Virtual nexus: Female players’ emotional attachment in a role-playing video game

Casual gaming has become an increasingly important part of leisure in the last decade, and role-playing games (RPGs) are one of the most popular genres. RPGs prompt players to develop emotional attachments, so exploring players’ user emotions toward RPGs is of academic significance. This study applied the qualitative method to examine the development of female user emotional attachment in the RPG Onmyoji. The resulting model of developing user emotions between female players and the game and the three stages of the development are structured. The results indicate that female players can form emotional connections with the game, and they generate an emotional ‘virtual nexus’ in the process.

Personal Assistant with Emotion Recognition Based on Artificial Intelligence

This paper deals with a method adapted to create a personal assistant based on emotion recognition. It explains how the emotions of users can be recognized and suggests the relevant methods. Now, we have an abundant number of personal assistants and chatbots: we took one of such bots a step ahead and made our assistant to automatically recognize users' emotions. Through this, our assistant is made to acquire empathy and usage with a lot of fun and advantage. The application has the potential to be used in mobile phones. Many people suffer from social anxiety and depression; our assistant is intended to help them by identifying their emotions and by suggesting ways to cheer them up. The assistant induces the user to be more connected to their phone and also to other people. Initially, our idea was to create a futuristic application, but later this turned out to be more than that. It proves to be advantageous for all mobile phone users. We created a simple personal assistant (Edith) with emotion-sensing ability. We used deep learning techniques to detect users' emotions. We illustrated our model with a lot of human faces displaying different kinds of emotions. Achieving a decent accuracy while recognizing the emotion was the hardest part in this attempt because very slight differences in facial muscles portray dissimilar emotions; identifying the pattern in similar feelings is challenging. We overcame these difficulties by choosing an appropriate dataset with many images to depict our model to recognize the emotions correctly. Our model achieved better accuracy by doing so.

Do Smart Glasses Dream of Sentimental Visions?

Emotion recognition in smart eyewear devices is valuable but challenging. One key limitation of previous works is that the expression-related information like facial or eye images is considered as the only evidence of emotion. However, emotional status is not isolated; it is tightly associated with people's visual perceptions, especially those with emotional implications. However, little work has examined such associations to better illustrate the causes of emotions. In this paper, we study the emotionship analysis problem in eyewear systems, an ambitious task that requires classifying the user's emotions and semantically understanding their potential causes. To this end, we describe EMOShip, a deep-learning-based eyewear system that can automatically detect the wearer's emotional status and simultaneously analyze its associations with semantic-level visual perception. Experimental studies with 20 participants demonstrate that, thanks to its awareness of emotionship, EMOShip achieves superior emotion recognition accuracy compared to existing methods (80.2% vs. 69.4%) and provides a valuable understanding of the causes of emotions. Further pilot studies with 20 additional participants further motivate the potential use of EMOShip to empower emotion-aware applications, such as emotionship self-reflection and emotionship life-logging.

Feature-guided Multimodal Sentiment Analysis towards Industry 4.0

• Advanced and efficient image-text multimodal fusion approach. • Clever use of matrix transformation to achieve alignment of different modal features. • Using attention mechanism to ensure the parallelism of the model and improve the training speed. • The unimodal and multimodal features are stitched together to ensure the complementarity of the modalities. • The pain point problem of difficult to obtain domain datasets is solved by constructing a generic dataset. Combining Artificial Intelligence (AI) to process rich media information has become an important part of Industry 4.0. Sentiment recognition in AI aims to analyze user emotions contained in rich media to facilitate service enhancement. Previous research on sentiment recognition has mainly focused on academia, and few have discussed algorithmic applications and innovations in industry. In this paper, we propose a general approach for multimodal sentiment recognition for images and text. The method provides a new approach for processing rich media information by fully considering the internal features of each modality itself as well as the correlations between the modalities. In the dataset constructed in this paper, the accuracy rate is improved by more than 4% compared with the method using single modality. The effectiveness and generality of the method in multimodal sentiment recognition is demonstrated by extending the experiments with a multimodal public dataset.

Visual Marketing

There are 524 participants completed the online and usable surveys were returned. This study uses structural equation modeling (SEM) analysis as a main method. Based on the identity, satisfaction, and emotions of Instagram users, this study has developed a theoretical framework on their self-presentation, perceived uniqueness, popularity, trust, involvement, aesthetics, and immersion in order to understand their continuance intention toward Instagram. According to the research results, self-presentation, perceived uniqueness, and popularity had a positive influence on identity; trust and involvement had a positive influence on satisfaction and immersion; and identity and satisfaction had a positive influence on continuance intention.

The Influence of Robot Designs on Human Compliance and Emotion: A Virtual Reality Study in the Context of Future Public Transport

As robots enter everyday environments, they start performing tasks originally performed by humans. One field of application is the public transport sector. The deployment of autonomous transport systems comes with a lack of human contact persons for help, guidance, and crowd management. This elicits challenges regarding redirecting and managing passengers. Current solutions on platforms can be replaced or enriched with service robots whose task includes crowd management as well as social interaction. This study investigates how the human-likeness of a robot influences the compliance and emotions of public transport users. A Virtual Reality experiment was conducted (N=33) to evaluate two different robot designs in a bus stop boarding scenario. The two robot designs differ in terms of humanoid appearance. In different experimental trials, participants had to perform a given task that was nullified by instructions from one of the two robots. Additionally, the dissonance of the situation was altered so that the environment either justified the robot's interference or not. Compliant behavior, pleasure, and arousal ratings, as well as task processing times were recorded. The experiment included an individual interview and a post-study questionnaire. The results suggest that future deployment of service robots has the potential to redirect passengers. In dissonant situations, clear reasoning must be given to make the robot effective. However, the robot's visual appearance has a more substantial impact on arousal and subjective preferences than on evoked behavior. The study implies that the presence of a service robot can influence peoples’ choices and gives hints about the importance of giving a reason. However, objectively, the level of the robot's humanoid appearance did not make a difference.

Because I’m Happy—An Overview on Fostering Positive Emotions Through Virtual Reality

In recent years, an increased demand for improving mental health and well-being led to developing procedures capable of enhancing positive experiences. One highly attractive candidate for evoking positive experiences is Virtual Reality (VR), as VR enables users to experience various situations in controlled and safe environments. This overview first investigates how positive emotions, well-being and VR are interconnected. Then, an overview about how and why to induce positive emotions in adult users is provided. Methodological and ethical considerations about VR technology, measurements of VR’s efficacy and user characteristics are reviewed. It emerges that VR is efficient in inducing positive emotions across the adult lifespan and in various settings. Levels of immersion, interactivity, Virtual environment contents, sensory modalities involved and users’ characteristics emerged as key determinants for successfully inducing positive emotions with VR. The main applications of positive VR experiences consist in using VR for relaxation, stress and pain management, motivation for physical activities, and gives promising results for apathy treatment in elderly users. Although VR is efficient in eliciting positive emotions and experiences, the underlying operating mechanisms remain unclear and are yet to be further investigated. Finally, the need for a user-centered approach when designing positive VR experiences, clear guidelines for the use of VR, and a better documentation of its potential adverse effects are addressed.

Face Emotion Recognition based Recommendation System

Face recognition technology has gotten a lot of press because of its wide range of applications and market potential. It is used in a variety of fields, including surveillance systems, digital video editing, and other technical advancements. In the fields of tourism, music, video, and film, these systems have overcome the burden of irrelevant knowledge by taking into account user desires and emotional states. Advice systems, emotion recognition, and machine learning are proposed as thematic categories in the analysis. Our vision is to develop a method for recommending new content that is based on the emotional reactions of the viewers. Music is a form of art that is thought to have a stronger connection to a person's emotions. It has the unique potential to boost one's mood, and video streaming services are becoming more prevalent in people's lives, necessitating the development of better video recommendation systems that respond to their users in a customised manner. Furthermore, many users will believe that travel would be a method to help them cope with their ongoing emotions. Our project aims to create a smart travel recommendation system based on the user's emotional state. This project focuses on developing an efficient music, video, movie, and tourism recommendation system that uses Facial Recognition techniques to assess the emotion of users. The system's overall concept is to identify facial expression and provide music, video, and movie recommendations based on the user's mood.

휘트니스 클럽 이용자의 관계 혜택과 참여지속의도의 관계에서 긍정적 감정의 매개효과

The purpose of this study is to understand whether there is a mediating effect of positive emotions in the relationship between the relationship benefits of fitness club users and the intention to continue participation. In this study, questionnaires and Google questionnaire url were used, and a total of 281 data in Seoul, Gyeonggi-do, and Daejeon were collected and analyzed using convenience sampling methods among non-probability sampling methods to verify the hypothesis. SPSS and AMOS programs were used for data processing, and data were analyzed through frequency analysis, reliability analysis, verification coefficient analysis, correlation analysis, and structural rotation model analysis. The research results are as follows. First, the relationship benefits of fitness club users had a positive effect on positive emotions. Second, the positive emotions of fitness club users had a statistically significant positive effect on participation persistence intention. Third, the relationship benefits of fitness club users had an important positive effect on the intention to continue participation. Fourth, positive emotions were found to be an important factor in the relationship benefits of fitness club users, and as a partial medium for the relationship benefits and participation persistence intention. According to the study, fitness club users are evaluated to have a high intention of continuing participation by increasing positive emotions.

Users opinion and emotion understanding in social media regarding COVID-19 vaccine

Online social platforms or social platforms such as Twitter, Facebook and Instagram have become popular platforms for a public discussion about social topics. Recent studies show that there is a growing tendency for people to talk about COVID-19 pandemic in these online channels. The rapid growth of the infected cases by COVID-19 pandemic makes a lots of anxiety and fear among people. With the recent released of Pfizer vaccine, people start posting a lot of rumors regarding the safety concerns of the vaccine, especially among the elderly people. The aim of this study is to bring out the fact that tweets containing all pertinent details about the COVID-19 vaccine and provides an analysis and understanding of users emotions regarding the recent release of COVID-19 vaccine. This study starts with the collection of tweets related to COVID-19 vaccine and then cleaning the dataset from redundant tweets. In this study, we use Twitter API and Web Scraping techniques to obtain a sample of 50,000 tweets talking about COVID-19 vaccine.Further, The analysis of users emotions is achieved by manually labeling and classifying the tweets to either positive or negative. Then, a deep learning based model is used to train the data and classify the people opinion about COVID-19 vaccine. The experimental results illustrate that high percentage of people have shown a positive attitude towards COVID1-19 vaccine. The proposed method is validated over Twitter datasets and the results also demonstrate that use of deep learning classifier can successfully improve the accuracy of people emotions analysis with an accuracy up to 98% for training set and the accuracy for testing set is 73%.

Sentiment Analysis using Optimized Feature Sets in Different Facebook/Twitter Dataset Domains using Big Data

Now days, in many real life applications, the sentiment analysis plays very vital role for automatic prediction of human being activities especially on online social networks (OSNs). Therefore since from last decade, the research on opinion mining and sentiment analysis is growing with increasing volume of online reviews available over the social media networks like Facebook OSNs. Sentiment analysis falls under the data mining domain research problem. Sentiment analysis is kind of text mining process used to determine the subjective attitude like sentiment from the written texts and hence becoming the main research interest in domain of natural language processing and data mining. The main task in sentiment analysis is classifying human sentiment with objective of classifying the sentiment or emotion of end users for their specific text on OSNs. There are number of research methods designed already for sentiment analysis. There are many factors like accuracy, efficiency, speed etc. used to evaluate the effectiveness of sentiment analysis methods. The MapReduce framework under the domain of big-data is used to minimize the speed of execution and efficiency recently with many data mining methods. The sentiment analysis for Facebook OSNs messages is very challenging tasks as compared to other sentiment analysis because of misspellings and slang words presence in twitter dataset. In this paper, different solutions recently presented are discussed in detail. Then proposed the new approach for sentiment analysis based on hybrid features extraction methods and multi-class Support Vector Machine (SVM). These algorithms are designed using the Big-data techniques to optimize the performance of sentiment analysis

Who is your Fitspiration? An Exploration of Strong and Weak Ties with Emotions, Body Satisfaction, and the Theory of Planned Behavior

ABSTRACT Physical inactivity has become an increasingly important concern for public health. “Fitspiration” social media posts may influence attitudes and intentions toward exercising. An online survey (N = 485) was conducted to examine the potential for fitspiration content from weak and strong ties to shape user emotions, attitudes, norms and behaviors related to exercising. Guided by the theory of planned behavior (TPB) and social cognitive theory (SCT), the results show that exposure to fitspiration content from both strong ties and weak ties were significant predictors of particular attitudes, norms, intentions, actual exercise frequency. However, our path analyses reveal significant indirect paths between exposure to fitspiration content and outcomes for weak ties through negative emotions (guilt), but only through positive emotions (hope and curiosity) for strong ties. Additionally, we found evidence that body satisfaction levels do matter in shaping the interrelationships between exposure to fitspiration content, emotions, and our outcome variables. In light of our results, we encourage researchers to consider the inclusion of emotional responses as antecedents to TPB outcomes and for practitioners to consider the role of source in campaign design.

Research on the Influence of Visual Factors on Emotion Regulation Interaction.

To guide the design direction of emotion regulation products that improve the positive emotions of users, investigation into the correlation between relevant visual factors and multi-dimensional complex emotions is needed. In the present study, an extended product emotion measurement method was adopted to describe the multi-dimensional emotional set of each influencing factor and calculate their weight according to the order. The positive and negative emotion indicators of all influencing factors were compared and the evaluation and ranking factors that affect users’ emotional value of emotion regulation products were analyzed. The experimental results reveal that specific emotion mapping scenes on positive emotion are the most significant among the key factors affecting user emotion. Further, the influence of emotional stickers, interactive data visualization, and text on positive emotions decreased in turn. The influence of emotional text on positive emotion was the lowest. Through investigating the visual factors that affect the psychological emotions of users, the development of emotion regulating products could be guided in a more scientific and reasonable manner.

Parameter-Specific Morphing Reveals Contributions of Timbre to the Perception of Vocal Emotions in Cochlear Implant Users.

Objectives:Research on cochlear implants (CIs) has focused on speech comprehension, with little research on perception of vocal emotions. We compared emotion perception in CI users and normal-hearing (NH) individuals, using parameter-specific voice morphing.Design:Twenty-five CI users and 25 NH individuals (matched for age and gender) performed fearful-angry discriminations on bisyllabic pseudoword stimuli from morph continua across all acoustic parameters (Full), or across selected parameters (F0, Timbre, or Time information), with other parameters set to a noninformative intermediate level.Results:Unsurprisingly, CI users as a group showed lower performance in vocal emotion perception overall. Importantly, while NH individuals used timbre and fundamental frequency (F0) information to equivalent degrees, CI users were far more efficient in using timbre (compared to F0) information for this task. Thus, under the conditions of this task, CIs were inefficient in conveying emotion based on F0 alone. There was enormous variability between CI users, with low performers responding close to guessing level. Echoing previous research, we found that better vocal emotion perception was associated with better quality of life ratings.Conclusions:Some CI users can utilize timbre cues remarkably well when perceiving vocal emotions.

A Socially Emotional Virtual Registrar Based on eBICA and Deep Learning

Virtual embodied social agents are expected to become socially acceptable and useful for emotion regulation with applications in the field of service-oriented artificial intelligence. In this study an eBICA-based model of emotional interaction is used to control behavior of a virtual hotel receptionist: the Registrar, implemented in a virtual environment (VE) and in virtual reality (VR), using a multimodal user-computer interface. The two tasks for the registrar are (1) to perceive and characterize user's personality type and the emotional state based on user's behavior, and (2) to be able to establish and maintain a social contact with the user and to control user's emotions by selecting an appropriate behavior. These functions, first implemented in a cognitive model, then need to be transferred into a neural network using deep learning on model-generated data. Here the general concept of the system is described, and preliminary results are presented. The expected long-term impact includes new smart emotion regulation technologies and new feasible means for emotional communication with and via artifacts.

Website user experience model: testing on journalists

This research aimed to analyse the components of the website user experience (WUX) and the influence of WUX on brand trust. Another objective was to build a WUX model from the perspective of journalist users. This research approach was quantitative with an online survey method. The research sample was 300 journalists. Partial least square-structural equation modelling (PLS-SEM) was used for data processing techniques and hypothesis testing. This research found two alternative WUX models. The first alternative WUX model showed a significant relationship between WUX (with six components in WUX) to brand trust. The second alternative WUX model showed a meaningful relationship between access speed, user value, user's emotion (three components in the WUX framework that were treated as variables) to brand trust. Of the two models, this research recommended the first alternative WUX model because the components of WUX were better and more complex.

A review on music recommendations based on facial expression

Deciding which music to listen to from the huge collection of existing options is often confusing. Depending on the user's mood, multiple suggestion frames are available for topics such as music, food, and shopping. The main purpose of this music recommendation system is to provide users with suggestions based on users' tastes. By analysing the user's facial expressions and emotions, it is possible to understand the user's current emotional and psychological state. Music and video are areas where there is a great opportunity to offer customers a wide range of choices, taking into account customer preferences and recorded information. It is well known that people use facial expressions to more clearly expresses what they want to say and the context of the words. More than 60% of the users believe that the song library has too many songs at any given time to find the one that needs to be played. Developing a recommendation system could help users decide which music to listen to and reduce stress levels. Users do not have to waste time searching and searching for songs; it will recognize the track that best fits the user's mood and present the songs to the user according to the user's mood. Music plays a role in emotions, which in turn affects mood. Books, movies, and Television (TV) shows are a few more means, but unlike these, music conveys its message in pure moments. It can help us when people feel low. When people listen to sad songs, their mood tends to drop. When they listen to happy songs, it makes them feel happier. This music recommendation model will mainly work to improve the mood of the user by providing a detection track for the user's facial expression and recommending the preferred song according to the user's expression. User images are captured using webcams. A user's picture is taken, and depending on the user's mood or feeling, appropriate songs are displayed from the user's playlist to meet the user's requirements.

Negative Emotions Sensitive Humanoid Robot with Attention-Enhanced Facial Expression Recognition Network

Lonely older adults and persons restricted in movements are apt to cause negative emotions, which is harmful to their mental health. A humanoid robot with audiovisual interactions is presented, which can correspondingly output positive facial expressions to relieve human's negative facial expressions. The negative emotions are identified through an attention-enhanced facial expression recognition (FER) network. The network is firstly trained on MMEW macro-and micro-expression databases to discover expression-related features. Then, macro-expression recognition tasks are performed by fine-tuning the trained models on several benchmarking FER databases, including CK+ and Oulu-CASIA. A transformer network is introduced to process the sequential features engineered by the FER network and output a final stable control order. This order is used to control the robot's facial motor units to generate different expressions, e.g., a smile expression. Evaluations on benchmarking databases verify that the proposed method can precisely recognize facial expressions. The joint modulation with the humanoid robot proves that the robot can respond effectively to the user's negative emotions.

Cross-Event User Reaction Prediction on a Social Network Platform

Social network surges with multiple tweets with mixture of multiple emotions by many users when events like rape, robbery, war and murder, we use this user data to analyze user emotions between cross-events and try to predict user reactions for the next possible such event. Cross-events are a series of events that belong under the same umbrella of topics and are related to the events occurring prior to it. The proposed system solve this problem using collaborative filtering using Topical and Social context. The Text Rank Algorithm is an unsupervised algorithm used for keyword extraction. Count Vectorizer is used on preprocessed text to get the frequency of words throughout the text which is used as training data to get a probability of emotion using a logistic regression model. We incorporated social context along with topical context to account for homophily and used the Low-rank matrix factorization method for user-topic prediction. The model as an output gives a total of 8 emotions which include Shame, Disgust, Anger, Fear, Sadness, Neutral, Surprise and Joy. Finally, the model is able to predict emotions with an accuracy of 95% considering cross events.