User's Mental State Research Articles

Analyzing and computing humans by means of the brain using Brain-Computer Interfaces - understanding the user - previous evidence, self-relevance and the user's self-concept as potential superordinate human factors of relevance.

Brain-computer interfaces (BCIs) are well-known instances of how technology can convert a user's brain activity taken from non-invasive electroencephalography (EEG) into computer commands for the purpose of computer-assisted communication and interaction. However, not all users are attaining the accuracy required to use a BCI consistently, despite advancements in technology. Accordingly, previous research suggests that human factors could be responsible for the variance in BCI performance among users. Therefore, the user's internal mental states and traits including motivation, affect or cognition, personality traits, or the user's satisfaction, beliefs or trust in the technology have been investigated. Going a step further, this manuscript aims to discuss which human factors could be potential superordinate factors that influence BCI performance, implicitly, explicitly as well as inter- and intraindividually. Based on the results of previous studies that used comparable protocols to examine the motivational, affective, cognitive state or personality traits of healthy and vulnerable EEG-BCI users within and across well-investigated BCIs (P300-BCIs or SMR-BCIs, respectively), it is proposed that the self-relevance of tasks and stimuli and the user's self-concept provide a huge potential for BCI applications. As potential key human factors self-relevance and the user's self-concept (self-referential knowledge and beliefs about one's self) guide information processing and modulate the user's motivation, attention, or feelings of ownership, agency, and autonomy. Changes in the self-relevance of tasks and stimuli as well as self-referential processing related to one's self (self-concept) trigger changes in neurophysiological activity in specific brain networks relevant to BCI. Accordingly, concrete examples will be provided to discuss how past and future research could incorporate self-relevance and the user's self-concept in the BCI setting - including paradigms, user instructions, and training sessions.

BCI-AMSH: A MATLAB based open-source brain computer interface assistive application for mental stress healing

Numerous Brain computer interface (BCI) assistive software applications or toolboxes help to monitor the status of brain through Electroencephalogram (EEG); however, currently there is no application available to alleviate mental stress along with EEG analysis. In this paper, a new MATLAB-based assistive application called BCI-AMSH (Brain-Computer Interface-Assistive Mental Stress Healing) is developed. This application includes EEG signal offline analysis and stress healing techniques, such as guided meditation and singing bowl sound therapy, combined with real-time EEG analysis using the Enobio-8 device. The EEG offline analysis and stress healing modules are designed to be platform-independent.EEG offline analysis module offers signal preprocessing methods, diverse EEG signal analysis techniques, feature extraction methods, such as time and frequency domain analysis, and a range of machine learning and deep learning models for classification, various performance evaluation metrics. Additionally, it generates comprehensive EEG report about the user's mental states such as focus, relax and stress both in real-time and offline analysis. Furthermore, the stress healing module provides a variety of guided breathing practices and singing bowl sound therapy practices to effectively reduce stress levels. This new application is designed to be very simple and user-friendly, making it accessible to users without prior experience. Its potential applications are significant, particularly in the clinical industry for mental health purposes, as well as for novice researchers seeking to explore new possibilities in BCI.

Impact of COVID-19 on mental health in China: analysis based on sentiment knowledge enhanced pre-training and XGBoost algorithm.

Coronavirus disease 2019 (COVID-19) is causing a serious impact on the people living in countries across the entire world. The spread of this pandemic globally has led people worry every day about losing their jobs or even being threatened by the virus. This pandemic caused people to experience more serious psychological problems than we realized. However, there has been little research on how COVID-19 affects the mental health of the people. In this article, we attempted to use the social text data about COVID-19 on Sina Weibo (the largest "tweet" platform in China, and we will also call Weibo as tweet in the following content), to explore the impact of COVID-19 on the mental health of Chinese people. First, we fifilter the tweet data by selecting examples that contain COVID-19 and COVID-19 correlated keywords. However, we segment the filtered tweets, extract meaningful words, and construct a word vector sparse matrix as the measurement of every tweet. Then, for the model's labels, we use sentiment knowledge enhanced pre-training model (SKEP), a deep learning framework published by Baidu that measures the user's mental state. Through SKEP, we can obtain the probabilities of the user's positive and negative mental states. Finally, we use the XGBoost algorithm to study the relationship between the word vector sparse matrix and the mental health state of users. Our research shows that social text data can, indeed, reflect the mental health state of users to a large extent, and social data can be used to explore the impact of COVID-19 on mental health, which can help frame the public health policy.

Facial Emotion Song Recommender System

Sometimes, it is very difficult for someone to determine whether a person wants to hear a particular music from the vast array of available choices. So, this paper has proposed a new concept of playing music that is based on the emotion of the user. The primary goal of the music recommendation system which is proposed in this paper is to offer customers recommendations that match their tastes. The most current view of the paper involves manually playing the jukebox, using wearable computers, or classifying based on auditory characteristics. Understanding the user's present emotional or mental state may result from analysing the user's facialexpression and emotions. One area is having a great possibility to provide the audience, with a vast variety of options that are based on their preferences and music and video. In this paper, the primary goal is to show a playlist of songs on any particular music application (YouTube/Spotify) based on each person’s mood. Several images of the user are collected at that precise moment using a camera with the user's consent. To determine a person's mood, these photos go through a thorough testing andtraining process. For this, the deep learning technique called CNN is used to categorize various emotions. After this, based on the trained model, the various emotions are categorized and based on this the music playlist is generated.

Anti-cocaine Vaccine Development: Where Are We Now and Where Are We Going?

Cocaine is one of the oldest and most widely used illicit drugs in the world and is responsible for major worldwide medical and social problems. Drug addiction is a disease state where the body relies on a substance for normal functioning and develops a physical dependence leading to compulsive and repetitive use despite negative consequences to the user's health, mental state, or social life. The primary driver for the development of anti-cocaine vaccines has been the failure to develop effective pharmacological treatments to combat cocaine dependence. Despite several decades of research, no approved pharmacological treatments for cocaine dependence are available to assist addicts to overcome cocaine withdrawal or to prevent drug relapse. This Perspective highlights the challenges associated with anti-cocaine vaccines, including the current state of anti-cocaine vaccines and catalytic antibody research to aid in the fight against cocaine addiction.

Mental State Adaptive Interfaces as a Remedy to the Issue of Long-term, Continuous Human Machine Interaction

In order to promote safer and more efficient human-machine interaction, this article advocates for the employment of adaptive systems that account for the user's mental state throughout the duration of lengthy, continuous usage. Perhaps what is needed are adaptive systems that can adjust to the user's mood. The operator's state of mind may be inferred using a combination of operator-independent metrics (for instance, time of day and weather) and behavior (for instance, lane deviation and response time) and physiological (for instance, heart activity and electroencephalography) indicators. Several changes may be made to the dynamic between the operator and the system to mitigate the impacts of the operator's diminished cognitive capacity and preserve the reliability and efficacy of operations. Depending on the specifics of the job at hand and the difficulties that must be overcome, adjustments may be made to factors such as the type of the information presented, the structure of the presentation, the prominence of the stimuli, and the order in which the tasks are performed, frequently using the predictions produced by machine learning.

Open multi-session and multi-task EEG cognitive Dataset for passive brain-computer Interface Applications

Brain-Computer Interfaces and especially passive Brain-Computer interfaces (pBCI), with their ability to estimate and monitor user mental states, are receiving increasing attention from both the fundamental research and the applied research and development communities. Testing new pipelines and benchmarking classifiers and feature extraction algorithms is central to further research within this domain. Unfortunately, data sharing in pBCI research is still scarce. The COG-BCI database encompasses the recordings of 29 participants over 3 separate sessions with 4 different tasks (MATB, N-Back, PVT, Flanker) designed to elicit different mental states, for a total of over 100 hours of open EEG data. This dataset was validated on a subjective, behavioral and physiological level, to ensure its usefulness to the pBCI community. Furthermore, a proof of concept is given with an example of mental workload estimation pipeline and results, to ensure that the data can be used for the design and evaluation of pBCI pipelines. This body of work presents a large effort to promote the use of pBCIs in an open science framework.

The present and future of neural interfaces.

The 2020's decade will likely witness an unprecedented development and deployment of neurotechnologies for human rehabilitation, personalized use, and cognitive or other enhancement. New materials and algorithms are already enabling active brain monitoring and are allowing the development of biohybrid and neuromorphic systems that can adapt to the brain. Novel brain-computer interfaces (BCIs) have been proposed to tackle a variety of enhancement and therapeutic challenges, from improving decision-making to modulating mood disorders. While these BCIs have generally been developed in an open-loop modality to optimize their internal neural decoders, this decade will increasingly witness their validation in closed-loop systems that are able to continuously adapt to the user's mental states. Therefore, a proactive ethical approach is needed to ensure that these new technological developments go hand in hand with the development of a sound ethical framework. In this perspective article, we summarize recent developments in neural interfaces, ranging from neurohybrid synapses to closed-loop BCIs, and thereby identify the most promising macro-trends in BCI research, such as simulating vs. interfacing the brain, brain recording vs. brain stimulation, and hardware vs. software technology. Particular attention is devoted to central nervous system interfaces, especially those with application in healthcare and human enhancement. Finally, we critically assess the possible futures of neural interfacing and analyze the short- and long-term implications of such neurotechnologies.

Retrospective on the First Passive Brain-Computer Interface Competition on Cross-Session Workload Estimation.

As is the case in several research domains, data sharing is still scarce in the field of Brain-Computer Interfaces (BCI), and particularly in that of passive BCIs-i.e., systems that enable implicit interaction or task adaptation based on a user's mental state(s) estimated from brain measures. Moreover, research in this field is currently hindered by a major challenge, which is tackling brain signal variability such as cross-session variability. Hence, with a view to develop good research practices in this field and to enable the whole community to join forces in working on cross-session estimation, we created the first passive brain-computer interface competition on cross-session workload estimation. This competition was part of the 3rd International Neuroergonomics conference. The data were electroencephalographic recordings acquired from 15 volunteers (6 females; average 25 y.o.) who performed 3 sessions-separated by 7 days-of the Multi-Attribute Task Battery-II (MATB-II) with 3 levels of difficulty per session (pseudo-randomized order). The data -training and testing sets-were made publicly available on Zenodo along with Matlab and Python toy code (https://doi.org/10.5281/zenodo.5055046). To this day, the database was downloaded more than 900 times (unique downloads of all version on the 10th of December 2021: 911). Eleven teams from 3 continents (31 participants) submitted their work. The best achieving processing pipelines included a Riemannian geometry-based method. Although better than the adjusted chance level (38% with an α at 0.05 for a 3-class classification problem), the results still remained under 60% of accuracy. These results clearly underline the real challenge that is cross-session estimation. Moreover, they confirmed once more the robustness and effectiveness of Riemannian methods for BCI. On the contrary, chance level results were obtained by one third of the methods-4 teams- based on Deep Learning. These methods have not demonstrated superior results in this contest compared to traditional methods, which may be due to severe overfitting. Yet this competition is the first step toward a joint effort to tackle BCI variability and to promote good research practices including reproducibility.

An embedded LSTM based scheme for depression detection and analysis

Depression also known as depressive disorder is a serious and common medical condition that has an adverse impact on how you feel, think, and behave. But it can also be treated. Depression causes feelings of sadness or a loss of interest in activities once you enjoyed. It can impair your ability to perform at work and at home and cause a number of mental and physical issues. Therefore, it is important to provide some solutions to overcome the depression problem. These days’ information and communications technology (ICT) is used in various domains to sort out their problems. In this paper, we propose an embedded long short-term memory (LSTM) based scheme for depression detection and analysis. It is a brain health mapping based system for the detection of depression. It is a first of its kind interdisciplinary brain cognition tool (software), which acts as a personalized support intelligence mechanism. It incorporates psychological traits for determining a user's mental health state. The proposed scheme utilizes natural language processing techniques to provide a novel framework to predict the sentimental and emotional state of the user based on the user's behavior in their interaction with the designed model. In order to actively track the user's mental state, proposed scheme analyses their text and identify specific keywords through the text that they input in different applications that the user utilizes in their machine. The practical implementation of the proposed scheme is provided and important results (i.e., accuracy and F1 score) are estimated. During the performance comparison, it has been observed that proposed scheme achieved better accuracy than the other existing schemes.

Machine Learning Based Psychotic Behaviors Prediction from Facebook Status Updates

With the advent of technological advancements and the widespread Internet connectivity during the last couple of decades, social media platforms (such as Facebook, Twitter, and Instagram) have consumed a large proportion of time in our daily lives. People tend to stay alive on their social media with recent updates, as it has become the primary source of interaction within social circles. Although social media platforms offer several remarkable features but are simultaneously prone to various critical vulnerabilities. Recent studies have revealed a strong correlation between the usage of social media and associated mental health issues consequently leading to depression, anxiety, suicide commitment, and mental disorder, particularly in the young adults who have excessively spent time on social media which necessitates a thorough psychological analysis of all these platforms. This study aims to exploit machine learning techniques for the classification of psychotic issues based on Facebook status updates. In this paper, we start with depression detection in the first instance and then expand on analyzing six other psychotic issues (e.g., depression, anxiety, psychopathic deviate, hypochondria, unrealistic, and hypomania) commonly found in adults due to extreme use of social media networks. To classify the psychotic issues with the user's mental state, we have employed different Machine Learning (ML) classifiers i.e., Random Forest (RF), Support Vector Machine (SVM), Naïve Bayes (NB), and K-Nearest Neighbor (KNN). The used ML models are trained and tested by using different combinations of features selection techniques. To observe the most suitable classifiers for psychotic issue classification, a cost-benefit function (sometimes termed as ‘Suitability’) has been used which combines the accuracy of the model with its execution time. The experimental evidence argues that RF outperforms its competitor classifiers with the unigram feature set.

Mental Effort Estimation by Passive BCI: A Cross-Subject Analysis.

Despite the technological advancements, the employment of passive brain computer interface (BCI) out of the laboratory context is still challenging. This is largely due to methodological reasons. On the one hand, machine learning methods have shown their potential in maximizing performance for user mental states classification. On the other hand, the issues related to the necessary and frequent calibration of algorithms and to the temporal resolution of the measurement (i.e. how long it takes to have a reliable state measure) are still unsolved. This work explores the performances of a passive BCI system for mental effort monitoring consisting of three frontal electroencephalographic (EEG) channels. In particular, three calibration approaches have been tested: an intra-subject approach, a cross-subject approach, and a free-calibration procedure based on the simple average of theta activity over the three employed channels. A Random Forest model has been employed in the first two cases. The results obtained during multi-tasking have shown that the cross-subject approach allows the classification of low and high mental effort with an AUC higher than 0.9, with a related time resolution of 45 seconds. Moreover, these performances are not significantly different from the intra-subject approach although they are significantly higher than the calibration-free approach. In conclusion, these results suggest that a light (three EEG channels) passive BCI system based on a Random Forest algorithm and cross-subject calibration could be a simple and reliable tool for out-of-the-lab employment.

Interface, interaction, and intelligence in generalized brain–computer interfaces

A brain-computer interface (BCI) establishes a direct communication channel between a brain and an external device. With recent advances in neurotechnology and artificial intelligence (AI), the brain signals in BCI communication have been advanced from sensation and perception to higher-level cognition activities. While the field of BCI has grown rapidly in the past decades, the core technologies and innovative ideas behind seemingly unrelated BCI systems have never been summarized from an evolutionary point of view. Here, we review various BCI paradigms and present an evolutionary model of generalized BCI technology which comprises three stages: interface, interaction, and intelligence (I3). We also highlight challenges, opportunities, and future perspectives in the development of new BCI technology.

Adaptive Virtual Reality Design Methodology

The article provides an integrated analysis of practical application of virtual reality (VR) in healthcare. Modern VR products have been created for the diagnosis and treatment of post-traumatic syndrome, Alzheimer's disease, alcohol dependence, phobic disorders, and teenage depression. Information on such products is limited to simple scenes, without adaptation to the user. The article shows a prospect for creating a virtual system with biological feedback. The proposed system is a set of portable and stationary devices for recording physiological signals and immersion into a virtual reality environment, as well as smart nodes for registration, processing, and decision-making to correct the user's mental state. Technologies for wire/wireless transfer of audio-visual and parametric information in virtual reality systems are discussed. The analyses include the features (advantages and disadvantages) of using virtual reality in conditions of optical information transfer, wireless protocols, and wire interfaces that provide user communication with virtual reality system and technical solutions.

A big data analytics framework for detecting user-level depression from social networks

Depression is one of the most common mental health problems worldwide. The diagnosis of depression is usually done by clinicians based on mental status questionnaires and patient's self-reporting. Not only do these methods highly depend on the current mood of the patient, but also people who experience mental illness are often reluctantly seeking help. Social networks have become a popular platform for people to express their feelings and thoughts with friends and family. With the substantial amount of data in social networks, there is an opportunity to try designing novel frameworks to identify those at risk of depression. Moreover, such frameworks can provide clinicians and hospitals with deeper insights about depressive behavioral patterns, thereby improving diagnostic process. In this paper, we propose a big data analytics framework to detect depression for users of social networks. In addition to syntactic and syntax features, it focuses on pragmatic features toward modeling the intention of users. User intention represents the true motivation behind social network behaviors. Moreover, since the behaviors of user's friends in the network are believed to have an influence on the user, the framework also models the influence of friends on the user's mental states. We evaluate the performance of the proposed framework on a massive real dataset obtained from Facebook and show that the framework outperforms existing methods for diagnosing user-level depression in social networks.

Classification of Movement Preparation Between Attended and Distracted Self-Paced Motor Tasks.

Brain-computer interface (BCI) systems aim to control external devices by using brain signals. The performance of these systems is influenced by the user's mental state, such as attention. In this study, we classified two attention states to a target task (attended and distracted task level) while attention to the task is altered by one of three types of distractors. A total of 27 participants were allocated into three experimental groups and exposed to one type of distractor. An attended condition that was the same across the three groups comprised only the main task execution (self-paced dorsiflexion) while the distracted condition was concurrent execution of the main task and an oddball task (dual-task condition). Electroencephalography signals were recorded from 28 electrodes to classify the two attention states of attended or distracted task conditions by extracting temporal and spectral features. The results showed that the ensemble classification accuracy using the combination of temporal and spectral features (spectro-temporal features, 82.3 ± 2.7%) was greater than using temporal (69 ± 2.2%) and spectral (80.3 ± 2.6%) features separately. The classification accuracy was computed using a combination of different channel locations, and it was demonstrated that a combination of parietal and centrally located channels was superior for classification of two attention states during movement preparation (parietal channels: 84.6 ± 1.3%, central and parietal channels: 87.2 ± 1.5%). It is possible to monitor the users' attention to the task for different types of distractors. It has implications for online BCI systems where the requirement is for high accuracy of intention detection.

Real-time Mental State Recognition using a Wearable EEG.

The increasing quality and availability of low-cost EEG systems offer new possibilities for non-medical purposes. Existing openly available algorithms to assess the user's mental state in real-time have been mainly performed with medical-grade equipment. In this paper, an approach to assess the user's Focus or Relax states in real-time using a consumer-grade, wearable EEG headband is evaluated. One naive measure and four entropy-based measures, computed using relative frequency band powers in the EEG signal, were introduced. Classifiers for relax and focus state detection, based on the estimation of probability distributions, were developed and evaluated in a user study. Results showed that the Tsallis entropy-based measure performed best for the Focus score, whereas the Renyi measure performed best for the Relax score. Sensitivities of 82.0% and 80.4% with specificities of 82.8% and 80.8% were achieved for the Focus and Relax scores, respectively. The results demonstrated the possibilities of using a wearable EEG system for real-time mental state recognition.

Mental effort detection using EEG data in E-learning contexts

E-learning becomes an alternative learning mode since the prevalence of the Internet. Especially, the advance of MOOC (Massive Open Online Course) technology enables a course to enroll tens of thousands of online learners. How to improve learners' online learning experiences on MOOC platforms becomes a crucial task for platform providers. In this research, based on Cognitive Load Theory, we built a system to capture and tag a user's mental states while s/he is watching online videos with a commercial EEG device, and used different normalization schemes and time window lengths to process EEG signals recorded from the EEG device. Finally, we adopted different supervised learning algorithms to train and test the classifiers, and then evaluated their classification performance. The results show that the proposed approach can effectively process EEG data to train classifiers, which achieve high accuracy, precision and recall rates compared with those of previous studies. This system can effectively facilitate users' self-awareness of mental efforts in online learning contexts to enable the automatic feedback in synchronous and asynchronous learning contexts, especially taking MOOCs as an example.

Using EEG for Predicting User Preferences of Physical Compliance in Human-Robot Cooperation

Using EEG for Predicting User Preferences of Physical Compliance in Human-Robot Cooperation

Passive BCI in Operational Environments: Insights, Recent Advances, and Future Trends.

This minireview aims to highlight recent important aspects to consider and evaluate when passive brain-computer interface (pBCI) systems would be developed and used in operational environments, and remarks future directions of their applications. Electroencephalography (EEG) based pBCI has become an important tool for real-time analysis of brain activity since it could potentially provide covertly-without distracting the user from the main task-and objectively-not affected by the subjective judgment of an observer or the user itself-information about the operator cognitive state. Different examples of pBCI applications in operational environments and new adaptive interface solutions have been presented and described. In addition, a general overview regarding the correct use of machine learning techniques (e.g., which algorithm to use, common pitfalls to avoid, etc.) in the pBCI field has been provided. Despite recent innovations on algorithms and neurotechnology, pBCI systems are not completely ready to enter the market yet, mainly due to limitations of the EEG electrodes technology, and algorithms reliability and capability in real settings. High complexity and safety critical systems (e.g., airplanes, ATM interfaces) should adapt their behaviors and functionality accordingly to the user' actual mental state. Thus, technologies (i.e., pBCIs) able to measure in real time the user's mental states would result very useful in such "high risk" environments to enhance human machine interaction, and so increase the overall safety.