Word Embedding Approach Research Articles

ESG practices through the lens of employees in hospitality: Insights from employee-generated data

Environmental, social, and governance (ESG) practices are garnering increased attention from hospitality companies, accompanied by a notable rise in awareness and expectations from key stakeholders. Nevertheless, research exploring how internal stakeholders perceive ESG initiatives and how such perceptions affect them is still scarce. This paper attempts to shed light on this area by analyzing 195,530 employee online reviews from 4182 U.S. hotel companies over 15 years. We measure employees’ perceptions of ESG practices by constructing a novel ESG lexicon using a word-embedding approach. Our results reveal that (a) employees’ perceived ESG practices increase gradually over the years, (b) perceived ESG practices are higher among current (vs. former) employees and managers (vs. non-managers), and increase with organizational tenure, (c) perceived ESG practices are related to higher job satisfaction and lower employee turnover, with organizational tenure and position level moderating these relationships.

Identifying contextual content-based risk drivers for advanced risk management strategies

This research proposes a profound contextual topic identifier that incorporates topic modelling and a word embedding technique to discover and quantify corporate risks from its self-identified risk disclosures and examines the association between each risk type and operating performance via artificial intelligence (AI) technique. Via topic modelling adoption, we are able to discover the most essential risks confronted by corporates in the near future and evaluate how they respond to these risks. To gain deeper insight, the study performs a bidirectional encoder representation from transformers (BERT) (one type of word embedding approach) to extract and quantify the semantic features embedded into each risk disclosure. The results show that operating performance significantly and positively relates to corporate-specific risks. This study offers solid and direct support for authorities that set accounting principles to encourage firm managers to add a new section on risk factors in annual reports.

Perceived Impact of COVID-19 in an Underserved Community: A Natural Language Processing Approach.

To utilise natural language processing (NLP) to analyse interviews about the impact of COVID-19 in underserved communities and to compare it to traditional thematic analysis in a small subset of interviews. NLP and thematic analysis were used together to comprehensively examine the interview data. Fifty transcribed interviews with purposively sampled adults living in underserved communities in the United States, conducted from June 2021 to May 2022, were analysed to explore the impact of the COVID-19 pandemic on social activities, mental and emotional stress and physical and spiritual well-being. NLP includes several stages: data extraction, preprocessing, processing using word embeddings and topic modelling and visualisation. This was compared to thematic analysis in a random sample of 10 interviews. Six themes emerged from thematic analysis: The New Normal, Juxtaposition of Emotions, Ripple Effects on Health, Brutal yet Elusive Reality, Evolving Connections and Journey of Spirituality and Self-Realisation. With NLP, four clusters of similar context words for each approach were analysed visually and numerically. The frequency-based word embedding approach was most interpretable and well aligned with the thematic analysis. The NLP results complemented the thematic analysis and offered new insights regarding the passage of time, the interconnectedness of impacts and the semantic connections among words. This research highlights the interdependence of pandemic impacts, simultaneously positive and negative effects and deeply individual COVID-19 experiences in underserved communities. The iterative integration of NLP and thematic analysis was efficient and effective, facilitating the analysis of many transcripts and expanding nursing research methodology. While thematic analysis provided richer, more detailed themes, NLP captured new elements and combinations of words, making it a promising tool in qualitative analysis. Not applicable. No patient or public contribution.

A Hybrid Deep BiLSTM-CNN for Hate Speech Detection in Multi-social media

Nowadays, means of communication among people have changed due to advancements in information technology and the rise of online multi-social media. Many people express their feelings, ideas, and emotions on social media sites such as Instagram, Twitter, Gab, Reddit, Facebook, and YouTube. However, people have misused social media to send hateful messages to specific individuals or groups to create chaos. For various governance authorities, manually identifying hate speech on various social media platforms is a difficult task to avoid such chaos. In this study, a hybrid deep-learning model, where bidirectional long short-term memory (BiLSTM) and convolutional neural network (CNN) are used to classify hate speech in textual data, is proposed. This model incorporates a GLOVE-based word embedding approach, dropout, L2 regularization, and global max pooling to get impressive results. Further, the proposed BiLSTM-CNN model has been evaluated on various datasets to achieve state-of-the-art performance that is superior to the traditional and existing machine learning methods in terms of accuracy, precision, recall, and F1-score.

Cognitive Hybrid Deep Learning-based Multi-modal Sentiment Analysis for Online Product Reviews

Recently the field of sentiment analysis has gained a lot of attraction in literature. The idea that a machine can dynamically spot the text’s sentiments is fascinating. In this paper, we propose a method to classify the textual sentiments in Twitter feeds. In particular, we focus on analyzing the tweets of products as either positive or negative. The proposed technique utilizes a deep learning schema to learn and predict the sentiment by extracting features directly from the text. Specifically, we use Convolutional Neural Networks with different convolutional layers. Further, we experiment with LSTMs and try an ensemble of multiple models to get the best results. We employ an n-gram-based word embeddings approach to get the machine-level word representations. Testing of the method is conducted on real-world datasets. We have discovered that the ensemble technique yields the best results after conducting experiments on a huge corpus of more than one million tweets. To be specific, we get an accuracy of 84.95%. The proposed method is also compared with several existing methods. An extensive numerical investigation has revealed the superiority of the proposed work in actual deployment scenarios.

Stereoisomers Are Not Machine Learning's Best Friends.

This study addresses the challenge of accurately identifying stereoisomers in cheminformatics, which originates from our objective to apply machine learning to predict the association constant between cyclodextrin and a guest. Identifying stereoisomers is indeed crucial for machine learning applications. Current tools offer various molecular descriptors, including their textual representation as Isomeric SMILES that can distinguish stereoisomers. However, such representation is text-based and does not have a fixed size, so a conversion is needed to make it usable to machine learning approaches. Word embedding techniques can be used to solve this problem. Mol2vec, a word embedding approach for molecules, offers such a conversion. Unfortunately, it cannot distinguish between stereoisomers due to its inability to capture the spatial configuration of molecular structures. This study proposes several approaches that use word embedding techniques to handle molecular discrimination using stereochemical information on molecules or considering Isomeric SMILES notation as a text in Natural Language Processing. Our aim is to generate a distinct vector for each unique molecule, correctly identifying stereoisomer information in cheminformatics. The proposed approaches are then compared to our original machine learning task: predicting the association constant between cyclodextrin and a guest molecule.

BiSAMNet: A Novel Approach in Maritime Data Completion Using Deep Learning and NLP Techniques

In the extensive monitoring of maritime traffic, maritime management frequently encounters incomplete automatic identification system (AIS) data. This deficiency poses significant challenges to safety management, requiring effective methods to infer corresponding ship information. We tackle this issue using a classification approach. Due to the absence of a fixed road network at sea unlike on land, raw trajectories are difficult to convert and cannot be directly fed into neural networks. We devised a latitude–longitude gridding encoding strategy capable of transforming continuous latitude–longitude data into discrete grid points. Simultaneously, we employed a compression algorithm to further extract significant grid points, thereby shortening the encoding sequence. Utilizing natural language processing techniques, we integrate the Word2vec word embedding approach with our novel biLSTM self-attention chunk-max pooling net (biSAMNet) model, enhancing the classification of vessel trajectories. This method classifies targets into ship types and ship lengths within static information. Employing the Taiwan Strait as a case study and benchmarking against CNN, RNN, and methods based on the attention mechanism, our findings underscore our model’s superiority. The biSAMNet achieves an impressive trajectory classification F1 score of 0.94 in the ship category dataset using only five-dimensional word embeddings. Additionally, through ablation experiments, the effectiveness of the Word2vec pre-trained embedding layer is highlighted. This study introduces a novel method for handling ship trajectory data, addressing the challenge of obtaining ship static information when AIS data are unreliable.

Exploration of Word Embeddings with Graph-Based Context Adaptation for Enhanced Word Vectors

In the aspect of information storage, text assumes a central role, necessitating streamlined and effective methods for swift retrieval. Among various text representations, the vector form stands out for its remarkable efficiency, especially when dealing with expansive datasets. Arranging words that are similar in meaning close to each other in the vectorized representation helps improve how well the system performs in different Natural Language Processing related tasks. Previous methods, primarily centered on capturing word context through neural language models, have fallen short in delivering high scores for word similarity problems. This paper investigates the connection between representing words in vector form and the improved performance and accuracy observed in Natural Language Processing tasks. It introduces a method to represent words as a graph, aiming to preserve their inherent relationships and to enhance overall capabilities in semantic representation. Experimental deployment of this technique across diverse text corpora underscores its superiority over conventional word embedding approaches. The findings contribute to the evolving landscape of semantic representation learning but also illuminates their implications for text classification tasks, especially within the context of dynamic embedding models.

The Effects of Sentiment Evolution in Financial Texts: A Word Embedding Approach

ABSTRACT We examine the evolutionary effects of sentiment words in financial text and their implications for various business outcomes. We propose an algorithm called Word List Vector for Sentiment (WOLVES) that leverages both a human-defined sentiment word list and the word embedding approach to quantify text sentiment over time. We then apply WOLVES to investigate the evolutionary effects of the most popular financial word list, Loughran and McDonald (LM) dictionary, in annual reports, conference calls, and financial news. We find that LM negative words become less negative over time in annual reports compared to conference calls and financial news, while LM positive words remain qualitatively unchanged. This finding reconciles with existing evidence that negative words are more subject to managers’ strategic communication. We also provide practical implications of WOLVES by correlating the sentiment evolution of LM negative words in annual reports with market reaction, earnings performance, and accounting fraud.

Deep Aspect-Sentinet: Aspect Based Emotional Sentiment Analysis Using Hybrid Attention Deep Learning Assisted BILSTM

Data mining and natural language processing researchers have been working on sentiment analysis for the past decade. Using deep neural networks (DNNs) for sentiment analysis has recently shown promising results. A technique of studying people’s attitudes through emotional sentiment analysis of data generated from various sources such as Twitter, social media reviews, etc. and classifying emotions based on the given data is related to text data generation. Therefore, the proposed study proposes a well-known deep learning technique for facet-based emotional mood classification using text data that can handle a large amount of content. Text data pre-processing uses stemming, segmentation, tokenization, case folding, and removal of stop words, nulls, and special characters. After data pre-processing, three word embedding approaches such as Assimilated N-gram Approach (ANA), Boosted Term Frequency Inverse Document Frequency (BT-IDF) and Enhanced Two-Way Encoder Representation from Transformers (E-BERT) are used to extract relevant features. The extracted features from the three different approaches are concatenated using the Feature Fusion Approach (FFA). The optimal features are selected using the Intensified Hunger Games Search Optimization (I-HGSO) algorithm. Finally, aspect-based sentiment analysis is performed using the Senti-BILSTM (Deep Aspect-EMO SentiNet) autoencoder based on the Hybrid Emotional Aspect Capsule autoencoder. The experiment was built on the yelp reviews dataset, IDMB movie review dataset, Amazon reviews dataset and the Twitter sentiment dataset. A statistical evaluation and comparison of the experimental results are conducted with respect to the accuracy, precision, specificity, the f1-score, recall, and sensitivity. There is a 99.26% accuracy value in the Yelp reviews dataset, a 99.46% accuracy value in the IMDB movie reviews dataset, a 99.26% accuracy value in the Amazon reviews dataset and a 99.93% accuracy value in the Twitter sentiment dataset.

An ensemble approach for imbalanced multiclass malware classification using 1D-CNN

Dependence on the internet and computer programs demonstrates the significance of computer programs in our day-to-day lives. Such demands motivate malware developers to create more malware, both in terms of quantity and variety. Researchers are constantly faced with hurdles while attempting to protect themselves from potential hazards and risks due to malware authors’ usage of code obfuscation techniques. Metamorphic and polymorphic variations are easily able to elude the widely utilized signature-based detection procedures. Researchers are more interested in deep learning approaches than machine learning techniques to analyze the behavior of such a vast number of virus variants. Researchers have been drawn to the categorization of malware within itself in addition to the classification of malware against benign programs to examine the behavioral differences between them. In order to investigate the relationship between the application programming interface (API) calls throughout API sequences and classify them, this work uses the one-dimensional convolutional neural network (1D-CNN) model to solve a multiclass classification problem. On API sequences, feature vectors for distinctive APIs are created using the Word2Vec word embedding approach and the skip-gram model. The one-vs.-rest approach is used to train 1D-CNN models to categorize malware, and all of them are then combined with a suggested ModifiedSoftVoting algorithm to improve classification. On the open benchmark dataset Mal-API-2019, the suggested ensembled 1D-CNN architecture captures improved evaluation scores with an accuracy of 0.90, a weighted average F1-score of 0.90, and an AUC score of more than 0.96 for all classes of malware.

Automatic assignment of microgenres to movies using a word embedding-based approach

Streaming services are increasingly leveraging Artificial Intelligence (AI) technologies for improved content cataloging, user experiences in content discovery, and personalization. A significant challenge in this domain is the automated assignment of microgenres to movies. This study introduces and evaluates approaches based on clustering, topic modeling, and word embedding to address this task. The evaluation employs a preprocessed dataset containing movie-related data—title tags, synopses, genres, and reviews—alongside a predefined microgenre list. Comparisons of three activation functions (binary step, ramp, and sigmoid) gauge their effectiveness in augmenting microgenre tags. Results demonstrate the superiority of the word embedding approach over clustering and topic modeling in terms of mean accuracy. Even more, the word embedding approach stands as the sole fully automated solution. Analysis indicates that incorporating review-based tags introduces noise and undermines accuracy. Besides, the word embedding approach yields optimal outcomes using the sigmoid function, effectively doubling assigned tags while maintaining matching quality. This sheds light on the potential of word embedding methods within the movie domain.

Topic Modeling based Text Classification Regarding Islamophobia using Word Embedding and Transformers Techniques

Islamophobia is a rising area of concern in the current era where Muslims face discrimination and receive negative perspectives towards their religion, Islam. Islamophobia is a type of racism that is being practiced by individuals, groups, and organizations worldwide. Moreover, the ease of access to social media platforms and their augmented usage has also contributed to spreading hate speech, false information, and negative opinions about Islam. In this research study, we focused to detect Islamophobic textual content shared on various social media platforms. We explored the state-of-the-art techniques being followed in text data mining and Natural Language Processing (NLP). Topic modelling algorithm Latent Dirichlet Allocation is used to find top topics. Then, word embedding approaches such as Word2Vec and Global Vectors for word representation (GloVe) are used as feature extraction techniques. For text classification, we utilized modern text analysis techniques of transformers-based Deep Learning algorithms named Bidirectional Encoders Representation from Transformers (BERT) and Generative Pre-Trained Transformer (GPT). For results comparison, we conducted an extensive empirical analysis of Machine Learning algorithms and Deep Learning using conventional textual features such as the Term Frequency-Inverse Document Frequency, N-gram, and Bag of words (BoW). The empirical based results evaluated using standard performance evaluation measures show that the proposed approach effectively detects the textual content related to Islamophobia. In the corpus of the study under Machine Learning models Support Vector Machine (SVM) performed best with an F1 score of 91%. The Transformer based core NLP models and the Deep Learning model Convolutional Neural Network (CNN) when combined with GloVe performed best among all the techniques except SVM with BoW. GPT, SVM when combined with BoW and BERT yielded the best F1 score of 92%, 92% and 91.9% respectively, while CNN performed slightly poor with an F1 score of 91%.

Comparison of Pre-trained vs Custom-trained Word Embedding Models for Word Sense Disambiguation

The prime objective of word sense disambiguation (WSD) is to develop such machines that can automatically recognize the actual meaning (sense) of ambiguous words in a sentence. WSD can improve various NLP and HCI challenges. Researchers explored a wide variety of methods to resolve this issue of sense ambiguity. However, majorly, their focus was on English and some other well-reputed languages. Urdu with more than 300 million users and a large amount of electronic text available on the web is still unexplored. In recent years, for a variety of Natural Language Processing tasks, word embedding methods have proven extremely successful. This study evaluates, compares, and applies a variety of word embedding approaches to Urdu Word embedding (both Lexical Sample and All-Words), including pre-trained (Word2Vec, Glove, and FastText) as well as custom-trained (Word2Vec, Glove, and FastText trained on the Ur-Mono corpus). Two benchmark corpora are used for the evaluation in this study: (1) the UAW-WSD-18 corpus and (2) the ULS-WSD-18 corpus. For Urdu All-Words WSD tasks, top results have been achieved (Accuracy=60.07 and F1=0.45) using pre-trained FastText. For the Lexical Sample, WSD has been achieved (Accuracy=70.93 and F1=0.60) using custom-trained GloVe word embedding method.

Measuring Political Narratives in African News Media: A Word Embeddings Approach

Measuring Political Narratives in African News Media: A Word Embeddings Approach

Incorporating Multiple Textual Factors into Unbalanced Financial Distress Prediction: A Feature Selection Methods and Ensemble Classifiers Combined Approach

Textual-based factors have been widely regarded as a promising feature that can be applied to financial issues. This study focuses on extracting both basic and semantic textual features to supplement the traditionally used financial indicators. The main is to improve Chinese listed companies’ financial distress prediction (FDP). A unique paradigm is proposed in this study that combines financial and multi-type textual predictive factors, feature selection methods, classifiers, and time spans to achieve the optimal FDP. The frequency counts, TF-IDF, TextRank, and word embedding approaches are employed to extract frequency count-based, keyword-based, sentiment, and readability indicators. The experimental results prove that financial domain sentiment lexicons, word embedding-based readability analysis approaches, and the basic textual features of Management Discussion and Analysis can be important elements of FDP. Moreover, the finding highlights the fact that incorporating financial and textual features can achieve optimal performance 4 or 5 years before the expected baseline year; applying the RF-GBDT combined model can also outperform other classifiers. This study makes an innovative contribution, since it expands the multiple text analysis method in the financial text mining field and provides new findings on how to provide early warning signs related to financial risk. The approaches developed in this research can serve as a template that can be used to resolve other financial issues.

The SPEAK study rationale and design: A linguistic corpus-based approach to understanding thought disorder.

Psychotic symptoms are typically measured using clinical ratings, but more objective and sensitive metrics are needed. Hence, we will assess thought disorder using the Research Domain Criteria (RDoC) heuristic for language production, and its recommended paradigm of "linguistic corpus-based analyses of language output". Positive thought disorder (e.g., tangentiality and derailment) can be assessed using word-embedding approaches that assess semantic coherence, whereas negative thought disorder (e.g., concreteness, poverty of speech) can be assessed using part-of-speech (POS) tagging to assess syntactic complexity. We aim to establish convergent validity of automated linguistic metrics with clinical ratings, assess normative demographic variance, determine cognitive and functional correlates, and replicate their predictive power for psychosis transition among at-risk youths. This study will assess language production in 450 English-speaking individuals in Australia and Canada, who have recent onset psychosis, are at clinical high risk (CHR) for psychosis, or who are healthy volunteers, all well-characterized for cognition, function and symptoms. Speech will be elicited using open-ended interviews. Audio files will be transcribed and preprocessed for automated natural language processing (NLP) analyses of coherence and complexity. Data analyses include canonical correlation, multivariate linear regression with regularization, and machine-learning classification of group status and psychosis outcome. This prospective study aims to characterize language disturbance across stages of psychosis using computational approaches, including psychometric properties, normative variance and clinical correlates, important for biomarker development. SPEAK will create a large archive of language data available to other investigators, a rich resource for the field.

A Bi-Directional GRU Architecture for the Self-Attention Mechanism: An Adaptable, Multi-Layered Approach with Blend of Word Embedding

Sentiment analysis (SA) has become an essential component of natural language processing (NLP) with numerous practical applications to understanding “what other people think”. Various techniques have been developed to tackle SA using deep learning (DL); however, current research lacks comprehensive strategies incorporating multiple-word embeddings. This study proposes a self-attention mechanism that leverages DL and involves the contextual integration of word embedding with a time-dispersed bidirectional gated recurrent unit (Bi-GRU). This work employs word embedding approaches GloVe, word2vec, and fastText to achieve better predictive capabilities. By integrating these techniques, the study aims to improve the classifier’s capability to precisely analyze and categorize sentiments in textual data from the domain of movies. The investigation seeks to enhance the classifier’s performance in NLP tasks by addressing the challenges of underfitting and overfitting in DL. To evaluate the model’s effectiveness, an openly available IMDb dataset was utilized, achieving a remarkable testing accuracy of 99.70%.

Animal disease surveillance: How to represent textual data for classifying epidemiological information.

The value of informal sources in increasing the timeliness of disease outbreak detection and providing detailed epidemiological information in the early warning and preparedness context is recognized. This study evaluates machine learning methods for classifying information from animal disease-related news at a fine-grained level (i.e., epidemiological topic). We compare two textual representations, the bag-of-words method and a distributional approach, i.e., word embeddings. Both representations performed well for binary relevance classification (F-measure of 0.839 and 0.871, respectively). Bag-of-words representation was outperformed by word embedding representation for classifying sentences into fine-grained epidemiological topics (F-measure of 0.745). Our results suggest that the word embedding approach is of interest in the context of low-frequency classes in a specialized domain. However, this representation did not bring significant performance improvements for binary relevance classification, indicating that the textual representation should be adapted to each classification task.

Data-Driven Solution to Identify Sentiments from Online Drug Reviews

With the proliferation of the internet, social networking sites have become a primary source of user-generated content, including vast amounts of information about medications, diagnoses, treatments, and disorders. Comments on previously used medicines, contained within these data, can be leveraged to identify crucial adverse drug reactions, and machine learning (ML) approaches such as sentiment analysis (SA) can be employed to derive valuable insights. However, given the sheer volume of comments, it is often impractical for consumers to manually review all of them before determining a purchase decision. Therefore, drug assessments can serve as a valuable source of medical information for both healthcare professionals and the general public, aiding in decision making and improving public monitoring systems by revealing collective experiences. Nonetheless, the unstructured and linguistic nature of the comments poses a significant challenge for effective categorization, with previous studies having utilized machine and deep learning (DL) algorithms to address this challenge. Despite both approaches showing promising results, DL classifiers outperformed ML classifiers in previous studies. Therefore, the objective of our study was to improve upon earlier research by applying SA to medication reviews and training five ML algorithms on two distinct feature extractions and four DL classifiers on two different word-embedding approaches to obtain higher categorization scores. Our findings indicated that the random forest trained on the count vectorizer outperformed all other ML algorithms, achieving an accuracy and F1 score of 96.65% and 96.42%, respectively. Furthermore, the bidirectional LSTM (Bi-LSTM) model trained on GloVe embedding resulted in an even better accuracy and F1 score, reaching 97.40% and 97.42%, respectively. Hence, by utilizing appropriate natural language processing and ML algorithms, we were able to achieve superior results compared to earlier studies.