Word Embedding Models Research Articles

Evaluating sentiment analysis and word embedding techniques on Brexit

<p>In this study, we investigate the effectiveness of pre-trained word embeddings for sentiment analysis on a real-world topic, namely Brexit. We compare the performance of several popular word embedding models such global vectors for word representation (GloVe), FastText, word to vec (word2vec), and embeddings from language models (ELMo) on a dataset of tweets related to Brexit and evaluate their ability to classify the sentiment of the tweets as positive, negative, or neutral. We find that pre-trained word embeddings provide useful features for sentiment analysis and can significantly improve the performance of machine learning models. We also discuss the challenges and limitations of applying these models to complex, real-world texts such as those related to Brexit.</p><p> </p>

A Machine learning approach for Post-Disaster data curation

Image data collected after natural disasters play an important role in the forensics of structure failures. However, curating and managing large amounts of post-disaster imagery data is challenging. In most cases, data users still have to spend much effort to find and sort images from the massive amounts of images archived for past decades in order to study specific types of disasters. This paper proposes a new machine learning based approach for automating the labeling and classification of large volumes of post-natural disaster image data to address this issue. More specifically, the proposed method couples pre-trained computer vision models and a natural language processing model with an ontology tailed to natural disasters to facilitate the search and query of specific types of image data. The resulting process returns each image with five primary labels and similarity scores, representing its content based on the developed word-embedding model. Validation and accuracy assessment of the proposed methodology was conducted with ground-level residential building panoramic images from Hurricane Harvey. The computed primary labels showed a minimum average difference of 13.32% when compared to manually assigned labels. This versatile and adaptable solution offers a practical and valuable solution for automating image labeling and classification tasks, with the potential to be applied to various image classifications and used in different fields and industries. The flexibility of the method means that it can be updated and improved to meet the evolving needs of various domains, making it a valuable asset for future research and development.

An efficient consolidation of word embedding and deep learning techniques for classifying anticancer peptides: FastText+BiLSTM.

Anticancer peptides (ACPs) are a group of peptides that exhibit antineoplastic properties. The utilization of ACPs in cancer prevention can present a viable substitute for conventional cancer therapeutics, as they possess a higher degree of selectivity and safety. Recent scientific advancements generate an interest in peptide-based therapies which offer the advantage of efficiently treating intended cells without negatively impacting normal cells. However, as the number of peptide sequences continues to increase rapidly, developing a reliable and precise prediction model becomes a challenging task. In this work, our motivation is to advance an efficient model for categorizing anticancer peptides employing the consolidation of word embedding and deep learning models. First, Word2Vec, GloVe, FastText, One-Hot-Encoding approaches are evaluated as embedding techniques for the purpose of extracting peptide sequences. Then, the output of embedding models are fed into deep learning approaches CNN, LSTM, BiLSTM. To demonstrate the contribution of proposed framework, extensive experiments are carried on widely-used datasets in the literature, ACPs250 and independent. Experiment results show the usage of proposed model enhances classification accuracy when compared to the state-of-the-art studies. The proposed combination, FastText+BiLSTM, exhibits 92.50% of accuracy for ACPs250 dataset, and 96.15% of accuracy for the Independent dataset, thence determining new state-of-the-art.

Evaluation of word embedding models used for diachronic semantic change analysis

In the last decade, the quantitative analysis of diachronic changes in language and lexical semantic changes have become the subject of active research. A significant role was played by the development of new effective techniques of word embedding. This direction has been effectively demonstrated in a number of studies. Some of them have focused on the analysis of the optimal type of word2vec models, hyperparameters for training, and evaluation techniques. In this research, we used Corpus of Historical American English (COHA). The paper demonstrates the results of multiple training runs and the comparison of word2vec models with different variations of hyperparameters used for lexical semantic change detection. In addition to traditional word similarities and analogical reasoning tests, we used testing on an extended set of synonyms. We have evaluated word2vec models on the set of more than 100,000 English synsets that were randomly selected from the WordNet database. We have shown that changing the word2vec model parameters (such as a dimension of word embedding, a size of context window, a type of model, a word discard rate etc.) can significantly impact on the resulting word embedding vector space and the detected lexical semantic changes. Additionally, the results strongly depended on properties of the corpus, such as word frequency distribution.

Understanding and Creating Word Embeddings

Word embeddings allow you to analyze the usage of different terms in a corpus of texts by capturing information about their contextual usage. Through a primarily theoretical lens, this lesson will teach you how to prepare a corpus and train a word embedding model. You will explore how word vectors work, how to interpret them, and how to answer humanities research questions using them.

MiTDS: Uncovering miRNA-mRNA interactions with deep learning for functional target prediction

MicroRNAs (miRNAs) are vital in regulating gene expression through binding to specific target sites on messenger RNAs (mRNAs), a process closely tied to cancer pathogenesis. Identifying miRNA functional targets is essential but challenging, due to incomplete genome annotation and an emphasis on known miRNA-mRNA interactions, restricting predictions of unknown ones. To address those challenges, we have developed a deep learning model based on miRNA functional target identification, named miTDS, to investigate miRNA–mRNA interactions. miTDS first employs a scoring mechanism to eliminate unstable sequence pairs and then utilizes a dynamic word embedding model based on the transformer architecture, enabling a comprehensive analysis of miRNA-mRNA interaction sites by harnessing the global contextual associations of each nucleotide. On this basis, miTDS fuses extended seed alignment representations learned in the multi-scale attention mechanism module with dynamic semantic representations extracted in the RNA-based dual-path module, which can further elucidate and predict miRNA and mRNA functions and interactions. To validate the effectiveness of miTDS, we conducted a thorough comparison with state-of-the-art miRNA-mRNA functional target prediction methods. The evaluation, performed on a dataset cross-referenced with entries from MirTarbase and Diana-TarBase, revealed that miTDS surpasses current methods in accurately predicting functional targets. In addition, our model exhibited proficiency in identifying A-to-I RNA editing sites, which represents an aberrant interaction that yields valuable insights into the suppression of cancerous processes.

A novel few-shot learning framework for rock images dually driven by data and knowledge

In the field of geosciences, the integration of artificial intelligence is transitioning from perceptual intelligence to cognitive intelligence. The simultaneous utilization of knowledge and data in the geoscience domain is a universally addressed concern. In this paper, based on the interpretability of deep learning models for rock images, rock features such as structure, texture, mineral and macroscopic identification characteristics were selected to extract a rock identification subgraph from the petrographic knowledge graph and carry out rock type similarity reasoning. Comparative experiments were conducted on few-shot learning of rock images under the supervision of rock type similarity knowledge. The results of the few-shot learning comparisons demonstrate that the supervision of rock type similarity knowledge significantly enhances performance. Additionally, rock type similarity knowledge exhibits a marginal effect on improving few-shot learning performance. Given the absence of Chinese word embedding and large-scale Chinese pre-trained language models in the geological domain, graph embedding based on domain-specific knowledge graphs in geosciences can offer computable geoscience knowledge for research dually propelled by data and knowledge.

A new word embedding model integrated with medical knowledge for deep learning-based sentiment classification

The development of intelligent systems that use social media data for decision-making processes in numerous domains such as politics, business, marketing, and finance, has been made possible by the popularity of social media platforms. However, the utilization of textual data from social media in the healthcare management industry is still somewhat limited when it is compared to other industries. Investigating how current machine learning and natural language processing technologies can be used in the healthcare industry to gauge public sentiment is an important study. Earlier works on healthcare sentiment analysis have utilized traditional word embedding models trained on the general and medical corpus. However, integration of medical knowledge to pre-trained word embedding models has not been considered yet. Word embedding models trained on the general corpus led to the problem of lacking medical knowledge and the models trained on the small size of the medical corpus have limitations in capturing semantic and syntactic properties. This research proposes a new word embedding model named Word Embedding Integrated with Medical Knowledge Vector (WE-iMKVec). The proposed model integrates sentiment lexicons and medical knowledgebases into the pre-trained word embedding to enrich the properties of word embedding. A new medical-aware sentiment polarity score is proposed for the utilization in learning neural-network sentiment and these vectors incorporate with the original pre-trained word vectors. The resulting vectors are enriched with lexicon vectors and the medical knowledge vectors: Adverse Drug Reaction (ADR) vector and Unified Medical Language System (UMLS) vector are used to build the proposed WE-iMKVec model. WE-iMKVec is validated on the five different social media healthcare review datasets and the empirical results showed its superiority over traditional word embedding models in medical sentiment analysis. The highest improvement can be found in the patients.info medical condition dataset where the proposed model outperforms three conventional word2vec models (Google-News, PubMed-PMC, and Drug Reviews) by 12.7 %, 31.4 %, and 25.4 % respectively in terms of F1 score.

An empirical assessment of different word embedding and deep learning models for bug assignment

Bug assignment, or bug triage, focuses on identifying the appropriate developers to repair newly discovered bugs, thereby managing them more effectively. Several deep learning-based approaches have been proposed for automated bug assignment. These approaches view automated bug assignment as a text classification task — the textual description of a bug report is utilized as the input and the potential fixers are regarded as the output labels. Such approaches typically depend on the classification performance of natural language processing and machine learning techniques. Various word embedding and deep learning models have emerged continuously. The effectiveness of those approaches depends on the chosen deep learning model, used for classification, and the word embedding model, used for representing bug reports. However, prior research does not empirically evaluate the impacts of various word embedding and deep learning models for automated bug assignment. In this paper, we conduct an empirical study to analyze the performance variations among 35 deep learning-based automated bug assignment approaches. These approaches are based on five word embedding techniques, i.e., Word2Vec, GloVe, NextBug, ELMo, and BERT, and seven text classification models, i.e., TextCNN, LSTM, Bi-LSTM, LSTM with attention, Bi-LSTM with attention, MLP, and Naive Bayes. We evaluated these combinations across three benchmark datasets, namely Eclipse JDT, GCC, and Firefox, and their mergence i.e., a cross-project dataset. Our main observations are: (1) Bi-LSTM with attention and Bi-LSTM using ELMo are significantly superior to other deep learning models on bug assignment tasks in terms of top-k (k = 1, 5, 10) accuracy and MRR; (2) Both the summary and description of bug reports are useful for bug assignment, but the description is more useful than the summary; (3) The training corpus for word embedding models has a significant impact on the performance of deep learning-based bug assignment methods. Our results show the importance of tuning different components (e.g. word embedding model, classification model, and textual input) in deep learning-based automated bug assignment methods and provide important insights for practitioners and researchers.

Research on Online Interactive Teaching Platform of College English Combined with Semantic Association Network Modeling

Abstract The emergence of semantic association networks has injected a new impetus for the development of online English teaching and provided a new model reference for the design of online education platforms. In this paper, the research and design of an online interactive teaching platform for college English draws on the algorithmic advantages of the semantic associative network model and utilizes the self-operation of the semantic associative network to realize the functions of autonomous addition, deletion, modification, and checking. The text semantic similarity is predicted by word embedding model, convolutional neural network, and other algorithms so as to better achieve the integration of teaching resources, connecting English knowledge and highlighting the teaching focus in the online teaching process of college English. Dynamic load balancing algorithms are used to solve the problems of short-term surges in the number of visits and the concentration of call requests, and the optimization of load balancing algorithms is further realized through genetic algorithms to finally complete the design of the online teaching interactive platform. Comparison experiments concluded that the semantic association network proposed in this paper could hold a more stable repair effect when cleaning inconsistent data in the dataset, highlighting the effectiveness of the semantic association network model in this paper. The online interactive teaching platform designed in this paper also performs well in the performance test, with only a 0.01% abnormality rate in the concurrency performance test, and the load balancing ability test also achieves the expected effect.

Mathematical enhancement of unsupervised machine learning algorithms for optimal lifeline donor knowledge extraction

Lifeline Donor groups are crucial in the field of humanitarian aid because they provide lifesaving aid in times of crisis. For effective resource management and aid coordination, it is crucial to draw relevant insights from donor data in a timely manner. Scalability, precision, and flexibility in responding to changing donor dynamics are all areas where traditional techniques of knowledge extraction fall short. This study offers a fresh method for optimizing knowledge extraction from Lifeline Donor databases by utilizing improved unsupervised machine learning methods. In order to extract, categorize, and prioritize donor knowledge from disparate data sources, this research presents a multi-faceted framework that includes cutting-edge machine learning algorithms. By merging cutting-edge developments in NLP and data mining, the proposed framework improves upon classic clustering and topic modelling methods. The program is able to capture subtle semantic links and underlying patterns in donor communication data by using methods like word embedding models and graph-based clustering.

Analyzing political party positions through multi-language twitter text embeddings.

Traditional monolingual word embedding models transform words into high-dimensional vectors which represent semantics relations between words as relationships between vectors in the high-dimensional space. They serve as productive tools to interpret multifarious aspects of the social world in social science research. Building on the previous research which interprets multifaceted meanings of words by projecting them onto word-level dimensions defined by differences between antonyms, we extend the architecture of establishing word-level cultural dimensions to the sentence level and adopt a Language-agnostic BERT model (LaBSE) to detect position similarities in a multi-language environment. We assess the efficacy of our sentence-level methodology using Twitter data from US politicians, comparing it to the traditional word-level embedding model. We also adopt Latent Dirichlet Allocation (LDA) to investigate detailed topics in these tweets and interpret politicians' positions from different angles. In addition, we adopt Twitter data from Spanish politicians and visualize their positions in a multi-language space to analyze position similarities across countries. The results show that our sentence-level methodology outperform traditional word-level model. We also demonstrate that our methodology is effective dealing with fine-sorted themes from the result that political positions towards different topics vary even within the same politicians. Through verification using American and Spanish political datasets, we find that the positioning of American and Spanish politicians on our defined liberal-conservative axis aligns with social common sense, political news, and previous research. Our architecture improves the standard word-level methodology and can be considered as a useful architecture for sentence-level applications in the future.

Leveraging Pre-Trained Word Embedding Models for Fake Review Identification

Leveraging Pre-Trained Word Embedding Models for Fake Review Identification

LSTM and Word Embedding: Classification and Prediction of Puskesmas Reviews Via Twitter

Puskesmas is a location for top-notch volunteer work that benefits the village and city governments alike. Therefore, patient feedback regarding the kinds of services offered by the community health center is required in an attempt to improve the quality service performance. Patient opinions can be expressed through reviews or opinions about the quality of patient care on social media sites like Facebook, Instagram, Twitter, WhatsApp, and Instagram. On the other hand, thoughts shared on social media are lengthy, unstructured texts. This complicates text analysis and makes it impossible to compare the caliber of services offered by Puskesmas managers. Furthermore, a number of Community Health Centers lack websites that allow users to rank Community Health Centers according to user interest and visual appeal and efficiency in operations. Thus, the purpose of this study is to classify and present sentiment analysis from Twitter about community health centers' health services. The scope focuses on five factors: administrative services, finances, mechanisms, health worker friendliness and skills, and administrative services. The LSTM word embedding model and the adadelta and adamax optimizers are used in word embedding for text mining. A confusion matrix was used to evaluate the developed model's degree of accuracy in categorizing and forecasting patient reviews. Results from the LSTM and Adamax models with a precision level of 76%, Recall 69% and Accuracy 71%. The results of this research show that the LSTM method and Adamax optimizer can classify and predict public opinion data about Puskesmas services via Twitter quite well. A high level of accuracy is very important to ensure that community opinions can be properly identified by the model, so that it can support the decision-making process in improving the type of Puskesmas services. To improve the model, further studies can be conducted on how to select parameters, select features, and create a quality dataset.

Deep Learning Approach for Health Question and Answer Text Segmentation based on Physician-Patient Communication Aspect

Doctor-patient interaction is a criterion for patient satisfaction in Online Health Consultation (OHC), influencing success in the patient's healing process. The interaction between doctors and patients in OHC is done asynchronously, where doctors and patients communicate indirectly through text. This communication process is challenging for doctors, who must apply competence in doctor-patient communication effectively and efficiently. A text segmentation method is needed to evaluate aspects of doctor-patient communication. This study proposes a methodology for segmenting text on six Indonesian language doctor-patient communication aspects and building a specific domain corpus as a word embedding corpus to create a word embedding model.

Translation of Metaphorical Information in Japanese Literature Combined with Text Mining Techniques

Abstract Japanese literature is renowned for its conciseness and the profound, multi-layered meanings conveyed through its economical use of language and rich symbolic content. This paper explores the creative impact of these linguistic features in Japanese literature by analyzing the deep meanings and varied classifications of literary metaphors. The construction of a Japanese literature corpus involved the collection, cleaning, and organization of Japanese literary texts, followed by the selection of specific linguistic features. The corpus was then aligned using a dynamic programming approximation algorithm. A semantic representation model of the Japanese literature text corpus was developed by integrating the Attention Word Embedding Model with the Extended Topic Information Model, and metaphoric corpus features were fused using the static fusion strategy of Extended Topic Information and Attention Word Embedding. This study also examines the translation of symbolic information in Japanese literature, focusing on linguistic and stylistic features. The findings reveal that translations into Chinese tend to employ augmentation translation, resulting in 1,706 additional sentences compared to the original Japanese texts. Furthermore, the lexical density in the Chinese translation reference corpus is 2.45 percentage points higher than in the Japanese literature corpus. Analysis of verb modifications shows that among the 90 altered verbs in the Chinese translation corpus, only 24 have corresponding instances in the original Japanese texts. Text mining techniques were employed to conduct a deep analysis of symbolic content, enhancing the understanding of the nuanced connotations inherent in Japanese literature.

Research on Text Analysis Methods in English Translation Teaching Based on Semantic Association Network Modeling

Abstract As the mainstream language of the world, English still occupies an unshakeable prime position, and English translation is one of the essential learning content for students. The semantic network association network model is introduced in this paper, and a text analysis model for English translation is proposed. The model obtains word vectors through the GloVe word embedding model, combines the relevance measure of the Jaccard index, and applies the improved RelArtNet algorithm to calculate semantic relevance in English texts. The translated data from a prominent English test is selected as a sample to analyze the relevance of each feature value of English translation scores. From the three aspects of word surface, conceptual fidelity, and semantic expression, the textual analysis of the experimental corpus by the model yields the comprehensive scoring effect of English translation by the model. The correlation between translation length and English translation scoring is the highest, and the correlation of scoring in both English to Chinese and Chinese to English is more significant than 0.58. The overall model scoring and manual scoring of English translation are closer, with a difference of 1.59 in scoring and a correlation coefficient of greater than 0.8, which verifies the practical value of the model for scoring English translations. The English translation scoring model can help teachers review translations and students practice independently, which has important practical significance.

Research on Efficient English Teaching Model Combining Corpus Microtext Analysis

Abstract Aiming at the goal of improving the quality of English teaching and combining the characteristics of English teaching, this paper constructs an efficient English discursive literacy teaching based on corpus micro-text analysis. Utilizing the unique advantages of online corpus to assist English teaching, it proposes the targeted teaching of English metaphorical vocabulary and combines the Word2Vec word embedding model and the cosine similarity algorithm to propose a model for analyzing the quality of English text coherence. Based on the characteristics of writing vocabulary in microtexts of the corpus, the analysis integrates the dimensional features of English writing vocabulary teaching. The effectiveness of the corpus-based efficient teaching model for English literacy teaching is verified through teaching practice, specifically utilizing the average values of pre- and post-tested vocabulary, sentences, and chapters of the two classes as comparison data. The results proved that the average value of the pre-test scores of all students in the experimental class is 7.78, and the average value of the post-test is 10.72, which is 2.94 higher than the pre-test. The English teaching model in colleges and universities based on corpus micro-text analysis can significantly improve the English writing level of students and achieve efficient English teaching.

America's racial framework of superiority and Americanness embedded in natural language.

America's racial framework can be summarized using two distinct dimensions: superiority/inferiority and Americanness/foreignness. We investigated America's racial framework in a corpus of spoken and written language using word embeddings. Word embeddings place words on a low-dimensional space where words with similar meanings are proximate, allowing researchers to test whether the positions of group and attribute words in a semantic space reflect stereotypes. We trained a word embedding model on the Corpus of Contemporary American English-a corpus of 1 billion words that span 30 years and 8 text categories-and compared the positions of racial/ethnic groups with respect to superiority and Americanness. We found that America's racial framework is embedded in American English. We also captured an additional nuance: Asian people were stereotyped as more American than Hispanic people. These results are empirical evidence that America's racial framework is embedded in American English.

Fusion of the word2vec word embedding model and cluster analysis for the communication of music intangible cultural heritage

This article aims to propose a method for computing the similarity between lengthy texts on intangible cultural heritage (ICH), to facilitate the swift and efficient acquisition of knowledge about it by the public and promote the dissemination and preservation of this culture. This proposed method builds on traditional text similarity techniques. The ultimate goal is to group together those lengthy texts on ICH that exhibit a high degree of similarity. First of all, the word2vec model is utilized to construct the feature word vector of music ICH communication. This includes the acquisition of long text data on music ICH, the word segmentation of music ICH communication based on the dictionary method in the field of ICH, and the creation of a word2vec model of music ICH communication. A clustering algorithm analyzes and categorizes ICH communication within the music. This procedure involves employing text semantic similarity, utilizing a similarity calculation method based on optimized Word Mover Distance (WMD), and designing long ICH communication clustering. The main objective of this analysis is to enhance the understanding and classification of the intricate nature of ICH within the musical realm. Finally, experiments are conducted to confirm the model’s effectiveness. The results show that: (1) the text word vector training based on the word2vec model is highly accurate; (2) with the increase in K value, the effect of each category of intangible word vector is improving; (3) the final F1-measure value of the clustering experiment based on the optimized WMD is 0.84. These findings affirm the usefulness and accuracy of the proposed methodology.