Word2vec Model Research Articles

IACVP: markedly enhanced identification of anti-coronavirus peptides using a dataset-specific word2vec model.

The COVID-19 pandemic caused several million deaths worldwide. Development of anti-coronavirus drugs is thus urgent. Unlike conventional non-peptide drugs, antiviral peptide drugs are highly specific, easy to synthesize and modify, and not highly susceptible to drug resistance. To reduce the time and expense involved in screening thousands of peptides and assaying their antiviral activity, computational predictors for identifying anti-coronavirus peptides (ACVPs) are needed. However, few experimentally verified ACVP samples are available, even though a relatively large number of antiviral peptides (AVPs) have been discovered. In this study, we attempted to predict ACVPs using an AVP dataset and a small collection of ACVPs. Using conventional features, a binary profile and a word-embedding word2vec (W2V), we systematically explored five different machine learning methods: Transformer, Convolutional Neural Network, bidirectional Long Short-Term Memory, Random Forest (RF) and Support Vector Machine. Via exhaustive searches, we found that the RF classifier with W2V consistently achieved better performance on different datasets. The two main controlling factors were: (i) the dataset-specific W2V dictionary was generated from the training and independent test datasets instead of the widely used general UniProt proteome and (ii) a systematic search was conducted and determined the optimal k-mer value in W2V, which provides greater discrimination between positive and negative samples. Therefore, our proposed method, named iACVP, consistently provides better prediction performance compared with existing state-of-the-art methods. To assist experimentalists in identifying putative ACVPs, we implemented our model as a web server accessible via the following link: http://kurata35.bio.kyutech.ac.jp/iACVP.

Lightweight IDS Framework Using Word Embeddings for In-Vehicle Network Security

As modern vehicle systems evolve into advanced cyber-physical systems, vehicle vulnerability to cyber threats has significantly increased. This paper discusses the need for advanced security in the Controller Area Network (CAN), which currently lacks security features. We propose a novel Intrusion Detection System (IDS) utilizing word embedding techniques from Natural Language Processing (NLP) for effective sequential pattern representations to improve intrusion detection in CAN traffic. This method transforms CAN identifiers into multi-dimensional vectors, enabling the model to capture complex sequential patterns of CAN traffic behaviors. Our methodology focuses on a lightweight neural network adaptable for automotive systems with limited computational resources. At first, a Word2Vec model is trained to make the embedding matrix of CAN IDs. Then, using the pre-trained embedding layer extracted from the Word2Vec network, the classifier analyzes embeddings from CAN data to detect intrusions. This model is viable for resource-constrained environments due to its low computational expense and memory usage. Key contributions of this research are (1) the application of word embeddings for intrusion detection in CAN traffic, (2) a streamlined neural network that balances accuracy with efficiency, and (3) a comprehensive evaluation showing our model’s competitive performance compared to relatively heavy deep learning models. Experimental results using the Car-Hacking dataset, widely used for automotive security research, demonstrate that our IDS effectively detects four different types of attacks on CAN. This work advances vehicle security technologies, contributing to safer transportation systems.

Evaluation of clustering techniques on Urdu News head-lines: a case of short length text

ABSTRACT Document clustering and topic modelling methods are useful in determining groups of semantically similar documents. In literature, various studies have been conducted to evaluate the performance of lexical and semantic features on a clustering task. However, the focus of such studies was on the English language which is considered a high resource language. This paper analyses the effect of various feature extraction techniques combined with clustering and topic modelling algorithms on grouping news headlines written in Urdu, a low-resource language. Various features were first extracted using different approaches such as term-frequency inverse document frequency (TF-IDF), word embeddings learned using Word2vec and fastText models, deep contextualised word embeddings from BERT, and document embeddings extracted from the Doc2Vec model. Three different clustering algorithms: K-Means, Affinity Propagation and Density-based spatial clustering of applications with noise (DBSCAN) were then used to produce clusters for a given set of news headlines. The Latent Dirichlet Allocation topic model was also assessed for comparison with clustering algorithms. The evaluation based on extrinsic measures revealed that higher-quality clusters were produced when the K-Means algorithm was applied on a frequent unigrams-based TF-IDF feature matrix. In addition, several word embedding models have been trained on the domain of Urdu News headlines using state-of-the-art techniques such as Word2Vec, Doc2Vec, and fastText.

Deep learning method for Chinese multisource point of interest matching

Multisource point of interest (POI) matching refers to the pairing of POIs that refer to the same geographic entity in different data sources. This also constitutes the core issue in geospatial data fusion and update. The existing methods cannot effectively capture the complex semantic information from a text, and the manually defined rules largely affect matching results. This study developed a multisource POI matching method based on deep learning that transforms the POI pair matching problem into a binary classification problem. First, we used three different Chinese word segmentation methods to segment the POI text attributes and used the segmentation results to train the Word2Vec model to generate the corresponding word vector representation. Then, we used the text convolutional neural network (Text-CNN) and multilayer perceptron (MLP) to extract the POI attributes' features and generate the corresponding feature vector representation. Finally, we used the enhanced sequential inference model (ESIM) to perform local inference and inference combination on each attribute to realize the classification of POI pairs. We used the POI dataset containing Baidu Map, Tencent Map, and Gaode Map from Chengdu to train, verify, and test the model. The experimental results show that the matching precision, recall rate, and F1 score of the proposed method exceed 98% on the test set, and it is significantly better than the existing matching methods.

LSTM (Long Short Term Memory) for Sentiment COVID-19 Vaccine Classification on Twitter

The implementation of the Covid-19 vaccination carried out by Indonesian government was ignited pros and contras among the public. Certainly, there will be pros and cons about the vaccination from the community. This attituded of pros and cons, which is also called sentiment, can influence people to accept or refuse to be vaccinated. Todays, people express their sentiment in social media in comments, post, or status. One of the methods used to detect sentiment on social media, whether positive or negative, is through a categorisation of text approach. This research provides a deep learning technique for sentiment classification on Twitter that uses Long Short Term Memory (LSTM), for positive, neutral and negative classes. The word2vec word embeddings was used as input, using the pretrained Bahasa Indonesia model from Wikipedia corpus. On the other hand, the topic-based word2vec model was also trained from the Covid-19 vaccination sentiment dataset which collected from Twitter. The data used after balanced is 2564 training data, 778 data validation data, and 400 test data with 1802 neutral data, 1066 negative data, and 566 positive data. The best results from various parameter processes give an F1-Score value of 54% on the test data, with an accuracy of 66%. The result of this research is a model that can classify sentiments with new sentences.

Detecting review spammer groups based on generative adversarial networks

The detection of spammer groups has recently gained more attention. However, the existing spammer group detection approaches rely on manual feature engineering to design spam indicators or extract features to capture the aggregated behavioral patterns exhibited by spammer groups. This is costly and time-consuming. To address this limitation, we formulate spammer group detection as a problem of finding distribution differences between normal user groups and spammer groups, and propose a review spammer group detection approach based on generative adversarial networks. First, we perform multiview sequence sampling on the dataset and utilize the Word2Vec model to obtain low-dimensional vector representations of users. Second, we construct the N-nearest neighbor user relationship graph according to the vector similarities in the embedding space. In the closed neighborhood of each node, we use the DBSCAN algorithm to find candidate groups. Finally, we construct a generative adversarial network model for spammer group detection, which uses the sum of generator reconstruction loss and discriminator loss to evaluate the spamicities of candidate groups. The experimental results on three real-world review datasets indicate that the proposed approach outperforms the baseline approaches in detection performance.

Imputation of missing time-activity data with long-term gaps: A multi-scale residual CNN-LSTM network model

Despite the increasing availability and spatial granularity of individuals' time-activity (TA) data, the missing data problem, particularly long-term gaps, remains as a major limitation of TA data as a primary source of human mobility studies. In the present study, we propose a two-step imputation method to address the missing TA data with long-term gaps, based on both efficient representation of TA patterns and high regularity in TA data. The method consists of two steps: (1) the continuous bag-of-words word2vec model to convert daily TA sequences into a low-dimensional numerical representation to reduce complexity; (2) a multi-scale residual Convolutional Neural Network (CNN)-stacked Long Short-Term Memory (LSTM) model to capture multi-scale temporal dependencies across historical observations and to predict the missing TAs. We evaluated the performance of the proposed imputation method using the mobile phone-based TA data collected from 180 individuals in western New York, USA, from October 2016 to May 2017, with a 10-fold out-of-sample cross-validation method. We found that the proposed imputation method achieved excellent performance with 84% prediction accuracy, which led us to conclude that the proposed imputation method was successful at reconstructing the sequence, duration, and spatial extent of activities from incomplete TA data. We believe that the proposed imputation method can be applied to impute incomplete TA data with relatively long-term gaps with high accuracy.

URBAN FUNCTIONAL ZONE IDENTIFICATION BY CONSIDERING THE HETEROGENEOUS DISTRIBUTION OF POINTS OF INTERESTS

Abstract. Urban Functional Zone (UFZ) identification facilitates the understanding of urban systems, which are complex and huge, and helps promote sustainable urban development. Existing studies on UFZ identification with Points of Interests (POIs) have focused much on more accurately extracting functional semantics, but ignored the fine delineation of UFZs in the spatial domain. The fine delineation of the spatial units of UFZs is also a key issue in UFZ identification. Since the sizes of UFZs can be different in practice, it is difficult to delineate spatially heterogeneous UFZs on a fixed scale. To solve the issue, a novel multi-scale spatial segmentation method was proposed in this study. Through taking the homogeneous socio-economic attributes of UFZs into account, we firstly generated a number of multi-scale spatial units by computing the mixed degree of POIs types, which reflects the mixed functions of each UFZs, using information entropy. Subsequently, we constructed the urban functional corpus of each spatial unit by measuring the spatial distribution pattern of POIs. The Word2Vec model was employed to obtain the semantic embedding vectors of UFZs, following which we adopted cosine distance-based K-means clustering method to group similar UFZs into one cluster. Finally, the enrichment factor was used to help annotate each functional cluster with a specific label. The UFZ identification results were compared with the Baidu e-maps and Baidu street view images for evaluation, and an accuracy of 82.7% was obtained. This study considering the heterogeneous distribution of POIs supports the fine-grained identification of UFZs, providing reference for urban planning.

Text Similarity Measurement Method and Application of Online Medical Community Based on Density Peak Clustering

Text similarity measurement is a link between basic research such as text modeling and upper-level application research of text potential information. In order to improve the accuracy of text similarity measurement, this paper proposes a semantic similarity calculation method integrating word2vec model and TF-IDF, and applies it to the density peak clustering of Chinese text data consulted by patients in online medical community. Experimental results show that the proposed similarity measurement method is superior to the traditional method. Furthermore, the study is among the first to apply the density peak clustering algorithm to online medical community, which offers a reference for how to find out user demands from medical text data in the big data environment.

Agenda Dynamics on Social Media During COVID-19 Pandemic: Interactions Between Public, Media, and Government Agendas

ABSTRACT Social media has profoundly altered the way the public, the media, and the government set their agendas. To examine the dynamics between public, government, and media agendas, this study collected posts on Sina Weibo during the COVID-19 pandemic in China from January 2020 to July 2020. The word2vec model and Granger causality analysis were used to measure agenda networks and examine the agenda dynamics. The results show that (1) pandemic statistics were the central issue in public, media, and government agenda networks, (2) the government’s agenda influenced the media’s agenda, and (3) the public agenda affected both media and government agendas. The findings suggest that the public agenda was more influential than the media agenda and the government agenda on Weibo during the COVID-19 pandemic in China. This study contributes to the literature by examining agenda-setting dynamics in a pandemic context. It also extends existing methods by modeling implicit relationships between attributes in agenda networks.

Word2vec model and HNSW index

Word2vec model and HNSW index

Research on the Construction of Emergency Network Public Opinion Emotional Dictionary Based on Emotional Feature Extraction Algorithm.

How to strengthen emergency management and improve the ability to prevent and respond to emergencies is an important part of building a harmonious socialist society. This paper proposes a domain emotion dictionary construction method for network public opinion analysis of public emergencies. Using the advantages of corpus and semantic knowledge base, this paper extracts the seed words based on the large-scale network public opinion corpus and combined with the existing emotion dictionary, trains the word vector through the word2vec model in deep learning, expands the emotion words, and obtains the candidate emotion words according to the semantic similarity calculation, So as to generate a domain emotion dictionary. The accuracy rate of emotion discrimination by the emotion dictionary constructed in this paper is 0.86, the recall rate is 0.92. Through the verification of accuracy and recall rate, the construction method proposed in this paper has good accuracy and reliability. Because of the great differences in different experiences and situations of different groups, there will be great differences in views and perspectives on the same event. The key to prevent the public from blindly following the crowd should be to reach groups close to emotional distance, and targeted prevention and control of public opinion can be conducted according to different characteristics of different groups.

The Construction and Trend of Feminist Literature Theory Based on Social Media Data Mining

Up to now, the development path of Chinese women's liberation movement and modern Chinese women's literature is fundamentally different from that of the West. Therefore, the study of Chinese women's literature can not only rely on the essentialism of western feminist literary theory but also must return to the social reality and cultural reality of China. Based on social media data mining, this study uses the Word2vec model to map the text content to a more abstract word vector space, improves the original Text Rank algorithm from three aspects, semantic association between words, word frequency, and word directionality, then carries out feature extraction, and applies this algorithm to the generation of user tags. The feasibility and superiority of the model are verified by comparative experiments on LFR benchmark network. The research in this study provides a reference for the analysis of users' interests and behaviors and has certain theoretical significance and application value.

Identification and Classification of Depressed Mental State for End-User over Social Media.

In researching social network data and depression, it is often necessary to manually label depressed and non-depressed users, which is time-consuming and labor-intensive. The aim of this study is that it explores the relationship between social network data and depression. It can also contribute to detecting and identifying depression. Through collecting and analyzing college students' microblog social data, a preliminary screening algorithm for college students' suspected depression microblogs based on depression keywords, and semantic expansion is researched; a comprehensive lexical grammar was proposed. This research provided has a preliminary screening method based on depression keywords and semantic expansion for college students' suspected depression microblogs, with a screening accuracy. This method forms a depression keyword table by constructing the basic keyword table and the semantic expansion based on the word embedding learning model Word2Vec. Finally, the word table is used to calculate the semantic similarity of the tested microblogs and then identify whether it is a suspected depression microblog. The experimental results on the microblog dataset of college students show that the comprehensive lexical method is better than the SDS questionnaire segmentation method and the expert lexical method in terms of screening accuracy; the comprehensive lexical approach can quickly and automatically screen out a tiny proportion of suspected doubts from a large number of college students' microblogs. Depression Weibo can reduce the workload of experts' annotation, improve annotation efficiency, and provide a suitable data processing basis for the subsequent accurate identification (classification problem) of patients with depression.

End-to-End Lip-Reading Open Cloud-Based Speech Architecture.

Deep learning technology has encouraged research on noise-robust automatic speech recognition (ASR). The combination of cloud computing technologies and artificial intelligence has significantly improved the performance of open cloud-based speech recognition application programming interfaces (OCSR APIs). Noise-robust ASRs for application in different environments are being developed. This study proposes noise-robust OCSR APIs based on an end-to-end lip-reading architecture for practical applications in various environments. Several OCSR APIs, including Google, Microsoft, Amazon, and Naver, were evaluated using the Google Voice Command Dataset v2 to obtain the optimum performance. Based on performance, the Microsoft API was integrated with Google’s trained word2vec model to enhance the keywords with more complete semantic information. The extracted word vector was integrated with the proposed lip-reading architecture for audio-visual speech recognition. Three forms of convolutional neural networks (3D CNN, 3D dense connection CNN, and multilayer 3D CNN) were used in the proposed lip-reading architecture. Vectors extracted from API and vision were classified after concatenation. The proposed architecture enhanced the OCSR API average accuracy rate by 14.42% using standard ASR evaluation measures along with the signal-to-noise ratio. The proposed model exhibits improved performance in various noise settings, increasing the dependability of OCSR APIs for practical applications.

Framing the Cacerolazo: An Analysis of a Social Protest in Ecuador

Objective/Context: In the hybrid media system, Twitter users can propose alternative framings to those defined by media. In the context of the 2019 social protest in Ecuador, the framing confrontation was organized around hashtag #cacerolazo: different frames interpreted differently the meaning of the protest designated with this name. Methodology: The study analyzed 119,756 tweets containing trending hashtags related to the protest. The most frequent terms were analyzed, and it was observed that the word “cacerolazo” was framed and organized into three groups according to its context: 1) opposition to the government, which was the most popular; 2) support for the government, and 3) neutral. The framing defined by digital media for the same word was examined and categorized into groups for comparison; this allowed disambiguating the meaning of the word “cacerolazo” in the tweets. As a result, 67% of the tweets with the hashtag were in group 1 (opposition to the government). Thus, it was determined that, for the community considered in this study, the act of performing a cacerolazo retains its meaning of protest and it is not a call for peace. Conclusions: The framing of hashtag #cacerolazo was mostly promoted by users whose publications were against the government, and it quoted and criticized the framing established by the media. Originality: The word2vec model was used, which is an automated methodology that helps identify different frames defined by social media users during a certain event.

Analysis of the causes of inferiority feelings based on social media data with Word2Vec

Feelings of inferiority are complex emotions that usually indicate perceived weakness and helplessness. A lack of timely and effective interventions may bring serious consequences to individuals with inferiority feelings. Due to privacy concerns, those people often hesitate to seek face-to-face help, but they usually spontaneously share their feelings on social media, which makes social media a good resource for ample inferiority-related data. We randomly selected a sample of posts indicating inferiority feelings to explore the causes of inferiority. Through language analysis and natural language processing, we constructed a Word2Vec model of inferiority based on social media data and applied it to the cause analysis of inferiority feelings. The main causes of inferiority feelings are personal experience, social interaction, love relationship, etc. People feeling inferior about their personal experiences usually are largely influenced by their ways of thinking and life attitudes. Social and emotional factors overlap somewhat in the development of inferiority. In love relationships, males are more prone to inferiority feeling than females. These findings will help relevant institutions and organizations better understand people with inferiority feelings and facilitate the development of targeted treatment for those with potential self-esteem problems.

A New Method for WebShell Detection Based on Bidirectional GRU and Attention Mechanism

With the rapid development of hacker technology, network security issues have become increasingly serious. Uploading WebShell is one of the most common attack methods used by network intruders. WebShell escape technology is changing with each passing day, and the traditional method based on feature matching is difficult to accurately detect. In order to detect WebShell more accurately and mitigate the threat caused by WebShell attacks, a WebShell detection method combining bidirectional GRU (gated recurrent unit) and attention mechanism is proposed for the first time. First, the sample is preprocessed to remove useless information such as annotations. Then, the sample is divided into a series of words, the word2vec model is used to obtain the word vector, and finally, the word vector is input into the network for prediction. According to the experimental results, compared with peer methods, the method in this study performs better in performance indicators such as accuracy rate, recall rate, and F1 value. The model not only detects the PHP-type WebShell but also has a good performance on the WebShell written in JSP, ASPX, or ASP languages. The detection accuracy of PHP-type, JSP-type, and ASP-type WebShell reached 99.36%, 99.23%, and 99.87%, respectively, and the recall rate was 98.6%, 99.13%, and 99.56%, respectively.

Detection of hyperpartisan news articles using natural language processing technique

Yellow journalism has increased the spread of hyperpartisan news on the internet. It is very difficult for online news article readers to distinguish hyperpartisan news articles from mainstream news articles. There is a need for an automated model that can detect hyperpartisan news on the internet and tag them as hyperpartisan so that it is very easy for readers to avoid that news. A hyperpartisan news detection article was developed by using three different natural language processing techniques named BERT, ELMo, and Word2vec. This research used the bi-article dataset published at SEMEVAL-2019. The ELMo word embeddings which are trained on a Random forest classifier has got an accuracy of 0.88, which is much better than other state of art models. The BERT and Word2vec models have got the same accuracy of 0.83. This research tried different sentence input lengths to BERT and proved that BERT can extract context from local words. Evidenced from the described ML models, this study will assist the governments, news’ readers, and other political stakeholders to detect any hyperpartisan news, and also helps policy to track, and regulate, misinformation about the political parties and their leaders.

Wiki sense bag creation using multilingual word sense disambiguation

Performance of word sense disambiguation (WSD) is one of the challenging tasks in the area of natural language processing (NLP). Generation of sense annotated corpus for multilingual word sense disambiguation is out of reach for most languages even if resources are available. In this paper we propose an unsupervised method using word and sense embedding or improving the performance of these systems using untagged. Corpora and create two bags namely ontological bag and wiki sense bag to generate the senses with highest similarity. Wiki sense bag provides external knowledge to the system required to boost the disambiguation accuracy. We explore Word2Vec model to generate the sense bag and observe significant performance gain for our dataset.