Continuous Bag Of Words Model Research Articles

The Study of Fishing Vessel Behavior Identification Based on AIS Data: A Case Study of the East China Sea

The goal of this paper is to strengthen the supervision of fishing behavior in the East China Sea and effectively ensure the sustainable development of fishery resources. Based on AIS data, this paper analyzes three types of fishing boats (purse seine operation, gill net operation and trawl operation) and uses the cubic spline interpolation algorithm to optimize the ship trajectory and construct high-dimensional features. It proposes a new coding method for fishing boat trajectory sequences. This method uses the Geohash algorithm to divide the East China Sea into grids and generate corresponding numbers. Then, the ship trajectory is mapped to the grid, the fishing boat trajectory points are associated with the divided grid, and the ship trajectory ID is extracted from the corresponding grid. The extracted complete trajectory sequence passes through the CBOW (continuous bag of words) model, and the correlation of trajectory points is fully learned. Finally, the fishing boat trajectory is converted from coordinate sequence to trajectory vector, and the processed trajectory sequence is trained by the LightGBM algorithm. In order to obtain the optimal classification effect, the optimal superparameter combination is selected. We put forward a LightGBM algorithm based on the Bayesian optimization algorithm, and obtained the classification results of three kinds of fishing boats. The final result was evaluated using the F1_score. Experimental results show that the F1_score trained with the proposed trajectory vectorization method is the highest, with a training accuracy of 0.925. Compared to XgBoost and CatBoost, the F1_score increased by 1.8% and 1.2%, respectively. The results show that this algorithm demonstrates strong applicability and effectiveness in fishery area evaluations and is significant for strengthening fishery resource management.

Sentence embedding approach using LSTM auto-encoder for discussion threads summarization

Online discussion forums are repositories of valuable information where users interact and articulate their ideas, opinions, and share experiences about nu merous topics. They are internet-based online communities where users can ask for help and find the solution to a problem. On online discussion forums, a new user becomes exhausted from reading the significant number of replies in a discussion. An automated discussion thread summarizing system (DTS) is necessary to create a candid view of the entire discussion of a query. Most of the previous approaches for automated DTS use the continuous bag of words (CBOW) model as a sentence embedding tool, which is poor at capturing the overall meaning of the sentence and is unable to grasp word dependency. To overcome this limitation, we introduce the LSTM Auto-encoder as a sentence embedding technique to improve the per formance of DTS. The empirical result in the context of average precision, recall, and F-measure of the proposed approach with respect to ROGUE-1 and ROUGE-2 of two standard experimental datasets proves the effectiveness and efficiency of the proposed approach and outperforms the state-of-the-art CBOW model in sentence embedding tasks by boosting the performance of the automated DTS model.

Intelligent deep learning‐based hierarchical clustering for unstructured text data

SummaryDocument clustering is a technique used to split the collection of textual content into clusters or groups. In modern days, generally, the spectral clustering is utilized in machine learning domain. By using a selection of text mining algorithms, the diverse features of unstructured content is captured for ensuing in rich descriptions. The main aim of this article is to enhance a novel unstructured text data clustering by a developed natural language processing technique. The proposed model will undergo three stages, namely, preprocessing, features extraction, and clustering. Initially, the unstructured data is preprocessed by the techniques such as punctuation and stop word removal, stemming, and tokenization. Then, the features are extracted by the word2vector using continuous Bag of Words model and term frequency‐inverse document frequency. Then, unstructured features are performed by the hierarchical clustering using the optimizing the cut‐off distance by the improved sensing area‐based electric fish optimization (FISA‐EFO). Tuned deep neural network is used for improving the clustering model, which is proposed by same algorithm. Thus, the results reveal that the model provides better clustering accuracy than other clustering techniques while handling the unstructured text data.

An Improved Cat Swarm Search-Based Deep Ensemble Learning Model for Group Recommender Systems

Recommender systems are often employed in different fields such as music, travel, and movies. The recommender systems are broadly utilised nowadays due to the emergence of social activities, in which particular recommendations are offered by group recommender systems. It is a system for recommending the items to a set of users together based on their preferences. The user preferences are used from the behavioural and social aspects of group members to enhance the quality of products recommended in various groups for generating the group recommendations. These group recommender systems solve the cold start problem, which is raised in an individual recommendation system. The ultimate aim of this paper is to design and develop a new Improved Deep Ensemble Learning Model (ID-ELM) for the group recommender systems concerning different application-oriented datasets. Initially, the datasets from different applications like healthcare, e-commerce, and e-learning are gathered from benchmark sources and split the data into various groups. These data are given to the pre-processing for making it fit for further processing. The pre-processing steps like stop word removal, stemming, and punctuation removal are performed here. Then the features are extracted using the Continuous Bag of Words Model (CBOW), and Principal Component Analysis (PCA) is used for dimension reduction. These features are fed to the ID-ELM, in which the optimised Convolutional Neural Network (CNN) extracts the significant features from the pooling layer, and the fully connected layer is replaced by a set of classifiers termed Neural Networks (NN), AdaBoost, and Logistic Regression (LR). Finally, the ranking of the ensemble learning model based on the group reviews extends the recommendation outcome. The optimised CNN is proposed by the Adaptive Seeking Range-based Cat Swarm Optimisation (ASR-CSO) for attaining better results. This model is validated on the benchmark datasets to show the efficiency of the designed model with different meta-heuristic-based algorithms and classification algorithms.

Deep Sentiment Analysis Using CNN-LSTM Architecture of English and Roman Urdu Text Shared in Social Media

Sentiment analysis (SA) has been an active research subject in the domain of natural language processing due to its important functions in interpreting people’s perspectives and drawing successful opinion-based judgments. On social media, Roman Urdu is one of the most extensively utilized dialects. Sentiment analysis of Roman Urdu is difficult due to its morphological complexities and varied dialects. The purpose of this paper is to evaluate the performance of various word embeddings for Roman Urdu and English dialects using the CNN-LSTM architecture with traditional machine learning classifiers. We introduce a novel deep learning architecture for Roman Urdu and English dialect SA based on two layers: LSTM for long-term dependency preservation and a one-layer CNN model for local feature extraction. To obtain the final classification, the feature maps learned by CNN and LSTM are fed to several machine learning classifiers. Various word embedding models support this concept. Extensive tests on four corpora show that the proposed model performs exceptionally well in Roman Urdu and English text sentiment classification, with an accuracy of 0.904, 0.841, 0.740, and 0.748 against MDPI, RUSA, RUSA-19, and UCL datasets, respectively. The results show that the SVM classifier and the Word2Vec CBOW (Continuous Bag of Words) model are more beneficial options for Roman Urdu sentiment analysis, but that BERT word embedding, two-layer LSTM, and SVM as a classifier function are more suitable options for English language sentiment analysis. The suggested model outperforms existing well-known advanced models on relevant corpora, improving the accuracy by up to 5%.

A Simple and Effective Usage of Word Clusters for CBOW Model

We propose a simple and effective method for incorporating word clusters into the Continuous Bag-of-Words (CBOW) model. Specifically, we propose to replace infrequent input and output words in CBOW model with their clusters. The resulting cluster-incorporated CBOW model produces embeddings of frequent words and a small amount of cluster embeddings, which will be fine-tuned in downstream tasks. We empirically show our replacing method works well on several downstream tasks. Through our analysis, we show that our method might be also useful for other similar models which produce word embeddings.

Bengali paper classification using ensemble machine learning algorithms

Text classification is one of the most challenging problems in natural language processing (NLP). Language models are at the heart of NLP. The ability to represent texts as numbers has given rise to many NLP tasks, for example, text categorisation, translation, and summarisation. Unfortunately, NLP for Bengali texts has not reached the state-of-art level of other Languages like English yet, mostly due to the scarcity of resources and the complexities seen in Bengali grammar. Therefore, not much work has been done in this field. In this paper, we have studied one of the word embedding methods, Word2vec, based on continuous bag of words (CBOW) with several ensemble machine learning algorithms, e.g., Adaptive Boosting Classifiers, Light Gradient Boosting Machine, XGboost, and random forest classifiers (RFC). The model is trained on a large corpus of Bengali newspapers of a considerable size that has 99283949 words and 8284804 sentences in 392772 documents. In our experiment, Word2vec CBOW model with XGboost algorithm performed much better than other models and achieved 92.24% accuracy.

Improved Dota2 lineup recommendation model based on a bidirectional LSTM

In recent years, e-sports has rapidly developed, and the industry has produced large amounts of data with specifications, and these data are easily to be obtained. Due to the above characteristics, data mining and deep learning methods can be used to guide players and develop appropriate strategies to win games. As one of the world's most famous e-sports events, Dota2 has a large audience base and a good game system. A victory in a game is often associated with a hero's match, and players are often unable to pick the best lineup to compete. To solve this problem, in this paper, we present an improved bidirectional Long Short-Term Memory (LSTM) neural network model for Dota2 lineup recommendations. The model uses the Continuous Bag Of Words (CBOW) model in the Word2vec model to generate hero vectors. The CBOW model can predict the context of a word in a sentence. Accordingly, a word is transformed into a hero, a sentence into a lineup, and a word vector into a hero vector, the model applied in this article recommends the last hero according to the first four heroes selected first, thereby solving a series of recommendation problems.

Research and Construction of Word Vector Extension Structure Based on Context

The CBOW (Continuous Bag-of-Words) model is a three-layer forward neural network that predicts the central word vector by the fixed-size window information. In fact, context has a very important role in understanding the meaning of words. However, the context information of the fixed window size is partial and it is not enough to represent the whole context. Due to the polysemy of Chinese words, the same word may have different semantics in different contexts, while the traditional CBOW method ignores the polysemy of words. Therefore, this paper proposes a context-based word vector extension structure for the above problems. The main contents are as follows: (1) Introducing the concept of context vector. The entire sentence in which the target word is located is represented by a vector. (2) Constructing the polysemy storage method of words, adding a contextual list to each word vector, so that the multiple semantics of the words can be effectively distinguished. (3) Based on the word vector extension structure of this paper, a new character vector generation model is proposed, and the effect of the new model and the traditional CBOW model in the news headline similarity sorting task is compared. The experimental results show that the new character vector generation model based on the extension structure of the word vector obtains better result.

A COMPARATIVE STUDY OF WORD REPRESENTATION METHODS WITH CONDITIONAL RANDOM FIELDS AND MAXIMUM ENTROPY MARKOV FOR BIO-NAMED ENTITY RECOGNITION

Bio-Named Entity Recognition (Bio-NER) is the process of identifying and semantically classifying biomedical technical terms and named entities in Biomedicine literature. Therefore, it is a major task in biomedical knowledge acquisition. Meanwhile, Natural Language Processing (NLP) plays an important role in Bio-NER in the biomedical domain. The first and most essential biomedical literature mining task incorporates biomedical entity recognition such as protein, gene, and chemicals. The most recent Bio-NER methods rely on predefined traditional features, which attempt to capture the specific surface properties of entity types. However, these empirically predefined feature sets differ between entity types and are manually constructed and complicated, which means developing them is costly. In this paper, we systematically present a comparative evaluation study of three methods, which are: the traditional feature representation method, the continuous bag-of-words (CBOW) model, and a new prototypical representation method with two popular sequence-labeling approaches (Conditional Random Fields (CRFs) and Maximum Entropy Markov Models (MEMM)). We evaluated these models with two major Bio-NER tasks, which involve the JNLPBA and GENETAG corpora. This paper examined the prototypical word representation method and found that Word2Vec can be successfully used for Bio-NER. Our results show that the new prototypical representation method improved the performance of the two machine learning models with different datasets. Also, the new prototypical representation method performed better than the traditional feature representation method and CBOW model for both datasets. Finally, our experiment proved that the CRF classifier with the new prototypical representation method achieved the best results when 90% data was used as training data, yielding overall F-measure values of 0.79% and 0.85% for the JNLPBA corpus and GENETAG corpus, respectively. In comparison, the results achieved using the ME classifier yielded overall F-measure values of 0.76% and 0.78% for the JNLPBA corpus and GENETAG corpus, respectively.

Discovering Finance Keywords via Continuous-Space Language Models

The growing amount of public financial data makes it increasingly important to learn how to discover valuable information for financial decision making. This article proposes an approach to discovering financial keywords from a large number of financial reports. In particular, we apply the continuous bag-of-words (CBOW) model, a well-known continuous-space language model, to the textual information in 10-K financial reports to discover new finance keywords. In order to capture word meanings to better locate financial terms, we also present a novel technique to incorporate syntactic information into the CBOW model. Experimental results on four prediction tasks using the discovered keywords demonstrate that our approach is effective for discovering predictability keywords for post-event volatility, stock volatility, abnormal trading volume, and excess return predictions. We also analyze the discovered keywords that attest to the ability of the proposed method to capture both syntactic and contextual information between words. This shows the success of this method when applied to the field of finance.