Wikipedia Corpus Research Articles

A hybrid BiLSTM-CNN deep learning model for Chinese sentiment analysis of online car reviews

Although sentiment analysis with datasets in English has achieved significant progress, there are still relatively few studies on sentiment analysis in the area of Chinesecar review texts. In addition, the existing Chinese text sentiment analysis methods cannot simultaneously extract the contextual features and the local semantic information from the review texts. To address the above issues, a hybrid deep learning model namely BiLSTM-CNN was designed and validated with Chinese car review texts. We trained word vectors with the CBOW model by combining the Chinese Wikipedia corpus with other open-source car-related corpora. Such word vectors include car-specific vocabulary, which improves sentimental classification accuracy. Experimental results show that the performance indices of the deep learning models (CNN, LSTM, BiLSTM) are much better than the KNN, SVM, Naïve Bayes, and RF model, which is attributed to deep learning's powerful feature extraction ability and nonlinear fitting ability. Furthermore, when compared to deep learning models such as CNN, LSTM, BiLSTM, and LSTM-CNN, the suggested hybrid BiLSTM-CNN model outperformed the others. The results may provide references for consumers to buy a car and for car companies to optimize their products.

Analyzing Semantic Properties of Word Embeddings Using Eigenvectors

Dense word vectors have demonstrated their efficacy in several downstream natural language processing (NLP) tasks in recent years. Nevertheless, the interpretability of these embeddings' dimensions remains challenging. In this paper, we investigate how eigenvectors can reveal different semantic properties captured by word embedding models. Hence, we train word embeddings (e.g., Word2Vec) on English Wikipedia corpus to analyze the top eigenvectors, identify specific semantic properties (e.g., sentiment, formality) associated with each, and explore how these properties are encoded in the embedding space. This paper also discussed the limitations and potential benefits of this approach compared to other methods for analyzing word embeddings.

Entity Linking for real-time geolocation of natural disasters from social network posts.

When a fast kinetic natural disaster occurs, it is crucial that crisis managers quickly understand the extent of the situation, especially through the development of "big picture" maps. For many years, great efforts have been made to use social networks to help build this situational awareness. While there are many models for automatically extracting information from posts, the difficulty remains in detecting and geolocating this information on the fly so that it can be placed on maps. Whilst most of the work carried out to date on this subject has been based on data in English, we tackle the problem of detecting and geolocating natural disasters from French messages posted on the Twitter platform (now renamed "X"). To this end, we first build an appropriate dataset comprised of documents from the French Wikipedia corpus, the dataset from the CAp 2017 challenge, and a homemade annotated Twitter dataset extracted during French natural disasters. We then developed an Entity-Linking pipeline in adequacy with our end-application use case: real-time prediction and peak resiliency. We show that despite these two additional constraints, our system's performances are on par with state-of-the-art systems. Moreover, the entities geolocated by our model show a strong coherence with the spatiotemporal signature of the natural disasters considered, which suggests that it could usefully contribute to automatic social network analysis for crisis managers.

A proposed semantic keywords search engine for Indonesian Qur’an translation based on word embedding

Obtaining relevant information from the Holy Qur’an can be really challenging for people who cannot speak Arabic, such as the Indonesian people. One technology implementation which is commonly used to tackle this problem is to develop a search engine application for Al-Qur’an verses. This paper proposes a search engine based on semantic representation keywords for the Indonesian translation of the Al-Qur’an which consists of 3 phases i.e., data preparation, document representation, and search engine development. In the first stage, the Al-Qur’an dataset was built using the official translation of the Al-Qur’an from the Ministry of Religion and then enriched with the Wikipedia corpus. The second phase is document representation which produces feature vectors by utilizing the Word2Vec algorithm. Finally, the development of a search engine that can find the most relevant verses by calculating the cosine similarity between the document and the keywords. It was found that the proposed search engine succeeded in exceeding the performance of ordinary search engines by finding wider information due to the use of semantic keywords. Apart from that, the proposed search engine succeeded in maintaining the relevance of search results by achieving precision and recall levels of 98.7% and 97.3% respectively.

Leveraging contextual representations with BiLSTM-based regressor for lexical complexity prediction

Lexical complexity prediction (LCP) determines the complexity level of words or phrases in a sentence. LCP has a significant impact on the enhancement of language translations, readability assessment, and text generation. However, the domain-specific technical word, the complex grammatical structure, the polysemy problem, the inter-word relationship, and dependencies make it challenging to determine the complexity of words or phrases. In this paper, we propose an integrated transformer regressor model named ITRM-LCP to estimate the lexical complexity of words and phrases where diverse contextual features are extracted from various transformer models. The transformer models are fine-tuned using the text-pair data. Then, a bidirectional LSTM-based regressor module is plugged on top of each transformer to learn the long-term dependencies and estimate the complexity scores. The predicted scores of each module are then aggregated to determine the final complexity score. We assess our proposed model using two benchmark datasets from shared tasks. Experimental findings demonstrate that our ITRM-LCP model obtains 10.2% and 8.2% improvement on the news and Wikipedia corpus of the CWI-2018 dataset, compared to the top-performing systems (DAT, CAMB, and TMU). Additionally, our ITRM-LCP model surpasses state-of-the-art LCP systems (DeepBlueAI, JUST-BLUE) by 1.5% and 1.34% for single and multi-word LCP tasks defined in the SemEval LCP-2021 task.

CsFEVER and CTKFacts: acquiring Czech data for fact verification

In this paper, we examine several methods of acquiring Czech data for automated fact-checking, which is a task commonly modeled as a classification of textual claim veracity w.r.t. a corpus of trusted ground truths. We attempt to collect sets of data in form of a factual claim, evidence within the ground truth corpus, and its veracity label (supported, refuted or not enough info). As a first attempt, we generate a Czech version of the large-scale FEVER dataset built on top of Wikipedia corpus. We take a hybrid approach of machine translation and document alignment; the approach and the tools we provide can be easily applied to other languages. We discuss its weaknesses, propose a future strategy for their mitigation and publish the 127k resulting translations, as well as a version of such dataset reliably applicable for the Natural Language Inference task—the CsFEVER-NLI. Furthermore, we collect a novel dataset of 3,097 claims, which is annotated using the corpus of 2.2 M articles of Czech News Agency. We present an extended dataset annotation methodology based on the FEVER approach, and, as the underlying corpus is proprietary, we also publish a standalone version of the dataset for the task of Natural Language Inference we call CTKFactsNLI. We analyze both acquired datasets for spurious cues—annotation patterns leading to model overfitting. CTKFacts is further examined for inter-annotator agreement, thoroughly cleaned, and a typology of common annotator errors is extracted. Finally, we provide baseline models for all stages of the fact-checking pipeline and publish the NLI datasets, as well as our annotation platform and other experimental data.

Causal Language Model Aided Sequential Decoding With Natural Redundancy

A high-performance communication receiver desires a sufficient and accurate recognition of transmitted sources. In this paper, we propose a sequential decoding algorithm for the robust reception of sources with natural redundancy (NR) over the AWGN channel. To fully exploit the abundant NR in the exemplified English text sources, a causal language modeling (CLM) with a powerful Transformer decoder neural network (NN) structure is adopted for modeling the joint probability distribution. For versatility of byte-level tokenization, the UTF-8 encoding scheme is considered for the Wikipedia corpus, namely the English edition of the Wiki-40B dataset. The tree search-based M-algorithm (MA) is integrated with CLM, denoted as CLM-MA algorithm, to synthesize the a priori probability of information sequence and likelihood values of polluted symbols. Both simulation experiments and hardware platform evaluations are conducted to analyze the performance of the CLM-MA algorithm. With an adequately large M value, tremendous performance profit is achievable, especially for high-efficiency modulation levels and extremely noisy conditions. This mechanism of adopting the pre-trained NN model merely desires self-supervised learning architecture and eliminates the requirement of high-cost and accurate labels, which paves a new way for communication receiver design.

On-screen keyboard controlled by gaze for Vietnamese people with amyotrophic lateral sclerosis

BACKGROUND: On-screen keyboard controlled by gaze is the most useful approach to help people with Amyotrophic Lateral Sclerosis (ALS) communicate. However, the design of the traditional on-screen keyboard hinders gaze typing for languages other than English. OBJECTIVE: The purpose of this paper is developing a specialized on-screen keyboard controlled by gaze for Vietnamese people with ALS. METHODS: Designed on-screen keyboard layouts and selection mechanism based on the requirements of eye typing and Vietnamese language characteristics from the statistics results on the Vietnamese Wikipedia Corpus dataset. During the experiment, we also absorbed and modified some design features according to the users’ suggestions. RESULTS: The authors proposed two new specialized on-screen keyboard layouts controlled by eye gaze for the Vietnamese language which both speed up typing and reduce typos. The results of the experiment are an average typing speed of respectively 23.1 and 26.1 characters per minute, and an average error rate of 1.0% and 0.5% for the two layouts. All participants were satisfied when using the system at their house. CONCLUSIONS: Although the effectiveness of the proposed method depends on the time of use and the familiarity of users with the new keyboard, the potential of the proposed method is great.

LSTM (Long Short Term Memory) for Sentiment COVID-19 Vaccine Classification on Twitter

The implementation of the Covid-19 vaccination carried out by Indonesian government was ignited pros and contras among the public. Certainly, there will be pros and cons about the vaccination from the community. This attituded of pros and cons, which is also called sentiment, can influence people to accept or refuse to be vaccinated. Todays, people express their sentiment in social media in comments, post, or status. One of the methods used to detect sentiment on social media, whether positive or negative, is through a categorisation of text approach. This research provides a deep learning technique for sentiment classification on Twitter that uses Long Short Term Memory (LSTM), for positive, neutral and negative classes. The word2vec word embeddings was used as input, using the pretrained Bahasa Indonesia model from Wikipedia corpus. On the other hand, the topic-based word2vec model was also trained from the Covid-19 vaccination sentiment dataset which collected from Twitter. The data used after balanced is 2564 training data, 778 data validation data, and 400 test data with 1802 neutral data, 1066 negative data, and 566 positive data. The best results from various parameter processes give an F1-Score value of 54% on the test data, with an accuracy of 66%. The result of this research is a model that can classify sentiments with new sentences.

Implementation of Deep Learning to Detect Indonesian Hoax News with Convolutional Neural Network Method

This study aims to establish and test a model that is used to determine valid news and hoax news. The method used is the Convolutional Neural Network (CNN) method and Word2Vec as embeddings. The research stages consist of data collection, pre-processing, word embeddings, model formation and testing the results obtained. The data used is 958 news. After testing with the distribution of data by 80% as training data and 20% as test data and 5 times epoch, the model that has been formed can determine valid news and hoax news well. In this study, a model with a vector dimension of 400 as input data and a multiple filter size of 3,4,5 became the best model. The resulting accuracy, precision and recall are 0.91. These results are influenced by the selection of the size of the vector dimensions on the output of Word2Vec, the selection of the filter size on the convolution layer and the addition of the Indonesian Wikipedia corpus into the corpus used.

BioTAGME: A Comprehensive Platform for Biological Knowledge Network Analysis.

The inference of novel knowledge and new hypotheses from the current literature analysis is crucial in making new scientific discoveries. In bio-medicine, given the enormous amount of literature and knowledge bases available, the automatic gain of knowledge concerning relationships among biological elements, in the form of semantically related terms (or entities), is rising novel research challenges and corresponding applications. In this regard, we propose BioTAGME, a system that combines an entity-annotation framework based on Wikipedia corpus (i.e., TAGME tool) with a network-based inference methodology (i.e., DT-Hybrid). This integration aims to create an extensive Knowledge Graph modeling relations among biological terms and phrases extracted from titles and abstracts of papers available in PubMed. The framework consists of a back-end and a front-end. The back-end is entirely implemented in Scala and runs on top of a Spark cluster that distributes the computing effort among several machines. The front-end is released through the Laravel framework, connected with the Neo4j graph database to store the knowledge graph.

Sentiment Analysis of COVID-19 Vaccines in Indonesia on Twitter Using Pre-Trained and Self-Training Word Embeddings

Sentiment analysis regarding the COVID-19 vaccine can be obtained from social media because users usually express their opinions through social media. One of the social media that is most often used by Indonesian people to express their opinion is Twitter. The method used in this research is Bidirectional LSTM which will be combined with word embedding. In this study, fastText and GloVe were tested as word embedding. We created 8 test scenarios to inspect performance of the word embeddings, using both pre-trained and self-trained word embedding vectors. Dataset gathered from Twitter was prepared as stemmed dataset and unstemmed dataset. The highest accuracy from GloVe scenario group was generated by model which used self-trained GloVe and trained on unstemmed dataset. The accuracy reached 92.5%. On the other hand, the highest accuracy from fastText scenario group generated by model which used self-trained fastText and trained on stemmed dataset. The accuracy reached 92.3%. In other scenarios that used pre-trained embedding vector, the accuracy was quite lower than scenarios that used self-trained embedding vector, because the pre-trained embedding data was trained using the Wikipedia corpus which contains standard and well-structured language while the dataset used in this study came from Twitter which contains non-standard sentences. Even though the dataset was processed using stemming and slang words dictionary, the pre-trained embedding still can not recognize several words from our dataset.

Syntactic variation and interactional coherence in online communication: The German conjunction “weil” in written interactions

Abstract The current study presents an analysis of a linguistic device supporting interactional coherence in computer-mediated communication (CMC). The analysis focused on the use of the German causative conjunction weil and, more specifically, on the syntactic variation of verb-final versus verb-second word order in the weil-clause. Verb-final word order in German indicates a dependent clause, whereas verb-second word order indicates an independent clause. While only the former is accepted in written Standard German, the latter is well documented in oral communication. Relying on freely available Facebook, WhatsApp, and Wikipedia corpus data, the analysis shows that both grammatical constructions are used in written CMC. In a specific usage across discrete messages, the syntactic variation of the weil-clause can serve to unambiguously identify its reference clause. While verb-final weil-clauses can be used to refer to previous messages by both an interlocutor and oneself, verb-second weil-clauses are predominantly used to relate to an own previous message. Thus, the syntactic variation in weil-clauses can be used for disambiguating references in written interactions.

Word embeddings trained on published case reports are lightweight, effective for clinical tasks, and free of protected health information

ObjectiveQuantify tradeoffs in performance, reproducibility, and resource demands across several strategies for developing clinically relevant word embeddings. Materials and methodsWe trained separate embeddings on all full-text manuscripts in the Pubmed Central (PMC) Open Access subset, case reports therein, the English Wikipedia corpus, the Medical Information Mart for Intensive Care (MIMIC) III dataset, and all notes in the University of Pennsylvania Health System (UPHS) electronic health record. We tested embeddings in six clinically relevant tasks including mortality prediction and de-identification, and assessed performance using the scaled Brier score (SBS) and the proportion of notes successfully de-identified, respectively. ResultsEmbeddings from UPHS notes best predicted mortality (SBS 0.30, 95% CI 0.15 to 0.45) while Wikipedia embeddings performed worst (SBS 0.12, 95% CI −0.05 to 0.28). Wikipedia embeddings most consistently (78% of notes) and the full PMC corpus embeddings least consistently (48%) de-identified notes. Across all six tasks, the full PMC corpus demonstrated the most consistent performance, and the Wikipedia corpus the least. Corpus size ranged from 49 million tokens (PMC case reports) to 10 billion (UPHS). DiscussionEmbeddings trained on published case reports performed as least as well as embeddings trained on other corpora in most tasks, and clinical corpora consistently outperformed non-clinical corpora. No single corpus produced a strictly dominant set of embeddings across all tasks and so the optimal training corpus depends on intended use. ConclusionEmbeddings trained on published case reports performed comparably on most clinical tasks to embeddings trained on larger corpora. Open access corpora allow training of clinically relevant, effective, and reproducible embeddings.

Word Sense Disambiguation Method Based on Graph Model and Word Vector

As a basic research of natural language processing, word sense disambiguation (WSD) has a very important influence on machine translation, classification tasks, retrieval tasks, etc. In order to solve the problem that existing disambiguation methods rely too much on knowledge base, a disambiguation method combining graph model and word vector is proposed in this paper. Firstly, in this method, the text data are preprocessed by removing punctuation marks and segmenting words. Secondly, the dependency relation is extracted by using the tool of PYLTP for dependency parsing, the words of dependency parent node are matched and the undirected graph is built, and the context knowledge of ambiguous words is selected according to the minimum path length set by the graph model. Finally, Word2Vec model is used to train Wikipedia corpus to obtain word vectors containing ambiguous words and contextual knowledge, and calculate the cross similarity of the word vector, the high mean similarity is regarded as the correct meaning of the ambiguous word. The effectiveness of the proposed method is verified by comparative experiments on the SEVAL-2007 Task# 5 dataset.

Transfer Learning for Spam Text Classification

Today’s world is mostly data-driven. To deal with the humongous amount of data, Machine Learning and Data Mining strategies are put into usage. Traditional ML approaches presume that the model is tested on a dataset extracted from the same domain from where the training data has been taken from. Nevertheless, some real-world situations require machines to provide good results with very little domain-specific training data. This creates room for the development of machines that are capable of predicting accurately by being trained on easily found data. Transfer Learning is the key to it. It is the scientific art of applying the knowledge gained while learning a task to another task that is similar to the previous one in some or another way. This article focuses on building a model that is capable of differentiating text data into binary classes; one roofing the text data that is spam and the other not containing spam using BERT’s pre-trained model (bert-base-uncased). This pre-trained model has been trained on Wikipedia and Book Corpus data and the goal of this paper is to highlight the pre-trained model’s capabilities to transfer the knowledge that it has learned from its training (Wiki and Book Corpus) to classifying spam texts from the rest.

Debunking health fake news with domain specific pre-trained model

During this covid pandemic it is clearer than ever how much health misinformation effects. It is much easier now to publish health related articles online without validation, these articles are shared across social media contributing to the spread of health fake news. This Health fake news are spread with intent to damage image of person or product, to increase sells of a product or to promote a product. In recent research papers, many useful health misinformation detection models use BERT (Bidirectional Encoder Representations from Transformers) which is pretrained on unlabeled data extracted from English Wikipedia and book corpus and are mostly dealt with health misinformation on social media. Therefore, a self - ensemble SCIBERT (Scientific BERT) based model that makes use of domain specific word embeddings is proposed for detection of health misinformation specifically in news which is less explored and a dataset combining existing FakeHealth dataset and custom dataset that contains health articles scraped from news fact checking website Snopes.com. Classification results exhibits that the proposed model provides weighted F1 score of 0.715.

An unsupervised method for word sense disambiguation

Word sense disambiguation (WSD) finds the actual meaning of a word according to its context. This paper presents a novel WSD method to find the correct sense of a word present in a sentence. The proposed method uses both the WordNet lexical dictionary and the Wikipedia corpus. Initially, we find all the probable senses of the target word using WordNet. For each of the words present in a sense, we calculate the collocation extraction score with the other words in the sentence. The collocation extraction score finds the probability of the occurrence of two words together in the Wikipedia corpus. The maximum collocation extraction score assigns the proper sense for that context of the sentence. Our method is not limited to the bi-grams that are made up of only two consecutive words. Our method can find the probability of having two words together in a sentence when other words separate these two words. To compare our WSD method with current knowledge-based unsupervised and supervised systems, we use different Senseval and SemEval datasets for doing WSD on English words. Finally, the experimental analysis illustrates the significance of the proposed approach over many baseline and current systems.

English–Welsh Cross-Lingual Embeddings

Cross-lingual embeddings are vector space representations where word translations tend to be co-located. These representations enable learning transfer across languages, thus bridging the gap between data-rich languages such as English and others. In this paper, we present and evaluate a suite of cross-lingual embeddings for the English–Welsh language pair. To train the bilingual embeddings, a Welsh corpus of approximately 145 M words was combined with an English Wikipedia corpus. We used a bilingual dictionary to frame the problem of learning bilingual mappings as a supervised machine learning task, where a word vector space is first learned independently on a monolingual corpus, after which a linear alignment strategy is applied to map the monolingual embeddings to a common bilingual vector space. Two approaches were used to learn monolingual embeddings, including word2vec and fastText. Three cross-language alignment strategies were explored, including cosine similarity, inverted softmax and cross-domain similarity local scaling (CSLS). We evaluated different combinations of these approaches using two tasks, bilingual dictionary induction, and cross-lingual sentiment analysis. The best results were achieved using monolingual fastText embeddings and the CSLS metric. We also demonstrated that by including a few automatically translated training documents, the performance of a cross-lingual text classifier for Welsh can increase by approximately 20 percent points.

Evaluating Complexity of Digital Learning in a Multilingual Context: A Cross-Linguistic Study on WHO's Emergency Learning Platform.

Reproduction of knowledge, especially tacit knowledge can be expensive during a pandemic. One of the most common causes is the reduced information accessibility during the translation process. Having the ability to assess the linguistic complexity of any given contents could potentially improve knowledge reproduction. Authors conduct two cross-linguistic studies on the World Health Organization (WHO)'s emergency learning platform to assess the linguistic complexity of two online courses in 10 languages. Morpho-syntactically annotated treebanks, unannotated materials from Wikipedia and language-specific corpora are set as control groups. Preliminary findings reveal a clear reduced complexity of learning contents in the most candidate languages while retaining the maximum amount of information. Creating a baseline study on low-resourced languages on the learning genre could be potentially useful for measuring impact of normative products at country and local level.