Stopword List Research Articles

Stop-Word Lists in Keyphrase Extraction: Their Influence and Comparison

Stop-Word Lists in Keyphrase Extraction: Their Influence and Comparison

Approaches to improve preprocessing for Latent Dirichlet Allocation topic modeling

As a part of natural language processing (NLP), the intent of topic modeling is to identify topics in textual corpora with limited human input. Current topic modeling techniques, like Latent Dirichlet Allocation (LDA), are limited in the pre-processing steps and currently require human judgement, increasing analysis time and opportunities for error. The purpose of this research is to allay some of those limitations by introducing new approaches to improve coherence without adding computational complexity and provide an objective method for determining the number of topics within a corpus. First, we identify a requirement for a more robust stop words list and introduce a new dimensionality-reduction heuristic that exploits the number of words within a document to infer importance to word choice. Second, we develop an eigenvalue technique to determine the number of topics within a corpus. Third, we combine all of these techniques into the Zimm Approach, which produces higher quality results than LDA in determining the number of topics within a corpus. The Zimm Approach, when tested against various subsets of the 20newsgroup dataset, produced the correct number of topics in 7 of 9 subsets vs. 0 of 9 using highest coherence value produced by LDA.

Threatening language detection from Urdu data with deep sequential model

The Urdu language is spoken and written on different social media platforms like Twitter, WhatsApp, Facebook, and YouTube. However, due to the lack of Urdu Language Processing (ULP) libraries, it is quite challenging to identify threats from textual and sequential data on the social media provided in Urdu. Therefore, it is required to preprocess the Urdu data as efficiently as English by creating different stemming and data cleaning libraries for Urdu data. Different lexical and machine learning-based techniques are introduced in the literature, but all of these are limited to the unavailability of online Urdu vocabulary. This research has introduced Urdu language vocabulary, including a stop words list and a stemming dictionary to preprocess Urdu data as efficiently as English. This reduced the input size of the Urdu language sentences and removed redundant and noisy information. Finally, a deep sequential model based on Long Short-Term Memory (LSTM) units is trained on the efficiently preprocessed, evaluated, and tested. Our proposed methodology resulted in good prediction performance, i.e., an accuracy of 82%, which is greater than the existing methods.

Occupational groups prediction in Turkish Twitter data by using machine learning algorithms with multinomial approach

A lot of research has been done on personality and sentiment analysis, demographic and professional aspects using user shares in social networks. In particular, information extraction and value are produced based on Twitter data. This study aims to predict the users, occupational groups, who share in Turkish on Twitter, using machine learning methods. First, occupational groups and the Twitter accounts of the occupations in these occupational groups were determined manually and the tweets shared in these accounts were scraped. All tweets were then grouped by occupation into groups of one, five and ten, creating datasets with different characteristics, each containing more than 500,000 tweets. Some datasets were preprocessed using the Zemberek library, which is used in many Turkish NLP studies, and experiments were conducted out with a total 6 datasets. During the preprocessing phase, since the ready-made stopwords lists were not considered sufficient, unnecessary word lists consisting of single and binary words were created manually. Count and TF-IDF vectorizers are used to convert textual data into numerical. Since each word represents a variable in the text classification study, new variables were created by combining double and triple word phrases (ngrams) with feature extraction. In the experiments in which 24 different models were run, instead of using all the features created, the method of “determining the optimal number of features”, which consists of the most valuable features, was used. It was found that the most successful model in the experiments using machine learning algorithms with a multinomial approach achieved 97.3% success in all calculated metrics.

A Systematic Study on the Dilemma and Innovative Path of Rural Family Education Development in the Context of Deep Learning

Abstract In order to better improve the level and quality of rural family education, this paper adopts text mining technology to interpret the relevant data of rural family education under the background of deep learning. On the basis of deep learning algorithms, web crawler technology is used to obtain rural family education research data. It is found that there are null values and repetitive values in the acquired research data, so it is necessary to further clean the research data. After washing, the combined Bi-LSTM-CRF algorithm is used to process the data for Chinese word segmentation, and at the same time, the stop word list is improved in order to achieve the best word segmentation effect. The LDA thematic model was used to determine the influencing factors of rural family education, and then the linear regression algorithm was used to analyze rural family education empirically. Rural teachers with different education levels were not significantly different in Q1 (1.608), Q2 (0.117), Q3 (2.019), Q4 (1.733), and Q5 (2.042), according to the results. The adjusted R2 is 0.133, indicating a fair fit between the model obtained from the regression analysis and the actual data, and the regression coefficient of family education is 0.411 after eliminating the confounding interference of grade and political appearance. This study purposefully proposes a path for the innovative development of rural family education, which can provide a theoretical reference program for the development and construction of modern rural family education.

Filtering Big Data with Optimized Hybrid Algorithm for IoT-Based Data Selection

Data management across servers has grown problematic because of technological advancements in data processing and storage capacities. Data that is neither organized nor labelled adds an additional layer of difficulty to the storing and retrieving processes. This data, which is not tagged, requires analytic techniques that are more powerful and time efficient. Clustering has long been regarded as one of the most effective methods for managing large amounts of data; nonetheless, larger volumes can lead to unexpectedly poor accuracy when using conventional clustering methodologies. In this study, we suggest the use of a novel framework for the clustering of large amounts of data. The preprocessing stage is one of the most important parts in the data cleansing process; hence, a global stop-word list is used to filter the contents of the files before sending them on to the cluster distribution stage. A meta-heuristic focused Genetic Algorithm (GA) is utilized to eradicate the redundant information present in the datasets. In addition to the generalized attributable fitness function, an attribute-based innovative fitness function (f) is being developed. To determine how well proposed method performs, it is compared to a variety of alternative clustering approaches. When comparing the distributions of clusters for the purpose of evaluation, the Standard Error (SE), root mean squared error (RMSE), and corrected R squared error are all computed.

LiHiSTO: a comprehensive list of Hindi stopwords

A preliminary preprocessing step in text analytics is the removal of words with no semantic meaning, otherwise known as stopwords. English stopwords are very easily accessible and created due to the broad usability of the English language. However, a standard list of Hindi stopwords is still missing. This paper proposes an exhaustive list of generic Hindi stopwords and a Python package for easy distribution and usage. The methodology uses a dual mechanism for creating a list of Hindi stopwords. First, the famous English stopwords are collected and translated into meaningful Hindi words (group 1). Second, unique Hindi stopwords from multiple sources are fetched (group 2). Finally, the respective Hindi stopwords from groups 1 and 2 are combined, which resulted in a significantly large set of 820 Hindi stopwords. Additionally, the list of Hindi stopwords is made openly available for use at the Python Package Index (PyPI) repository as a Python package, which is named LiHiSTO. With the help of illustrative implementations, it is shown that LiHiSTO provides abstract and easy access to the list of stopwords for users to perform Hindi text analytics.

A deep CNN architecture with novel pooling layer applied to two Sudanese Arabic sentiment data sets

Arabic sentiment analysis has become an important research field in recent years. Initially, work focused on Modern Standard Arabic (MSA), which is the most widely used form. Since then, work has been carried out on several different dialects, including Egyptian, Levantine and Moroccan. Moreover, a number of data sets have been created to support such work. However, up until now, no work has been carried out on Sudanese Arabic, a dialect which has 32 million speakers. In this article, two new public data sets are introduced, the two-class Sudanese Sentiment Data set (SudSenti2) and the three-class Sudanese Sentiment Data set (SudSenti3). In the preparation phase, we establish a Sudanese stopword list. Furthermore, a convolutional neural network (CNN) architecture, Sentiment Convolutional MMA (SCM), is proposed, comprising five CNN layers together with a novel Mean Max Average (MMA) pooling layer, to extract the best features. This SCM model is applied to SudSenti2 and SudSenti3 and shown to be superior to the baseline models, with accuracies of 92.25% and 85.23% (Experiments 1 and 2). The performance of MMA is compared with Max, Avg and Min and shown to be better on SudSenti2, the Saudi Sentiment Data set and the MSA Hotel Arabic Review Data set by 1.00%, 0.83% and 0.74%, respectively (Experiment 3). Next, we conduct an ablation study to determine the contribution to performance of text normalisation and the Sudanese stopword list (Experiment 4). For normalisation, this makes a difference of 0.43% on two-class and 0.45% on three-class. For the custom stoplist, the differences are 0.82% and 0.72%, respectively. Finally, the model is compared with other deep learning classifiers, including transformer-based language models for Arabic, and shown to be comparable for SudSenti2 (Experiment 5).

A study on corpus-based stopword lists in Indian language IR

We explore and evaluate the effect of different stopword lists (non-corpus-based and corpus-based) in the information retrieval (IR) tasks with different Indian languages such as Bengali, Marathi, Gujarati, Hindi and English. The issue was investigated from three viewpoints. Is there any performance difference between non-corpus-based and corpus-based stopword removal in chosen Indian languages? Can corpus-based stopword lists improve performance in Indian languages IR? If yes, to what extent? Among the different corpus-based stopword lists, which stopword list provides the best IR performance? Does the length of a corpus-based stopword list affect the retrieval performance in Indian languages? If yes, to what extent? It was observed that a corpus-based stopword list provides better retrieval performance than a non-corpus-based stopword list in different Indian languages. Among the different corpus-based stopword lists generated and experimented with, Zipf’s law-based stopword list (idf-based one) provides the best retrieval performance in various Indian languages. The aggregation1-based stopword list provides better retrieval than the aggregation2-based list in Indian languages, but in English, the aggregation2-based stopword list performs better than the aggregation1-based list. The best performing idf-based stopword list improves MAP score by 5.43% in Bengali, 1.91% in Marathi, 5.4% in Gujarati, 1.5% in Hindi and 2.12% in English, respectively, over their baseline counterparts. The probabilistic retrieval models (BM25 and TF-IDF) perform best in different Indian languages. A smaller length of corpus-based stopword lists performs better than a larger length of non-corpus-based stopword lists for all the Indian languages considered. The proposed schemes demonstrate that a stopword list can be heuristically generated in a language-independent statistical method and effectively used for IR tasks with performance comparable, to or even better than non-corpus-based stopword lists.

Comparing research trends through author-provided keywords with machine extracted terms: A ML algorithm approach using publications data on neurological disorders

Objective. This study aimed to identify the primary research areas, countries, and organizational involvement in publications on neurological disorders through an analysis of human-assigned keywords. These results were then compared with unsupervised and machine-algorithm-based extracted terms from the title and abstract of the publications to gain knowledge about deficiencies of both techniques. This has enabled us to understand how far machine-derived terms through titles and abstracts can be a substitute for human-assigned keywords of scientific research articles. Design/Methodology/Approach. While significant research areas on neurological disorders were identified from the author-provided keywords of downloaded publications of Web of Science and PubMed, these results were compared by the terms extracted from titles and abstracts through unsupervised based models like VOSviewer and machine-algorithm-based techniques like YAKE and CounterVectorizer. Results/Discussion. We observed that the post-covid-19 era witnessed more research on various neurological disorders, but authors still chose more generic terms in the keyword list than specific ones. The unsupervised extraction tool, like VOSviewer, identified many other extraneous and insignificant terms along with significant ones. However, our self-developed machine learning algorithm using CountVectorizer and YAKE provided precise results subject to adding more stop-words in the dictionary of the stop-word list of the NLTK tool kit. Conclusion. We observed that although author provided keywords play a vital role as they are assigned in a broader sense by the author to increase readability, these concept terms lacked specificity for in-depth analysis. We suggested that the ML algorithm being more compatible with unstructured data was a valid alternative to the author-generated keywords for more accurate results. Originality/Value. To our knowledge, this is the first-ever study that compared the results of author-provided keywords with machine-extracted terms with real datasets, which may be an essential lead in the machine learning domain. Replicating these techniques with large datasets from different fields may be a valuable knowledge resource for experts and stakeholders.

Civil aviation safety risk intelligent early warning model based on text mining and multi-model fusion

Aviation unsafe events often lead to major casualties and property losses. Aviation safety risk intelligent early warning is an important means to ensure the safe and reliable operation of aircraft. Therefore, an intelligent early warning model is urgently needed to quickly predict the risk level and identify potential risks to take targeted measures to realize the active management of safety. To realize the above process, the text mining method is used to extract key risk information from unsafe event reports and input it into the intelligent early warning model to predict its risk level, further constructing the priority processing index to achieve a rapid decision, and finally realize the intelligent safety management process of features extraction to early warning levels identification and then to priority processing. First, domain dictionary and Chinese stop word list are constructed to process the massive text data in the unsafe event’s report. Further, TF-IDF and TextRank are fused to extract key risk information and convert it into feature vectors. Second, the IHT algorithm is used to alleviate the sample class imbalance problem. After that, input the balanced risk information into an improved stacking multi-model fusion algorithm to accurately identify the early warning level and improve the level of active management and control via priority processing index ranking. The effectiveness and feasibility of the proposed method are demonstrated by testing the unsafe event text data of some aircraft maintenance companies and airlines, which promotes the practical application value of text mining technology in the aviation field.

A NEW COMPUTATIONAL MODEL FOR TURKIC LANGUAGES MORPHOLOGY AND PROCESSING

Effective communication between representatives of different nations in the modern global world has become a very relevant problem. Towards its solution, considerable support can come from artificial intelligence tools and, in particular, from natural language processing components. Along this direction, this article proposes the development and the exploitation of new computational morphology model for Turkic languages, based on a complete set of endings (CSE - model). Based on the CSE-model of morphology, a methodology has been developed for the creation and use of universal programs (data-driven) for processing natural languages. These include word stemming, text segmentation and morphological analysis. One advantage of the proposed methodology is that it is oriented towards linguists that only have to prepare i) a list of complete sets of endings for new languages ​​according to the described method, and ii) a list of stop words that do not have endings. Then, based on the prepared lists, the developed universal programs for stemming, segmentation, morphological analysis are used. Experiments carried out for the Kazakh, Kyrgyz and Uzbek languages ​​show a high efficiency of the proposed morphology model, algorithms and tools.

Impact of Similarity Measures in Graph-based Automatic Text Summarization of Konkani Texts

Automatic text summarization is a popular area in Natural Language Processing and Machine Learning. In this work, we adopt a graph-based text summarization approach, using PageRank algorithm, for automatically summarizing Konkani text documents. Konkani is an Indo-Aryan language spoken primarily in the state of Goa, which is on the west coast of India. It is a low-resource language with limited language processing tools. Such tools are readily available in other popular languages of choice for automatic text summarization, like English. The Konkani language dataset used for this purpose is based on Konkani folktales. We examine the impact of various language-independent and language-dependent similarity measures on the construction of the graph. The language-dependent similarity measures use pre-trained fastText word embeddings. A fully connected undirected graph is constructed for each document with the sentences represented as the graph's vertices. The vertices are connected to each other based on how strongly they are related to one another. Thereafter, PageRank algorithm is used for ranking the scores of the vertices. The top-ranking sentences are used to generate the summary. ROUGE toolkit was used for evaluating the quality of these system-generated summaries, and the performance was evaluated against human generated “gold-standard” abstracts and also compared with baselines and benchmark systems. The experimental results show that language-independent similarity measures performed well compared to language-dependent similarity measures despite not using language-specific tools, such as stop-words list, stemming, and word embeddings.

Enhancing relevant concepts extraction for ontology learning using domain time relevance

Concepts extraction is the backbone task for an ontology construction system. Identifying the relevant concepts is considered one of the main challenges of automatic ontology construction (called Ontology Learning (OL)). This research introduces a novel relevance metric that estimates the sustainability distribution of concepts to identify the modern and relevant concepts, called Domain Time Relevance (DTR). Also, this research proposes a Developed Concepts extraction Model based on the proposed DTR, namely DTR-DCEM. This proposed model uses the proposed Concepts extraction stopwords (CE-stopwords) list to avoid noise data. This proposed model, DTR-DCEM, aims to extract the relevant concepts from scientific publications. The experiments are conducted in two different datasets: DL2019 and ACLRelAcS. The experimental results show that the proposed model DTR-DCEM outperforms the state-of-the-art models. It got between (3.32∼51.07)% better performance than comparative models for the DL2019 dataset and between (3.29∼24.51)% better performance for the ACLRelAcS dataset. Moreover, the statistical significance of the proposed model DTR-DCEM has been proved using paired t-test.

SEA-PS: Semantic embedding with attention to measuring patent similarity by leveraging various text fields

Similarity metrics are critical to identifying the relationships between patents. While many bibliometric methods such as co-citation and co-classification fail to use the vast majority of technical information existing in the text, most text mining methods focus on keywords in only one text field of the patent document. This article aims to leverage various text fields to measure pairwise patent similarity according to their technological bases. A novel approach called semantic embedding with attention for patent similarity (SEA-PS) is proposed. First, the method identifies technological bases and models the semantic relatedness. To achieve this, we put forward an additional patent stop-word list to help extract technical terms with an n-gram-based statistical method. The technical terms are then mapped into a vector space using word embedding. Second, we propose a graph-based method to allocate weights to distinguish the technical focus, considering the linkages between technologies. Finally, we assess the feasibility of the text fields, and integrate their semantics at the patent-level with an attention layer to conduct similarity metrics. The validations are from two perspectives: content validity (coverage of technical information, the validity of semantic representations and effectiveness of text field combinations), and external validity against existing methods via an expert panel. The results demonstrate the superiority of SEA-PS to existing methods, and suggest that ‘abstracts’, ‘claims’ and ‘technical descriptions’ are more effective than ‘titles’. SEA-PS is a fundamental tool for patent retrieval and classification. It also has a broad range of practical applications in innovation and strategy studies, including identifying technological frontiers and studying knowledge spillovers.

Creation of a Russian Stop Word List

Creation of a Russian Stop Word List

A Study on the Teaching Design of a Hybrid Civics Course Based on the Improved Attention Mechanism

As an important vehicle for moral education, the moral indicators of civics and political science textbooks are naturally some of the most important criteria for revising textbooks. However, the textbook text dataset has too much textual information, ambiguous features, unbalanced sample distributions, etc. To address these problems, this paper combines a novel data enhancement method to obtain classification results based on word vectors. Additionally, for the problem of unbalanced sample sizes, this paper proposes a network model based on the attention mechanism, which combines the ideas of SMOTE and EDA, and uses a self-built stop word list and synonym word forest to conduct synonym queries, achieve a few categories of oversampling, and randomly disrupt the sentence order and intra-sentence word order to build a balanced dataset. The experimental results also show that the data augmentation method used in this paper’s model can effectively improve the performance of the model, resulting in a higher boost in the F1-measure of the model. The model incorporating the attention mechanism has better model generalization compared to the one without the attention mechanism, as well as a significant advantage compared to the reference model in other settings. The experimental results show that, compared with the original text classifier, the scheme of this paper effectively improves the evaluation effect and the reliability design for teaching a civics course.

DBTechVoc: A POS-tagged Vocabulary of Tokens and Lemmata of the Database Technical Domain

Vocabulary of a language has a great role to play in the Natural Language Processing (NLP) applications. Such applications make use of lists like stop-word list, general service list, academic word list and technical domain word list. The technical domain word list differs with each domain and though it is available for fields like medicine, biology, computer science, physics and law, the domain of databases in specific has still not been explored. For the first time, we propose technical vocabulary comprising of POS-tagged unigram tokens and POS-tagged unigram lemmata for the technical domain of databases. This vocabulary has been called DBTechVoc with a coined term. Notably, the multi-word phrases have also been considered, without their further tokenization, to maintain their semantics. The empirical results, with more than 1000 high quality research papers collected over a period of 45 years from 1976 to 2021, prove that the technical general word list of the domain of computer science is different from the technical and specific word list of the domain of databases. The overlap was found to be less than 2%. The research titles use 6% Rainbow stop words while 13% of the words used for the research paper titles are inflectional forms of lemmata.

Technology theme mining of integrated circuit manufacturing industry chain based on patents

Excavate and compare the technical themes of IC manufacturing industry chain of domestic and foreign patents, so as to objectively and comprehensively reveal the technical themes and differences of IC manufacturing in domestic and foreign, hoping to provide valuable reference for domestic IC manufacturing. Based on the integrated circuit manufacturing process, established the patent retrieval formula of each process and retrieved the patent data. Through TF-IDF algorithm and LDA model to mine the technical topics of domestic and foreign patents on the basis of self-built thesaurus and stop-words list, and summarized mining results. Through the analysis of results, it is found that the technical topics of integrated circuit manufacturing in domestic and foreign have similarities and differences, and the upper concept is strong, but the technical topics are more detailed of foreign patents.

Creation of a Russian stop-word list

Creation of a Russian stop-word list