Stop Word Filtering Research Articles

NAKNN: An Efficient Classification of Indonesian News Texts with Nazief-Adriani and KNN

Internet usage in Indonesia has seen a significant increase, reaching 215.63 million users in 2022-2023, or 78.19% of the population. With the ease of internet access, digital news portals like Narasi TV have become a primary source of information for many people. However, the large number of news articles makes manual categorizing challenging. This study aims to classify Indonesian-language news documents from Narasi TV using the Nazief-Adriani algorithm for stemming and the K-Nearest Neighbor (KNN) method for classification. The text mining process begins with preprocessing, which includes case folding, tokenizing, stop-word filtering, and stemming. Using a dataset of 500 news documents, the study demonstrated that with a 90:10 data split, the average accuracy reached 93%, with the highest value being 100%. For the 80:20 data split, the average accuracy was 89%, with the highest value being 93%, and for a 70:30 data split, the average accuracy was 87%, with the highest value being 89%. In conclusion, the combination of the Nazief-Adriani algorithm and the KNN method with optimal k selection and random states obtained high accuracy, obtaining an average accuracy of 93%) in classifying Indonesian-language news documents. These results demonstrate the significant potential of text mining and classification techniques to manage digital news.

DBTagger

Translating Natural Language Queries (NLQs) to Structured Query Language (SQL) in interfaces deployed in relational databases is a challenging task, which has been widely studied in database community recently. Conventional rule based systems utilize series of solutions as a pipeline to deal with each step of this task, namely stop word filtering, tokenization, stemming/lemmatization, parsing, tagging, and translation. Recent works have mostly focused on the translation step overlooking the earlier steps by using adhoc solutions. In the pipeline, one of the most critical and challenging problems is keyword mapping; constructing a mapping between tokens in the query and relational database elements (tables, attributes, values, etc.). We define the keyword mapping problem as a sequence tagging problem, and propose a novel deep learning based supervised approach that utilizes POS tags of NLQs. Our proposed approach, called DBTagger (DataBase Tagger), is an end-to-end and schema independent solution, which makes it practical for various relational databases. We evaluate our approach on eight different datasets, and report new state-of-the-art accuracy results, 92.4% on the average. Our results also indicate that DBTagger is faster than its counterparts up to 10000 times and scalable for bigger databases.

Using Natural Language Processing to Examine the Uptake, Content, and Readability of Media Coverage of a Pan-Canadian Drug Safety Research Project: Cross-Sectional Observational Study.

BackgroundIsotretinoin, for treating cystic acne, increases the risk of miscarriage and fetal abnormalities when taken during pregnancy. The Health Canada–approved product monograph for isotretinoin includes pregnancy prevention guidelines. A recent study by the Canadian Network for Observational Drug Effect Studies (CNODES) on the occurrence of pregnancy and pregnancy outcomes during isotretinoin therapy estimated poor adherence to these guidelines. Media uptake of this study was unknown; awareness of this uptake could help improve drug safety communication.ObjectiveThe aim of this study was to understand how the media present pharmacoepidemiological research using the CNODES isotretinoin study as a case study.MethodsGoogle News was searched (April 25-May 6, 2016), using a predefined set of terms, for mention of the CNODES study. In total, 26 articles and 3 CNODES publications (original article, press release, and podcast) were identified. The article texts were cleaned (eg, advertisements and links removed), and the podcast was transcribed. A dictionary of 1295 unique words was created using natural language processing (NLP) techniques (term frequency-inverse document frequency, Porter stemming, and stop-word filtering) to identify common words and phrases. Similarity between the articles and reference publications was calculated using Euclidian distance; articles were grouped using hierarchical agglomerative clustering. Nine readability scales were applied to measure text readability based on factors such as number of words, difficult words, syllables, sentence counts, and other textual metrics.ResultsThe top 5 dictionary words were pregnancy (250 appearances), isotretinoin (220), study (209), drug (201), and women (185). Three distinct clusters were identified: Clusters 2 (5 articles) and 3 (4 articles) were from health-related websites and media, respectively; Cluster 1 (18 articles) contained largely media sources; 2 articles fell outside these clusters. Use of the term isotretinoin versus Accutane (a brand name of isotretinoin), discussion of pregnancy complications, and assignment of responsibility for guideline adherence varied between clusters. For example, the term pregnanc appeared most often in Clusters 1 (14.6 average times per article) and 2 (11.4) and relatively infrequently in Cluster 3 (1.8). Average readability for all articles was high (eg, Flesch-Kincaid, 13; Gunning Fog, 15; SMOG Index, 10; Coleman Liau Index, 15; Linsear Write Index, 13; and Text Standard, 13). Readability increased from Cluster 2 (Gunning Fog of 16.9) to 3 (12.2). It varied between clusters (average 13th-15th grade) but exceeded the recommended health information reading level (grade 6th to 8th), overall.ConclusionsMedia interpretation of the CNODES study varied, with differences in synonym usage and areas of focus. All articles were written above the recommended health information reading level. Analyzing media using NLP techniques can help determine drug safety communication effectiveness. This project is important for understanding how drug safety studies are taken up and redistributed in the media.

Balanced undersampling: a novel sentence-based undersampling method to improve recognition of named entities in chemical and biomedical text

The class imbalance problem is a key factor that affects the performance of many classification tasks when using machine learning methods. This mainly refers to the problem where the number of samples in certain classes is much greater than in others. Such imbalance considerably affects the performance of classifiers in which the majority class or classes are often favored, thus resulting in high-precision/low-recall classifiers. Named entity recognition in free text suffers from this problem to a large extent because in any given free text, many samples do not belong to a specific entity. Furthermore, the data used in this specific type of classification is in sequenced mode and is different than that used in other common classification tasks such as image classification, spam detection, and text classification in which no semantic or syntactic relation exists between samples. In this study, we propose an undersampling approach for sequenced data that preserves existing correlations between sequenced samples that comprise sentences and thus improve the performance of classifiers. We call this method balanced undersampling (BUS). Considering the recent increased interest in the use of NER in the chemical and biomedical domains, the proposed method is developed and tested on four recent state-of-the-art corpora in these domains, including BioCreative IV ChemDNER, Bio-entity Recognition Challenge of JNLPBA (JNLPBA), SemEval2013 DDI DrugBank, and SemEval2013 DDI Medline datasets. The performance of the proposed method is evaluated against two other common undersampling methods: random undersampling and stop-word filtering. Our method is shown to outperform both methods with respect to F-score for all datasets used.

AuDis: an automatic CRF-enhanced disease normalization in biomedical text.

Diseases play central roles in many areas of biomedical research and healthcare. Consequently, aggregating the disease knowledge and treatment research reports becomes an extremely critical issue, especially in rapid-growth knowledge bases (e.g. PubMed). We therefore developed a system, AuDis, for disease mention recognition and normalization in biomedical texts. Our system utilizes an order two conditional random fields model. To optimize the results, we customize several post-processing steps, including abbreviation resolution, consistency improvement and stopwords filtering. As the official evaluation on the CDR task in BioCreative V, AuDis obtained the best performance (86.46% of F-score) among 40 runs (16 unique teams) on disease normalization of the DNER sub task. These results suggest that AuDis is a high-performance recognition system for disease recognition and normalization from biomedical literature.Database URL: http://ikmlab.csie.ncku.edu.tw/CDR2015/AuDis.html

The role of statistical and semantic features in single-document extractive summarization

This paper reports on the further results of the ongoing research analyzing the impact of a range of commonly used statistical and semantic features in the context of extractive text summarization. The features experimented with include word frequency, inverse sentence and term frequencies, stopwords filtering, word senses, resolved anaphora and textual entailment. The obtained results demonstrate the relative importance of each feature and the limitations of the tools available. It has been shown that the inverse sentence frequency combined with the term frequency yields almost the same results as the latter combined with stopwords filtering that in its turn proved to be a highly competitive baseline. To improve the suboptimal results of anaphora resolution, the system was extended with the second anaphora resolution module. The present paper also describes the first attempts of the internal document data representation.