Self Learning of News Category Using AI Techniques

Abstract

Numerous e-news channels publish the daily happenings in the world from different sources. These huge amounts of news articles have lamentably conceived the information overload issue among the users. Hence text mining, which aims in extracting previously unknown information from unstructured text, has been widely used by several researchers to segregate full news articles however, the news headlines categorization is still specifically limited. Therefore, considering this limitation, the current research aims to propose a framework that will self-learn and automatically classify any given news headline into its corresponding news category using artificial intelligence methods i.e. text mining and machine learning algorithms. The proposed framework consists of three stages: Exploratory Data Analysis, Text Pre-processing, and Text Classification. For exploratory data analysis, the top 10 most frequent balanced news categories are chosen so that further processing of data can be done on a more balanced version of the dataset. After exploring the data, text pre-processing techniques are applied to make the data transformed, normalized, and structured. Finally, text classification is carried out with two approaches: unsupervised classification using Mean Shift and K-means algorithms and supervised classification using Logistic Regression with Bag of Words and TF-IDF algorithm. To depict the working of the proposed framework, a case study is presented on a news headlines dataset which accurately performed news headlines classification.