Logistic Regression Matching Pursuit algorithm for text classification

Abstract

Text classification is a challenging problem due to the high dimensionality of the text, which can limit classification performance. The orthogonal matching pursuit (OMP) algorithm is one of the most popular sparse recovery algorithms. An OMP based text classification algorithm, called the Logistic-OMP algorithm, was recently proposed by Skianis et al.. Simulation tests indicate that Logistic-OMP has excellent performance in text dimensionality reduction. This paper optimizes the Logistic-OMP algorithm, and proposes a new text classification algorithm called the Logistic Regression Matching Pursuit (LRMP) algorithm. The LRMP algorithm defines a new loss function and residual update function. It requires only one iteration to solve the negative log likelihood minimization problem, and its classification performance is guaranteed by the strong Wolfe condition, which makes it optimizes the classification accuracy while significantly speeding up the training speed. Simulation tests on topic classification and sentiment analysis from 20Newsgroups, Amazon product reviews, and movie reviews datasets show that the LRMP algorithm has a shorter computation time of a single iteration than the Logistic-OMP algorithm, with a total training time of 8.08%–21.16% shorter than that of the Logistic-OMP algorithm, and the memory usage is 3.20%–6.21% lower than that of the Logistic-OMP algorithm. Furthermore, the average Accuracy and F1-Score of the LRMP algorithm are improved by 1.61%–26.55% and 1.83%–25.46%, respectively, compared with the benchmark classifier. Compared with the advanced classifiers (including Logistic-OMP), the average Accuracy and F1-Score of the LRMP algorithm are improved by 0.46%–8.97% and 0.57%–9.40%, respectively.