Effect Analysis of Dimension Reduction on Support Vector Machines

Abstract

Support vector machines (SVMs) are extensively used for text categorization, and dimension reduction is optional, not imperative for SVMs. But in some time-critical applications, dimension reduction of feature space is still necessary. In this paper, universal dimension reduction methods: feature selection and feature extraction, are applied to SVMs. At the same time, we also examine the influence of different kernel functions on the performance of dimension reduction. In the feature selection case, experimental results show that when the linear kernel is used for SVMs, the performance is close to the baseline system, sometimes even better, and when nonlinear kernel is employed, feature selection methods get the performance decrease sharply. On the contrary, principal component analysis (PCA), one of feature extraction methods, gets excellent performance with both linear and nonlinear kernel functions.