Design of a modified one-against-all SVM classifier

Abstract

Support Vector Machine (SVM) is one of the state-of-the-art tools for linear and nonlinear pattern classification. One of the design issues in SVM classifier is reducing the number of support vectors without compromising the classification accuracy. In this paper, a novel technique which requires only a subset of the support vectors is proposed. The subset is obtained by including only those support vectors for which Lagrange multiplier is greater than a threshold. In order to find the subset which yields the highest classification accuracy with the least number of support vectors in the subset, the recognition performance corresponding to subsets with different threshold values are to be evaluated and compared. The proposed technique is applied for SVM based isolated digit recognition system and is studied using speaker dependent as well as multispeaker dependent TI46 database of isolated digits. Two feature extraction techniques, one using LPC and another using MFCC are applied to the speech from the above database and the features are mapped using SOFM. This in turn is used by the SVM classifier to evaluate the recognition accuracy. The proposed technique is applied to one-against-all (OAA) scheme and is denoted as modified one-against-all (M-OAA) approach in this paper. Based on this study, it is found that for MFCC feature input, the proposed M-OAA based SVM classifier approach results in reduction of support vectors by a factor of 1.86 to 18.3 with no compromise in recognition accuracy. For LPC feature input, the M-OAA based SVM classifier results in reduction of support vectors by a factor of 1.59 to 2.52 without any compromise in recognition accuracy for some cases and with a maximum of 1% degradation in recognition accuracy for some cases. The proposed approach is also applicable for other schemes such as half-against-half (HAH) and directed acyclic graphs (DAG) based SVM classifiers as well as for any other classification problem such as face recognition, fingerprint recognition, target recognition, speaker recognition and speaker verification.