Evaluation of SVM, RVM and SMLR for Accurate Image Classification With Limited Ground Data

Abstract

The accuracy of a conventional supervised classification is in part a function of the training set used, notably impacted by the quantity and quality of the training cases. Since it can be costly to acquire a large number of high quality training cases, recent research has focused on methods that allow accurate classification from small training sets. Previous work has shown the potential of support vector machine (SVM) based classifiers. Here, the potential of the relevance vector machine (RVM) and sparse multinominal logistic regression (SMLR) approaches is evaluated relative to SVM classification. With both airborne and spaceborne multispectral data sets, the RVM and SMLR were able to derive classifications of similar accuracy to the SVM but required considerably fewer training cases. For example, from a training set comprising 600 cases acquired with a conventional stratified random sampling design from an airborne thematic mapper (ATM) data set, the RVM produced the most accurate classification, 93.75%, and needed only 7.33% of the available training cases. In comparison, the SVM yielded a classification that had an accuracy of 92.50% and needed 4.5 times more useful training cases. Similarly, with a Landsat ETM+ (Littleport, Cambridgeshire, UK) data set, the SVM required 4.0 times more useful training cases than the RVM. For each data set, however, the classifications derived by each classifier were of similar magnitude, differing by no more than 1.25%. Finally, for both the ATM and ETM+ (Littleport) data sets, the useful training cases by SVM and RVM had distinct and potentially predictable characteristics. Support vectors were generally atypical but lay in the boundary region between classes in feature space while the relevance vectors were atypical but anti-boundary in nature. The SMLR also tended to mostly, but not always, use extreme cases that lay away from class boundary. The results, therefore, suggest a potential to design classifier-specific intelligent training data acquisition activities for accurate classification from small training sets, especially with the SVM and RVM.