A group lasso based sparse KNN classifier

Abstract

Sparse features have been shown effective in applications of computer vision, machine learning, signal processing, etc. Group sparsity was proposed by considering that there exist natural group structures in many problems. Previous research mainly focuses on improving the way to extract sparse features, such as lasso, group lasso, overlapped group lasso, sparse group lasso. In existing work, sparse features are usually taken as input for classifiers, such as SVM, KNN, or SRC (Sparse Representation based Classification). In this paper, we find that, instead of using sparse group features as input for classifiers, sparse group features are good candidates for selection of most relevant classes/groups. We design a new classifier to improve classification accuracy: (1) we use sparse group lasso to select K most relevant classes/groups, which makes this approach robust, because it filters out unrelated classes/groups in group level, instead of individual sample level; (2) KSVD is used to get exact desired sparsity (k nonzero entries) and thus eliminates the difficulty of hyperparameter tuning; (3) simple summation of regression weights within each class/group contains sufficient class discriminant information, and the chance of a sample belonging to a specific class is denoted simply by the summation of corresponding regression weights within each class, which is in line with the need of Explainable AI (XAI). The K most relevant groups/classes can be considered as K neighbors of the correct class. Thus, we call this classifier Group Lasso based Sparse KNN (GLSKNN). Compared to 8 other approaches, GLSKNN classifier outperforms other methods in term of classification accuracy for two public image datasets and images with different occlusion/noise levels.