Combining bag of visual words-based features with CNN in image classification

Abstract

Abstract Although traditional image classification techniques are often used in authentic ways, they have several drawbacks, such as unsatisfactory results, poor classification accuracy, and a lack of flexibility. In this study, we introduce a combination of convolutional neural network (CNN) and support vector machine (SVM), along with a modified bag of visual words (BoVW)-based image classification model. BoVW uses scale-invariant feature transform (SIFT) and Oriented Fast and Rotated BRIEF (ORB) descriptors; as a consequence, the SIFT–ORB–BoVW model developed contains highly discriminating features, which enhance the performance of the classifier. To identify appropriate images and overcome challenges, we have also explored the possibility of utilizing a fuzzy Bag of Visual Words (BoVW) approach. This study also discusses using CNNs/SVM to improve the proposed feature extractor’s ability to learn more relevant visual vocabulary from the image. The proposed technique was compared with classic BoVW. The experimental results proved the significant enhancement of the proposed technique in terms of performance and accuracy over state-of-the-art models of BoVW.