Image Classification Based on the Combination of Text Features and Visual Features

Abstract

With more and more text-image co-occurrence data becoming available on the Web, we are interested in how text especially Chinese context around images can aid image classification. The goal is to construct a classification system for images, and we used the context of the images to improve the classification system. First, we extracted three kinds of features, including global visual features, local visual features, and text features using both the image content and context. Then, we tried various feature combination methods and train classifiers for each kind of feature vector. Finally, we used a classifier fusion strategy based on weight learning, combining classifier outputs together, and we obtained the category of unlabeled images. In our experiments on the data set extracted from Google Image Search, we demonstrated the benefit of using context to help image classification. By comparing different feature combination methods on our feature set, we adopted the most effective one. Meanwhile, the classifier fusion approach improves the classification accuracy.