Large-scale Image Classification Research Articles

Comparing HMAX and BoVW Models for Large-Scale Image Classification

Image classification is one of the most important topics in computer vision. It became crucial for large image datasets. In the literature, several image classification approaches are proposed. In this context, Bag-of-Visual Words (BoVW) model has been widely used. The BoVW model relies on building visual vocabulary and images are represented as histograms of visual words. However, recently, attention has been shifted to the use of complex architectures which are characterized by multilevel processing. HMAX (Hierarchical Max-pooling model) model has attracted a great deal of attention in image classification, due to its architecture, which alternates layers of feature extraction with layers of pooling. This paper aims at comparing bags of visual words model to HMAX model for image classification using large datasets. To achieve this goal, we study the use of image features obtained by BoVW model with SIFT (Scale-Invariant Feature Transform) descriptors, and we compare them to HMAX features. Image classification is performed by using the support vector machine (SVM) classifiers. Both HMAX and BoVW models are tested on ImageNet and OpenImages datasets and results have shown that the classification performance obtained by HMAX model outperforms the classification using BoVW model.

A Deep Multiview Active Learning for Large-Scale Image Classification

Multiview active learning (MAL) is a technique which can achieve a large decrease in the size of the version space than traditional active learning and has great potential applications in large-scale data analysis. In this paper, we present a new deep multiview active learning (DMAL) framework which is the first to combine multiview active learning and deep learning for annotation effort reduction. In this framework, our approach advances the existing active learning methods in two aspects. First, we incorporate two different deep convolutional neural networks into active learning which uses multiview complementary information to improve the feature learnings. Second, through the properly designed framework, the feature representation and the classifier can be simultaneously updated with progressively annotated informative samples. The experiments with two challenging image datasets demonstrate that our proposed DMAL algorithm can achieve promising results than several state-of-the-art active learning algorithms.

Segment-Based Clustering of Hyperspectral Images Using Tree-Based Data Partitioning Structures

Hyperspectral image classification has been increasingly used in the field of remote sensing. In this study, a new clustering framework for large-scale hyperspectral image (HSI) classification is proposed. The proposed four-step classification scheme explores how to effectively use the global spectral information and local spatial structure of hyperspectral data for HSI classification. Initially, multidimensional Watershed is used for pre-segmentation. Region-based hierarchical hyperspectral image segmentation is based on the construction of Binary partition trees (BPT). Each segmented region is modeled while using first-order parametric modelling, which is then followed by a region merging stage using HSI regional spectral properties in order to obtain a BPT representation. The tree is then pruned to obtain a more compact representation. In addition, principal component analysis (PCA) is utilized for HSI feature extraction, so that the extracted features are further incorporated into the BPT. Finally, an efficient variant of k-means clustering algorithm, called filtering algorithm, is deployed on the created BPT structure, producing the final cluster map. The proposed method is tested over eight publicly available hyperspectral scenes with ground truth data and it is further compared with other clustering frameworks. The extensive experimental analysis demonstrates the efficacy of the proposed method.

Cyclic CNN: Image Classification With Multiscale and Multilocation Contexts

Improving the capability of models at limited computational cost is an urgent demand in many vision-based Internet-of-Things applications. Recent progress on deep convolutional neural network (CNN) has largely accelerated the development of image classification. Although the hierarchical structure of CNN naturally helps to extract image features in different scales and locations progressively, conventional convolution can only handle contexts of one scale and on a limited area of a single location in a specific layer, limiting the utilization of multiscale and multilocation information. In this work, we present a cyclic CNN framework, which enables sufficient utilization of multiscale and multilocation contexts in a single layer of convolution. The cyclic CNN is an extremely simple but effective improvement upon conventional convolution, which occupies no additional parameter and negligible computation (even less than 0.1%). Moreover, cyclic CNN can be easily plugged into many existing CNN pipelines, e.g., the ResNet family, obtaining extremely low-cost performance gain upon them. Extensive experiments on both small-scale (CIFAR10 and CIFAR100) and large-scale (ILSVRC2012) image classification benchmarks demonstrate that a consistent performance promotion is obtained with the help of cyclic CNN.

Multiobjective Task Scheduling for Energy-Efficient Cloud Implementation of Hyperspectral Image Classification

Cloud computing has become a promising solution to efficient processing of remotely sensed big data, due to its high-performance and scalable computing capabilities. However, existing cloud solutions generally involve the problems of low resource utilization and high energy consumption when processing large-scale remote sensing datasets, affecting the quality-of-service of the cloud system. Aiming at hyperspectral image classification applications, this article proposes an energy-efficient cloud implementation by employing a multiobjective task scheduling algorithm. We first present a parallel computing mechanism for a fusion-based classification method based on Apache Spark. With the general classification flow represented by a workflow model, we formulate a multiobjective scheduling framework that jointly minimizes the total execution time as well as energy consumption. We further develop an effective scheduling algorithm to solve the multiobjective optimization problem and produce a set of Pareto-optimal solutions, providing the tradeoff between computational efficiency and energy efficiency. Experimental results demonstrate that the multiobjective scheduling approach proposed in this work can substantially reduce the execution time and energy consumption for performing large-scale hyperspectral image classification on Spark. In addition, our proposed algorithm can generate better tradeoff solutions to the multiobjective scheduling problem as compared to competing scheduling algorithms.

CEB Improves Model Robustness.

Intuitively, one way to make classifiers more robust to their input is to have them depend less sensitively on their input. The Information Bottleneck (IB) tries to learn compressed representations of input that are still predictive. Scaling up IB approaches to large scale image classification tasks has proved difficult. We demonstrate that the Conditional Entropy Bottleneck (CEB) can not only scale up to large scale image classification tasks, but can additionally improve model robustness. CEB is an easy strategy to implement and works in tandem with data augmentation procedures. We report results of a large scale adversarial robustness study on CIFAR-10, as well as the ImageNet-C Common Corruptions Benchmark, ImageNet-A, and PGD attacks.

Are open set classification methods effective on large-scale datasets?

Supervised classification methods often assume the train and test data distributions are the same and that all classes in the test set are present in the training set. However, deployed classifiers often require the ability to recognize inputs from outside the training set as unknowns. This problem has been studied under multiple paradigms including out-of-distribution detection and open set recognition. For convolutional neural networks, there have been two major approaches: 1) inference methods to separate knowns from unknowns and 2) feature space regularization strategies to improve model robustness to novel inputs. Up to this point, there has been little attention to exploring the relationship between the two approaches and directly comparing performance on large-scale datasets that have more than a few dozen categories. Using the ImageNet ILSVRC-2012 large-scale classification dataset, we identify novel combinations of regularization and specialized inference methods that perform best across multiple open set classification problems of increasing difficulty level. We find that input perturbation and temperature scaling yield significantly better performance on large-scale datasets than other inference methods tested, regardless of the feature space regularization strategy. Conversely, we find that improving performance with advanced regularization schemes during training yields better performance when baseline inference techniques are used; however, when advanced inference methods are used to detect open set classes, the utility of these combersome training paradigms is less evident.

Automated Visual Content Analysis (AVCA) in Communication Research: A Protocol for Large Scale Image Classification with Pre-Trained Computer Vision Models

ABSTRACT The increasing volume of images published online in a wide variety of contexts requires communication researchers to address this reality by analyzing visual content at a large scale. Ongoing advances in computer vision to automatically detect objects, concepts, and features in images provide a promising opportunity for communication research. We propose a research protocol for Automated Visual Content Analysis (AVCA) to enable large-scale content analysis of images. It offers inductive and deductive ways to use commercial pre-trained models for theory building in communication science. Using the example of corporations’ website images on sustainability, we show in a step-by-step fashion how to classify a large sample (N = 21,876) of images with unsupervised and supervised machine learning, as well as custom models. The possibilities and pitfalls of these approaches are discussed, ethical issues are addressed, and application examples for future communication research are detailed.

Visual interaction networks: A novel bio-inspired computational model for image classification

Inspired by biological mechanisms and structures in neuroscience, many biologically inspired visual computational models have been presented to provide new solutions for visual recognition task. For example, convolutional neural network (CNN) was proposed according to the hierarchical structure of biological vision, which could achieve superior performance in large-scale image classification. In this paper, we propose a new framework called visual interaction networks (VIN-Net), which is inspired by visual interaction mechanisms. More specifically, self-interaction, mutual-interaction, multi-interaction, and adaptive interaction are proposed in VIN-Net, forming the first interactive completeness of the visual interaction model. To further enhance the representation ability of visual features, the adaptive adjustment mechanism is integrated into the VIN-Net model. Finally, our model is evaluated on three benchmark datasets and two self-built textile defect datasets. The experimental results demonstrate that the proposed model exhibits its efficiency on visual classification tasks. Furthermore, a textile industrial application shows that the proposed architecture outperforms the state-of-the-art approaches in classification performance.

Contextual encoder–decoder network for visual saliency prediction

Predicting salient regions in natural images requires the detection of objects that are present in a scene. To develop robust representations for this challenging task, high-level visual features at multiple spatial scales must be extracted and augmented with contextual information. However, existing models aimed at explaining human fixation maps do not incorporate such a mechanism explicitly. Here we propose an approach based on a convolutional neural network pre-trained on a large-scale image classification task. The architecture forms an encoder–decoder structure and includes a module with multiple convolutional layers at different dilation rates to capture multi-scale features in parallel. Moreover, we combine the resulting representations with global scene information for accurately predicting visual saliency. Our model achieves competitive and consistent results across multiple evaluation metrics on two public saliency benchmarks and we demonstrate the effectiveness of the suggested approach on five datasets and selected examples. Compared to state of the art approaches, the network is based on a lightweight image classification backbone and hence presents a suitable choice for applications with limited computational resources, such as (virtual) robotic systems, to estimate human fixations across complex natural scenes. Our TensorFlow implementation is openly available at https://github.com/alexanderkroner/saliency.

Semi-Supervised Hyperspectral Image Classification Based on Label Propagation via Selected Path

Most graph-based semi-supervised classification methods do not perform well in hyperspectral image classification tasks due to their high complexity and other limitations. This paper proposes a label propagation semi-supervised classification algorithm which uses a few selected important paths to considerably reduce computational costs. It establishes that the most important paths for label propagation are minimum cost paths. It proves that minimum cost paths exist in minimum cost trees (MCT), and proposes a method based on a variant minimum spanning tree (MST) combined with priority queue to construct MCTs. The algorithm propagates labels from unlabeled nodes to labeled ones, a unique way different from any other studies where propagation is in the opposite direction, which brings about several clear advantages. These include that only one propagation path is required for each unlabeled node, improving both timing and memory performance. It also helps to solve a problem posed by sparse graphs where some image pixels cannot be classified, a situation which is especially problematic in large-scale image classification. The proposed method has the advantages of linear computational complexity, is independent of data dimension, has fewer parameters and is insensitive to the values of parameters. Moreover, it does not need large numbers of labelled pixels nor complex training processes. Experiments on hyperspectral images have shown that, compared with several existing algorithms, the proposed method achieves better performance in less time. The paper addresses some fundamental issues regarding propagating labels in graph based semi-supervised classifications. Due to the simplicity and the fast speed of the algorithm, it is also suitable to be integrated into both state-of-the-art and future hyperspectral image classification frameworks which have a label propagation stage.

Feature Selection-Based Hierarchical Deep Network for Image Classification

In this paper, a novel hierarchical deep network is proposed to combine the deep convolutional neural network and the feature selection-based tree classifier efficiently for image classification. First, the concept ontology is built for organizing large-scale image classes hierarchically in a coarse-to-fine fashion. Second, a novel selective orthogonal algorithm is proposed to make sure deep features extracted for each level classifiers more in line with the requirements of different classification tasks. Also, the role of useful feature components in multi-level deep features are improved. The experimental results on three datasets show that adding a feature selection module in a hierarchical deep network can perform better performance in large-scale image classification.

Multiple Lesions Detection of Fundus Images Based on Convolution Neural Network Algorithm With Improved SFLA

In order to effectively solve the problem of interlaced overlap in the fundus image lesions, large and small blood vessels packed densely and severely affected by light, and to achieve multi-label classification of fundus images. In this paper, a single population leapfrog optimization convolutional neural network algorithm (SFCNN) is proposed to detect and classify various fundus lesions. The algorithm uses the efficient search ability of the shuffled frog leaping algorithm to optimize the weight initialization and back propagation of the convolutional neural network. In order to deal with the problem of fundus image classification in the big data environment, the novel grouping optimization strategy is presented to effectively combine Spark platform and SFCNN algorithm to achieve large-scale fundus image classification and detection of multiple lesions. The experiment of the detection of fundus image lesions shows that the accuracy rate of SFCNN is better improved in both single lesion detection and overall detection, compared with other algorithms.

Large-scale multi-label classification using unknown streaming images

In this paper, we investigate the large-scale multi-label image classification problem when images with unknown novel classes come in stream during the training stage. It coincides with the practical requirement that usually novel classes are detected and used to update an existing image recognition system. Most existing multi-label image classification methods cannot be directly applied in this scenario, where the training and testing stages must have the same label set. In this paper, we proposed to learn a multi-label classifier and a novel-class detector alternately to solve this problem. The multi-label classifier is learned using a convolutional neural network (CNN) from the images in the known classes. We proposed a recurrent novel-class detector which is learned in the supervised manner to detect the novel class by encoding image features with the multi-label information. In the experiment, our method is evaluated on several large-scale multi-label benchmarks including MS COCO. The results show the proposed method is comparable to most existing multi-label image classification methods, which validate its efficacy when encountering streaming images with unknown classes.

Image Classification using Supervised Convolutional Neural Network

Deep learning algorithms, in particular Convolutional Neural Networks have made notable accomplishments in many large-scale image classification tasks in the past decade. In this paper, image classification is performed using Supervised Convolutional Neural Network (SCNN). In supervised learning model, algorithm learns on a labeled dataset. SCNN architecture is built with 15 layers viz, input layer, 9 middle layers and 5 final layers. Two datasets of different sizes are tested on SCNN framework on single CPU. With CIFAR10 dataset of 60000 images the network yielded an accuracy of 73% taking high processing time, while for 3000 images taken from MIO-TCD dataset resulted 96% accuracy with less computational time.

Multi-Precision Quantized Neural Networks via Encoding Decomposition of {-1,+1}

The training of deep neural networks (DNNs) requires intensive resources both for computation and for storage performance. Thus, DNNs cannot be efficiently applied to mobile phones and embedded devices, which seriously limits their applicability in industry applications. To address this issue, we propose a novel encoding scheme of using {−1, +1} to decompose quantized neural networks (QNNs) into multibranch binary networks, which can be efficiently implemented by bitwise operations (xnor and bitcount) to achieve model compression, computational acceleration and resource saving. Based on our method, users can easily achieve different encoding precisions arbitrarily according to their requirements and hardware resources. The proposed mechanism is very suitable for the use of FPGA and ASIC in terms of data storage and computation, which provides a feasible idea for smart chips. We validate the effectiveness of our method on both large-scale image classification tasks (e.g., ImageNet) and object detection tasks. In particular, our method with lowbit encoding can still achieve almost the same performance as its full-precision counterparts.

An Entropy and MRF Model-Based CNN for Large-Scale Landsat Image Classification

Large-scale Landsat image classification is essential for the production of land cover maps. The rise of convolutional neural networks (CNNs) provides a new idea for the implementation of Landsat image classification. However, pixels in Landsat images have higher uncertainty compared with high-resolution images due to its 30-m spatial resolution. In addition, the current deep learning methods tend to lose detailed information such as boundaries along with the stacking of convolutional and pooling layers. To solve these problems, we propose a new method called entropy and MRF model (EMM)-CNN based on Pyramid Scene Parsing Network. The EMM-CNN uses entropy to decrease the uncertainty of pixels. Then, the Markov random filed (MRF) model is employed to construct the connections between neighboring pixels and defined a prior distribution to prevent the cross entropy from sacrificing detailed information for the overall accuracy. Finally, transfer learning based on the pretrained ImageNet is introduced to overcome the shortage of training samples and boost the speed of the training process. Experimental results demonstrate that the proposed EMM-CNN is able to obtain classification results with fine structure by decreasing the uncertainty and retaining detailed information of the detected image.

Ontology-driven hierarchical sparse coding for large-scale image classification

An ontology-driven hierarchical sparse representation is developed in this paper, which aims to support hierarchical learning for large scale image classification. Firstly, a two-layer ontology (semantic ontology and visual ontology) is built to organize large number of image classes hierarchically, where WordNet is used to construct semantic ontology and deep features extracted by Inception V3 are used to construct visual ontology (visual tree). Secondly, a novel algorithm based on Split Bregman Iteration is developed to learn hierarchical sparse representation, i.e., learning a shared dictionary and a set of class-specified dictionaries depending on the two-layer ontology. For multi-class image classification, a tree classifier is trained according to the two-layer ontology by using the hierarchical sparse representation. Thirdly, for a given test image, multiple paths are simultaneously evaluated to achieve optimal prediction of its class label. Our proposed approach has been evaluated over three benchmark datasets: ILSVRC2010, SUN397, and Caltech256, and the experimental results have demonstrated that our approach is better than the original joint dictionary learning method and can achieve better accuracy compared with other approaches which use handcrafted features.

Learning multi-layer coarse-to-fine representations for large-scale image classification

Recent studies on large-scale image classification mainly focus on categorizing images into 1000 object classes, and all these 1000 object classes are atomic and mutually exclusive in the semantic space. However, for a much larger set of image categories (such as the ImageNet 10k dataset), some of them may come from the high-level (non-leaf) nodes of the concept ontology and could contain some other lower-level categories semantically. The research that classifies images into large numbers of image categories with such inter-category subsumption correlations has received rare attention. In this paper, a Visual-Semantic Tree is learned to organize 10k image categories hierarchically in a coarse-to-fine fashion, where both the inter-category visual similarities and inter-category semantic correlations are seamlessly integrated for tree construction. Additionally, a deep learning method is developed by integrating the Visual-Semantic Tree with deep CNNs to learn more discriminative tree classifiers for large-scale image classification. Our experimental results have demonstrated that the proposed Visual-Semantic Tree can effectively organize large-scale structural image categories and significantly boost the classification accuracy rates for both atomic image categories and high-level image categories.

Fast Spectral Clustering for Unsupervised Hyperspectral Image Classification

Hyperspectral image classification is a challenging and significant domain in the field of remote sensing with numerous applications in agriculture, environmental science, mineralogy, and surveillance. In the past years, a growing number of advanced hyperspectral remote sensing image classification techniques based on manifold learning, sparse representation and deep learning have been proposed and reported a good performance in accuracy and efficiency on state-of-the-art public datasets. However, most existing methods still face challenges in dealing with large-scale hyperspectral image datasets due to their high computational complexity. In this work, we propose an improved spectral clustering method for large-scale hyperspectral image classification without any prior information. The proposed algorithm introduces two efficient approximation techniques based on Nyström extension and anchor-based graph to construct the affinity matrix. We also propose an effective solution to solve the eigenvalue decomposition problem by multiplicative update optimization. Experiments on both the synthetic datasets and the hyperspectral image datasets were conducted to demonstrate the efficiency and effectiveness of the proposed algorithm.