Caltech Dataset Research Articles

Attend and Guide (AG-Net): A Keypoints-Driven Attention-Based Deep Network for Image Recognition.

This article presents a novel keypoints-based attention mechanism for visual recognition in still images. Deep Convolutional Neural Networks (CNNs) for recognizing images with distinctive classes have shown great success, but their performance in discriminating fine-grained changes is not at the same level. We address this by proposing an end-to-end CNN model, which learns meaningful features linking fine-grained changes using our novel attention mechanism. It captures the spatial structures in images by identifying semantic regions (SRs) and their spatial distributions, and is proved to be the key to modeling subtle changes in images. We automatically identify these SRs by grouping the detected keypoints in a given image. The "usefulness" of these SRs for image recognition is measured using our innovative attentional mechanism focusing on parts of the image that are most relevant to a given task. This framework applies to traditional and fine-grained image recognition tasks and does not require manually annotated regions (e.g. bounding-box of body parts, objects, etc.) for learning and prediction. Moreover, the proposed keypoints-driven attention mechanism can be easily integrated into the existing CNN models. The framework is evaluated on six diverse benchmark datasets. The model outperforms the state-of-the-art approaches by a considerable margin using Distracted Driver V1 (Acc: 3.39%), Distracted Driver V2 (Acc: 6.58%), Stanford-40 Actions (mAP: 2.15%), People Playing Musical Instruments (mAP: 16.05%), Food-101 (Acc: 6.30%) and Caltech-256 (Acc: 2.59%) datasets.

Deep Fuzzy Multi-Object Categorization in Scene

Object Categorization is the process of, identifying and labelling the various distinct Classes (Categories), in the given input image. The Deep Fuzzy Multi-Object Categorization (DFMOC) model, combines the learning capability of Convolution Neural Networks (CNN) and the uncertainty-managing ability of Fuzzy system, for carrying out the categorization task. This work starts with Background Elimination process for ensuring the image clarity, followed by Fuzzification and Fuzzy Entropy computation. Simple fuzzy sets are to be framed, by employing Fuzzy C-Means (FCM) algorithm, for fuzzification of the input image. Thresholding Block is incorporated, for determining the clusters . The Fuzzy Entropy Computation (FEC) is done, to minimize the Fuzziness rate of the acquired input and consequently, the layers of CNN are trained in accordance with that. Caltech-101 Dataset is been utilized for analysis. Average Precision Rate of Categorization (APRC), along with other metrics namely Time taken and Error Rate, shows that DFMOC model performs better than other models.

Color Channel Perturbation Attacks for Fooling Convolutional Neural Networks and A Defense Against Such Attacks

The convolutional neural networks (CNNs) have emerged as a very powerful data dependent hierarchical feature extraction method. It is widely used in several computer vision problems. The CNNs learn the important visual features from training samples automatically. It is observed that the network overfits the training samples very easily. Several regularization methods have been proposed to avoid the overfitting. In spite of this, the network is sensitive to the color distribution within the images which is ignored by the existing approaches. In this paper, we discover the color robustness problem of CNN by proposing a color channel perturbation (CCP) attack to fool the CNNs. In CCP attack new images are generated with new channels created by combining the original channels with the stochastic weights. Experiments were carried out over widely used CIFAR10, Caltech256 and TinyImageNet datasets in the image classification framework. The VGG, ResNet and DenseNet models are used to test the impact of the proposed attack. It is observed that the performance of the CNNs degrades drastically under the proposed CCP attack. Result show the effect of the proposed simple CCP attack over the robustness of the CNN trained model. The results are also compared with existing CNN fooling approaches to evaluate the accuracy drop. We also propose a primary defense mechanism to this problem by augmenting the training dataset with the proposed CCP attack. The state-of-the-art performance using the proposed solution in terms of the CNN robustness under CCP attack is observed in the experiments. The code is made publicly available at https://github.com/jayendrakantipudi/Color-Channel-Perturbation-Attack .

A Unified Multi-Task Learning Architecture for Fast and Accurate Pedestrian Detection

We present a unified multi-task learning architecture for fast and accurate pedestrian detection. Different from existing methods which often focus on either a new loss function or architecture, we propose an improved multi-task convolutional neural network learning architecture to effectively and efficiently interfuse the task of pedestrian detection and semantic segmentation. To achieve this, we integrate a lightweight semantic segmentation branch to Faster R-CNN detection framework that enables end-to-end hard parameter sharing in order to boost the detection performance and maintain computational efficiency as follows. Firstly, a Semantic Segmentation to Feature Module (SS2FM) refines the convolutional features in RPN stage by integrating the features generated from the semantic segmentation branch. Secondly, a Semantic Segmentation to Confidence Module (SS2CM) refines the classification confidence in RPN stage by fusing it with the semantic segmentation confidence. We also introduce an effective anchor matching point transform to alleviate the problem of feature misalignment for heavily occluded pedestrians. The proposed unified multi-task learning architecture lends itself well to more robust pedestrian detection in diverse scenarios with negligible computation overhead. In addition, the proposed architecture can achieve high detection performance with low resolution input images, which significantly reduces the computational complexity. Experiment results on CityPersons and Caltech datasets show that our method is the fastest among all state-of-the-art pedestrian detection methods while exhibiting competitive detection performance.

A NOVEL SELF-TAUGHT LEARNING FRAMEWORK USING SPATIAL PYRAMID MATCHING FOR SCENE CLASSIFICATION

Abstract. Remote sensing earth observation images have a wide range of applications in areas like urban planning, agriculture, environment monitoring, etc. While the industrial world benefits from availability of high resolution earth observation images since recent years, interpreting such images has become more challenging than ever. Among many machine learning based methods that have worked out successfully in remote sensing scene classification, spatial pyramid matching using sparse coding (ScSPM) is a classical model that has achieved promising classification accuracy on many benchmark data sets. ScSPM is a three-stage algorithm, composed of dictionary learning, sparse representation and classification. It is generally believed that in the dictionary learning stage, although unsupervised, one should use the same data set as classification stage to get good results. However, recent studies in transfer learning suggest that it might be a better strategy to train the dictionary on a larger data set different from the one to classify.In our work, we propose an algorithm that combines ScSPM with self-taught learning, a transfer learning framework that trains a dictionary on an unlabeled data set and uses it for multiple classification tasks. In the experiments, we learn the dictionary on Caltech-101 data set, and classify two remote sensing scene image data sets: UC Merced LandUse data set and Changping data set. Experimental results show that the classification accuracy of proposed method is compatible to that of ScSPM. Our work thus provides a new way to reduce resource cost in learning a remote sensing scene image classifier.

Randomized non-linear PCA networks

PCANet is an unsupervised Convolutional Neural Network (CNN), which uses Principal Component Analysis (PCA) to learn convolutional filters. One drawback of PCANet is that linear PCA cannot capture nonlinear structures within data. To address this problem, a straightforward approach is utilizing kernel methods by equipping the PCA method in PCANet with a kernel function. However, this practice leads to a network having cubic complexity with respect to the number of training image patches. In this paper, we propose a network called Randomized Nonlinear PCANet (RNPCANet), which uses an explicit kernel PCA to learn the convolutional filters. Although RNPCANet utilizes kernel methods for nonlinear processing of data, using kernel approximation techniques to define an explicit feature space in each stage, we theoretically show that the complexity of this model is not much higher than that of PCANet. We also show that our method links PCANets to Convolutional Kernel Networks (CKNs) as the proposed model maps the patches to a kernel feature space similar to CKNs. We evaluate our model on image recognition tasks including Coil-20, Coil-100, ETH-80, Caltech-101, MNIST, and C-Cube datasets. The experimental results show that the proposed method has superiority over PCANet and CKNs in terms of recognition accuracy.

Associative memory optimized method on deep neural networks for image classification

Deep neural networks have achieved excellent performance in the field of image classification. However, even to state-of-the-art deep neural networks, there are still many critical images that are difficult to be classified effectively. Enlightened by the brain function of associative memory, we propose a novel classification optimization method based on deep neural networks to improve image classifiers. Psychologists have studied associative memory for a long time. A popular theory is that ideas and memory are associated together in the mind through experience. By applying this theory to object recognition, our method focuses on using the association among different images of the same category to improve image classifiers based on deep neural networks. The association, which is memorized by the LSTM network in our method, could infer a sequence of associative images and form inner data augmentation effectively. Further, we introduce the LSTM network into an end-to-end deep learning framework to boost the performance of image classifiers. Experiments on four benchmark datasets reveal that our method produces a consistent improvement to the existing powerful classifiers. Moreover, as far as we know, our method achieves the best classification accuracies of 97.3%, 96.0%, and 89.1% on Flower102, Caltech101, and Caltech256 datasets, respectively.

Bridging the Gap of Lane Detection Performance Between Different Datasets: Unified Viewpoint Transformation

Convolutional neural networks (CNNs) have shown excellent performance for vision-based lane detection. However, maintaining the performance of the trained models under new test scenarios still remains challenging due to the dataset bias between the training and test datasets; In lane detection processes, the dataset bias can be categorized into lane position bias and lane pattern bias, with the former one particularly influences the lane detection performance. To tackle this dataset bias, this article proposes a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">unified viewpoint transformation (UVT)</i> method that transforms the camera viewpoints of different datasets into a common virtual world coordinate system, such that the mismatched lane position distributions can be effectively aligned. Experiments are conducted on multiple datasets including the Caltech <xref ref-type="bibr" rid="ref1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[1]</xref> , Tusimple <xref ref-type="bibr" rid="ref2" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[2]</xref> , and KITTI <xref ref-type="bibr" rid="ref3" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[3]</xref> dataset. The results demonstrate the effectiveness of the UVT algorithm in improving the lane detection performance on the test datasets. Moreover, by incorporating the UVT into other techniques that tackling the dataset bias, the lane position and pattern differences are disentangled and separately minimized. As a result, the performance gap between the training data and the test scenarios can be bridged. Specifically, the model trained on the KITTI dataset have achieved high performance in the Tusimple and the Caltech dataset (F1-score: 84.8 and 87.1%). With the proposed algorithm, a lane detection model trained on one dataset can be effectively applied to datasets with different camera settings in vastly different localities, and achieve better generalization ability compared to the state of the art methods.

An efficient content based image retrieval using enhanced multi-trend structure descriptor

This paper contributes a new variant of multi-trend structure descriptor (MTSD) for efficient content based image retrieval. The proposed variant of MTSD encodes color/edge orientation/texture quantized values versus orientation of equal, small and large trends instead of color/edge orientation/texture quantized values versus equal, small and large trends. In addition, it also encodes color/edge orientation/texture quantized values versus average location of distribution of pixel values for equal, small and large trends at each orientation. To reduce the time cost of the proposed variant of MTSD with the preservation of its accuracy, the image is decomposed into fine level using discrete Haar wavelet transform and the fine level for the decomposition of an image is determined empirically. Comprehensive experiments are conducted using the benchmark Corel-1k, Corel-5k, Corel-10k, Caltech-101, LIDC-IDRI-CT, VIA/I-ELCAP-CT and OASIS-MRI image datasets and the results evident that the proposed variant of MTSD achieves the state-of-the-art performance for natural, textural and biomedical image retrieval. Precision and recall are the measures used to measure the accuracy. Euclidean similarity measure is used to calculate the similarity information between query and target images.

A hybrid codebook model for object categorization using two-way clustering based codebook generation method

Both the visual codebook and the codebook model are considered as two main parts of most object classification frameworks. In the original codebook model, each image descriptor is encoded using a single codebook obtained usually using a clustering approach. In this paper, we propose a hybrid codebook model for an object classification task. A simultaneous clustering approach is applied to image descriptors to generate two variant codebooks and used them separately to encode and represent an image through a patch-based codebook model and a feature-based codebook model respectively. The proposed codebook model has been tested on the Caltech-101 dataset. Experimental results demonstrate state-of-the-art performance compared to typical clustering-based codebook model.

Multi-scale feature network for few-shot learning

Few-shot learning aims to learn a classifier that has good generalization performance in new classes, where each class only a small number of labeled examples are available. The existing few-shot classification methods use the single-scale image do not learn effective feature representation. Moreover, most of previous methods still depend on standard metrics to calculate visual similarities, such as Euclidean or cosine distance. Standard metrics are independent of data and lack nonlinear internal structure that captures the similarity between data. In this paper, we propose a new method for few-shot learning problem, which learns a multi-scale feature space, and classification is performed by computing similarities between the multi-scale representation of the image and the label feature of each class (i.e. class representation). Our method, called the Multi-Scale Feature Network (MSFN), is trained end-to-end from scratch. The proposed method improves 1-shot accuracy from 50.44% to 54.48% and 5-shot accuracy from 68.2% to 69.06% on MiniImagenet dataset compared to competing approaches. Experimental results on Omniglot, MiniImagenet, Cifar100, CUB200, and Caltech256 datasets demonstrate the effectiveness of the proposed method.

A Study of Dimensionality Reduction Impact on an Approach to People Detection in Gigapixel Images

Digital images are found in several sizes and are easily displayed on a computer screen using techniques that can reduce their dimensions. Moreover, algorithms are used to process images to perform several tasks, for instance, detection of people. Recently, gigapixel images emerged, providing a huge amount of data; however, algorithms for people detection have been usually tested only on regular size images. This paper presents an impact analysis of the resolution reduction in the detection of people in gigapixel images. People detectors were trained with the INRIA and CALTECH data sets and results show that, although gigapixel images provide a huge false positive rate, the resolution reduction significantly decreases the number of bounding boxes and false positives, however, increasing the rate of missing people.

Automating Configuration of Convolutional Neural Network Hyperparameters Using Genetic Algorithm

In recent years, Convolutional Neural Networks (CNN) have been widely used for real-world applications in the field of computer vision. Their class-leading performance, however, depends heavily on the architecture used for a given problem. In most cases, the architectures are manually optimized by the researchers, a time-consuming process hard to achieve without prior knowledge of CNN. In this paper, we propose a new genetic algorithm for the optimization of the CNN architecture for a given image classification problem. This algorithm extends and refines existing research in the field, by allowing depth exploration, introducing a novel sequential crossover operator, using an incremental selective pressure schedule over evolution (favoring higher diversity in early generations) and by evaluating individual performances over the validation set with early stopping. The technique is validated in three image classification dataset, namely, CIFAR10, MNIST and Caltech256 datasets, which are widely used benchmarks for image classification algorithms. We evaluate the performance and total execution time over these datasets, and compare our results with those achieved by the best genetic methods published so far. In all cases, we achieve better results in terms of test accuracy, consistently over different datasets, while remaining in the same orders of magnitude of execution time of the fastest approaches.

A Novel Image Retrieval Method with Improved DCNN and Hash

In large-scale image retrieval, deep features extracted by Convolutional Neural Network (CNN) can effectively express more image information than those extracted by traditional manual methods. However, the deep feature dimensions obtained by Deep Convolutional Neural Network (DCNN) are too high and redundant, which leads to low retrieval efficiency. We propose a novel image retrieval method, which combines deep features selection with improved DCNN and hash transform based on high-dimension features reduction to gain lowdimension deep features and realizes efficient image retrieval. Firstly, the improved network is based on the existing deep model to build a more profound and broader network by adding multiple groups of different branches. Therefore, it is named DFS-Net (Deep Feature Selection Network). The adaptive learning deep features of the Network can effectively alleviate the influence of over-fitting and improve the feature expression of image content. Secondly, the information gain rate method is used to filter the extracted deep features to reduce the feature dimension and ensure the information loss is small. The last step of the method, hash Transform, sparsifies and binarizes this representation to reduce the computation and storage pressure while maintaining the retrieval accuracy. Finally, the scheme is based on the distinguished ResNet50, InceptionV3, and MobileNetV2 models, and studied and evaluated deeply on the CIFAR10 and Caltech256 datasets. The experimental results show that the novel method can train the deep features with stronger recognition ability on limited training samples, and improve the accuracy and efficiency of image retrieval effectively.

Image Retrieval Based on Deep Feature Extraction and Reduction with Improved CNN and PCA

With the rapid development of information technology, the speed and efficiency of image retrieval are increasingly required in many fields, and a compelling image retrieval method is critical for the development of information. Feature extraction based on deep learning has become dominant in image retrieval due to their discrimination more complete, information more complementary and higher precision. However, the high-dimension deep features extracted by CNNs (convolutional neural networks) limits the retrieval efficiency and makes it difficult to satisfy the requirements of existing image retrieval. To solving this problem, the high-dimension feature reduction technology is proposed with improved CNN and PCA quadratic dimensionality reduction. Firstly, in the last layer of the classical networks, this study makes a well-designed DR-Module (dimensionality reduction module) to compress the number of channels of the feature map as much as possible, and ensures the amount of information. Secondly, the deep features are compressed again with PCA (Principal Components Analysis), and the compression ratios of the two dimensionality reductions are reduced, respectively. Therefore, the retrieval efficiency is dramatically improved. Finally, it is proved on the Cifar100 and Caltech101 datasets that the novel method not only improves the retrieval accuracy but also enhances the retrieval efficiency. Experimental results strongly demonstrate that the proposed method performs well in small and medium-sized datasets.

A Category-Specific Dictionary Learning Method Tailored for Reconstruction-Based Feature Coding

Bag-of-Visual-Words (BoVW) is still a useful image classification model when there is not enough data to use Deep Learning. In BoVW model, the practice of reducing the reconstruction errors of local features can improve the classification accuracy owing to the decrease of information loss. Many reconstruction-based coding methods are proposed to learn a visual dictionary and encode local features via minimizing the reconstruction errors of local features with constraints. Besides this, the accuracy can also be improved by learning the category-specific dictionaries and then encoding features based on these dictionaries. By considering the two practices together, we propose a simple category-specific dictionary learning method tailored for reconstruction-based feature coding. Our method can be used as a universal one to improve the classification accuracies of many reconstruction-based coding methods, which is the highlight of our method. Concretely, a universal dictionary is learned by employing a reconstruction-based coding method and then refined for each category to obtain the category-specific dictionary of this category. When encoding a feature by a category-specific dictionary, the visual words for encoding it are decided in advance by the indices, which correspond to the non-zero elements of its coding vector obtained with the universal dictionary. The effectiveness of our method is validated by observing whether there is an accuracy improvement after applying our method. Our results on Scene-15, Caltech-101, and UIUC-Sports datasets show that the accuracies of four representative coding methods are improved by about 0.3% to 2.7%, which experimentally demonstrates the universality and effectiveness of our method.

Multiple Kernel Based Transfer Learning for the Few-Shot Recognition Task in Smart Home Scene

With the development of artificial intelligence, smart home plays an increasingly important role in daily life. Since new objects may constantly appear at home and the collecting of enough training samples sometimes can be hard, the few-shot recognition task is essential and practical in smart home scene. MULticlass Transfer Incremental LEarning (MULTIpLE) is an effective algorithm that can perform transfer learning for class increment with few new samples based on Support Vector Machine (SVM). But the features of images are generally high-dimensional and the selection for kernel function affects the performance of MULTIpLE. In this paper, a new transfer learning algorithm based on multiple kernel learning, termed MULticlass Transfer Incremental LEarning based on Multiple Kernel Learning (MULTIpLE-MKL), is proposed for the few-shot recognition task in smart home scene. There are two main steps for the MULTIpLE-MKL, including the first multiple kernel learning stage and the second transfer learning stage. Specifically, multiple kernel learning is first applied in the construction of SVM models to optimize the selection of kernel function. When different kernels are calculated based on different features, the sparse kernel coefficients achieve the key feature selection. Second, the SVM models learn a new class from few samples by the virtue of the transfer learning algorithm, MULTIpLE. Compared with conventional methods, experiments based on the benchmark Caltech-256 dataset demonstrate that the proposed MULTIpLE-MKL not only maintains the good performance in original classes but also shows improving recognition ability for new classes only with few training samples.

Применение метода мажоризации-минимизации к алгоритму Чана --- Везе в задаче сегментации изображений

The purpose of the study was to modify Chan --- Vese algorithm in order to overcome its shortcomings, such as high computational complexity and the use of approximations. In the considered modification, optimization is carried out by the majorization-minimization method, the main idea of which is to reduce the complexity of the problem using the majority function. Due to the proposed optimization method, it is possible to use the Heaviside step function and Dirac delta function. This enabled the same or better saturation levels when optimization is done by the graph cut method in a smaller number of iterations, which reduced the operation time. The proposed algorithm was tested on a Caltech101 dataset. The algorithm is general, does not depend on the subject area and does not require prior training. This allows it to be used as the basis for a wide range of image segmentation algorithms.

Learning Semantic Text Features for Web Text-Aided Image Classification

The good generalization performance of conventional pattern classifiers often relies on the size of training data labeled by costly human labor. These days, publicly available web resources grow explosively, and this allows us to easily obtain abundant and cheap web data. Yet, web data are usually not as cooperative as human labeled data. In this paper, we explore the use of web text data to aid image classification. Without requiring the previous collection of auxiliary data from the web, we directly retrieve the web text information with the aid of the powerful reverse image search engine. We develop a novel textual modeling method named semantic matching neural network (SMNN) that is capable of learning semantic features from the associated text of web images. The SMNN text features have improved reliability and applicability, compared to the text features obtained from other methods. The SMNN text features and convolutional neural network (CNN) visual features are merged into a shared representation, which learns to capture the correlations between the two modalities. Experimental results on benchmark UIUC-Sports, Scene-15, Caltech-256, and Pascal VOC-2012 data sets show that the visual and text modalities of data from different sources are remarkably complementary and the fusion of them achieves substantial performance improvement.

Image Feature Selection using Ant Colony Optimization

With the enhancement in imaging technology, huge amount of information is being created whose classification is a challenging task among researchers. The classification performance is solely dependent on the quality of the features defining the respective images. Hence, feature extraction from images and feature selection from this huge data is required. Image Feature Selection is a vital step which can significantly affect the performance of image classification system. This paper proposes an efficient combination of image feature extraction technique and feature selection strategy using Ant Colony Optimization (ACO). ACO utilizes overall subsets performance and local feature importance to fetch problem domain finding prime solutions. For ACO based feature selection in images most of the researchers use priori information of features. However, in this work, the features are extracted separately through the detailed analytical process which carries the best suited features for different classes of images. The performance in terms of classification accuracy has been enhanced through the optimization of features in the dataset. For experimentation and evaluating the performance of proposed work, Corel and Caltech datasets are used. Satisfactory results have been obtained with the proposed approach.