Caltech Dataset Research Articles

Image Feature Selection using Ant Colony Optimization

With the enhancement in imaging technology, huge amount of information is being created whose classification is a challenging task among researchers. The classification performance is solely dependent on the quality of the features defining the respective images. Hence, feature extraction from images and feature selection from this huge data is required. Image Feature Selection is a vital step which can significantly affect the performance of image classification system. This paper proposes an efficient combination of image feature extraction technique and feature selection strategy using Ant Colony Optimization (ACO). ACO utilizes overall subsets performance and local feature importance to fetch problem domain finding prime solutions. For ACO based feature selection in images most of the researchers use priori information of features. However, in this work, the features are extracted separately through the detailed analytical process which carries the best suited features for different classes of images. The performance in terms of classification accuracy has been enhanced through the optimization of features in the dataset. For experimentation and evaluating the performance of proposed work, Corel and Caltech datasets are used. Satisfactory results have been obtained with the proposed approach.

Learning Rate Scheduling Policies

With the availability of high processing capability hardwares at less expensive prices, it is possible to successfully train multi-layered neural networks. Since then, several training algorithms have been developed, from algorithms which are statically initialized to algorithms which adaptively change. It is observed that to improve the training process of neural networks, the hyper-parameters are to be fine tuned. Learning Rate, Decay rate, number of epochs, number of hidden layers and number of neurons in the network are some of the hyper-parameters in concern. Of these, the Learning rate plays a crucial role in enhancing the learning capability of the network. Learning rate is the value by which the weights are adjusted in a neural network with respect to the gradient descending towards the expected optimum value. This paper discusses four types of learning rate scheduling which helps to find the best learning rates in less number of epochs. Following these scheduling methods, facilitates to find better initial learning rate value and step-wise updation during the later phase of the training process. In addition the discussed learning rate schedules are demonstrated using COIL-100, Caltech-101 and CIFAR-10 datasets trained on ResNet. The performance is evaluated using the metrics, Precision, Recall and F1-Score. The results analysis show that, depending on the nature of the dataset, the performance of the Learning Rate Scheduling policy varies. Hence the choice of the scheduling policy to train a neural network is made, based on the data.

High-Level Semantic Networks for Multi-Scale Object Detection

To better solve scale variance problem, deep multi-scale methods usually detect objects of different scales by different in-network layers. However, the semantic levels of features from different layers are usually inconsistent. In this paper, we propose a multi-branch and high-level semantic network by gradually splitting a base network into multiple different branches. As a result, the different branches have same depth and the output features of different branches have similarly high-level semantics. Due to the difference of receptive fields, the different branches are suitable to detect objects of different scales. Meanwhile, the multi-branch network does not introduce additional parameters by sharing the convolutional weights of different branches. To further improve detection performance, skip-layer connections are used to add context to the branch of relatively small receptive field, and dilated convolution is incorporated to enlarge the resolutions of output feature maps. When they are embedded into Faster RCNN architecture, the weighted scores of proposal generation network and proposal classification network are further proposed. Experiments on three pedestrian datasets (i.e., the KITTI dataset, the Caltech dataset, and the Citypersons dataset), one face dataset (i.e., the WIDER FACE dataset), and two general object datasets (i.e., the COCO benchmark and the PASCAL VOC dataset) demonstrate the effectiveness and generality of proposed method. On these datasets, our method achieves state-of-the-art performance.

Rethinking compact abating probability modeling for open set recognition problem in Cyber-physical systems

Abstract Machine learning (ML) components, especially convolutional neural network (CNN) based components, are becoming increasingly vital in cyber-physical systems (CPS), existing great influence on the dependability of CPS. Different to the popular ML model with close set hypothesis, which limits the classes of testing samples to seen classes in training samples, ML in CPS should face the problem of real-world recognition. Open set recognition (OSR) is a more suitable model for improving the dependability of ML components, which allows the presence of unknown classes at testing time. However, the ability to reject those unknown classes is concurrently required in the model. In this paper, we find the defect in compact abating probability application to most CNN-based OSR model. In order to mitigate the defect, two different methods, OpenSoftMax and OPEB are respectively proposed. Finally, extensive experiments on caltech256 data set have been conducted to verify the effectiveness and efficiency of our proposals. Compared to the well-known CNN-based OSR method OpenMax, our first method OpenSoftMax achieves 15.55% average accuracy performance gain and 28.35% F1-score performance gain, while our second method OPEB achieves 13.17% average accuracy performance gain and 25.70% F1-score performance gain.

Estimation of ergodicity limits of bag-of-words modeling for guaranteed stochastic convergence

This paper suggests an efficient dual ergodicity limits-based bag-of-words (DEL-BoW) modeling technique. The suggested DEL-BoW technique estimates two limits of ergodicity of a discrete random variable (drv) that is formed from the BoW classification performance of multiple runs. The first limit of ergodicity is estimated with a relatively larger ball of convergence to keep the drv shorter. Hence both robustness against random initialization and estimation of the optimal model-order are realized with a reduced number of iterations. Once the optimal model-order is estimated, the radius of ball of convergence is reduced and a second limit of ergodicity is estimated. Reducing the ball of convergence enlarges the size of the considered performance drv that enhances the classification performance. Experiments conducted on Caltech-101, Caltech-256, 15-Scenes, and Flower-102 datasets resulted in classification accuracy of 86.91%, 72.57%, 90.57%, and 90.86%, respectively. Comparison with state-of-the-art techniques shows the excellent performance of the DEL-BoW modeling process.

A Real-Time 17-Scale Object Detection Accelerator With Adaptive 2000-Stage Classification in 65 nm CMOS

Machine learning has become ubiquitous in applications including object detection, image/video classification, and natural language processing. While machine learning algorithms have been successfully used in many practical applications, accurate, fast, and low-power hardware implementations of such algorithms is still a challenging task, especially for mobile systems such as Internet of Things (IoT), autonomous vehicles, and smart drones. This paper presents an energy-efficient programmable ASIC accelerator for object detection. Our ASIC accelerator supports multi-class (e.g., face, traffic sign, car license plate, and pedestrian) that are programmable, many-object (up to 50) in one image with different sizes (17-scale support with 6 down-/11 up-scaling), and high accuracy (AP of 0.87/0.81/0.72/0.76 for FDDB/AFW/BTSD/Caltech datasets). We designed an integral channel detector with 2,000 classifiers for rigid boosted templates, where the number of stages used for classification can be adaptively controlled depending on the content of the search window. This can be implemented with a more modular hardware, compared to support vector machine (SVM) and deformable parts model (DPM) designs. By jointly optimizing the algorithm and the efficient hardware architecture, the prototype chip implemented in 65nm CMOS demonstrates real-time object detection of 20–50 frames/s with low power consumption of 22.5–181.7 mW (0.54–1.75 nJ/pixel) at 0.58–1.1 V supply.

Cross-Modality Bridging and Knowledge Transferring for Image Understanding

The understanding of web images has been a hot research topic in both artificial intelligence and multimedia content analysis domains. The web images are composed of various complex foregrounds and backgrounds, which makes the design of an accurate and robust learning algorithm a challenging task. To solve the above significant problem, first, we learn a cross-modality bridging dictionary for the deep and complete understanding of a vast quantity of web images. The proposed algorithm leverages the visual features into the semantic concept probability distribution, which can construct a global semantic description for images while preserving the local geometric structure. To discover and model the occurrence patterns between intra- and inter-categories, multi-task learning is introduced for formulating the objective formulation with Capped-ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> penalty, which can obtain the optimal solution with a higher probability and outperform the traditional convex function-based methods. Second, we propose a knowledge-based concept transferring algorithm to discover the underlying relations of different categories. This distribution probability transferring among categories can bring the more robust global feature representation, and enable the image semantic representation to generalize better as the scenario becomes larger. Experimental comparisons and performance discussion with classical methods on the ImageNet, Caltech-256, SUN397, and Scene15 datasets show the effectiveness of our proposed method at three traditional image understanding tasks.

Efficient mobilenet architecture as image recognition on mobile and embedded devices

The introduction of a modern image recognition that has millions of parameters and requires a lot of training data as well as high computing power that is hungry for energy consumption so it becomes inefficient in everyday use. Machine Learning has changed the computing paradigm, from complex calculations that require high computational power to environmentally friendly technologies that can efficiently meet daily needs. To get the best training model, many studies use large numbers of datasets. However, the complexity of large datasets requires large devices and requires high computing power. Therefore large computational resources do not have high flexibility towards the tendency of human interaction which prioritizes the efficiency and effectiveness of computer vision. This study uses the Convolutional Neural Networks (CNN) method with MobileNet architecture for image recognition on mobile devices and embedded devices with limited resources with ARM-based CPUs and works with a moderate amount of training data (thousands of labeled images). As a result, the MobileNet v1 architecture on the ms8pro device can classify the caltech101 dataset with an accuracy rate 92.4% and 2.1 Watt power draw. With the level of accuracy and efficiency of the resources used, it is expected that MobileNet's architecture can change the machine learning paradigm so that it has a high degree of flexibility towards the tendency of human interaction that prioritizes the efficiency and effectiveness of computer vision.

Deep Spiking Convolutional Neural Network Trained With Unsupervised Spike-Timing-Dependent Plasticity

Spiking neural networks (SNNs) have emerged as a promising brain inspired neuromorphic-computing paradigm for cognitive system design due to their inherent event-driven processing capability. The fully connected (FC) shallow SNNs typically used for pattern recognition require large number of trainable parameters to achieve competitive classification accuracy. In this paper, we propose a deep spiking convolutional neural network (SpiCNN) composed of a hierarchy of stacked convolutional layers followed by a spatial-pooling layer and a final FC layer. The network is populated with biologically plausible leaky-integrate-and-fire (LIF) neurons interconnected by shared synaptic weight kernels. We train convolutional kernels layer-by-layer in an unsupervised manner using spike-timingdependent plasticity (STDP) that enables them to self-learn characteristic features making up the input patterns. In order to further improve the feature learning efficiency, we propose using smaller 3×3 kernels trained using STDP-based synaptic weight updates performed over a mini-batch of input patterns. Our deep SpiCNN, consisting of two convolutional layers trained using the unsupervised convolutional STDP learning methodology, achieved classification accuracies of 91.1% and 97.6%, respectively, for inferring handwritten digits from the MNIST data set and a subset of natural images from the Caltech data set.

Improving Image Classification Accuracy With ELM and CSIFT

We present a new image classification system by integrating color information and the extreme learning machine algorithm based on the localized soft-assignment coding classification framework. Our system is original with its unique structure. It improves the image classification accuracy significantly compared to the existing image classification systems. Specifically, on classification accuracy, our system can improve more than 4% on the Caltech-101 dataset and achieve up to 3% improvement on the Caltech-256 dataset compared to existing systems.

Deep Leaky Single-peaked Triangle Neural Networks

Recently, Deep learning has made a great deal of success in processing images, audios, and natural languages and so on. The activation function is one of the key factors in Deep learning. In this paper, according to characteristics of biological neurons, an improved Leaky Single-Peaked Triangle Linear Unit (LSPTLU) activation function is presented for the right-hand response unbounded of Rectified Linear Unit (ReLU) and Leaky ReLU (LReLU). LSPTLU is more in line with the biological neuron essence and achieves the excellent performance of equivalent or beyond ReLU and LReLU on different datsets, e.g., MNIST, Fashion-MNIST, SVHN, IMAGENET, CALTECH101 and CIFAR10 datasets.

Boosting Occluded Image Classification via Subspace Decomposition-Based Estimation of Deep Features.

Classification of partially occluded images is a highly challenging computer vision problem even for the cutting-edge deep learning technologies. To achieve a robust image classification for occluded images, this article proposes a novel scheme using the subspace decomposition-based estimation (SDBE). The proposed SDBE-based classification scheme first employs a base convolutional neural network to extract the deep feature vector (DFV) and then utilizes the SDBE to compute the DFV of the original occlusion-free image for classification. The SDBE is performed by projecting the DFV of the occluded image onto the linear span of a class dictionary (CD) along the linear span of an occlusion error dictionary (OED). The CD and OED are constructed, respectively, by concatenating the DFVs of a training set and the occlusion error vectors of an extra set of image pairs. Two implementations of the SDBE are studied in this article: 1) the l1 -norm and 2) the squared l2 -norm regularized least-squares estimates. By employing the ResNet-152, pretrained on the ImageNet Large-Scale Visual Recognition Challenge 2012 (ILSVRC2012) training set, as the base network, the proposed SBDE-based classification scheme is extensively evaluated on the Caltech-101 and ILSVRC2012 datasets. Extensive experimental results demonstrate that the proposed SDBE-based scheme dramatically boosts the classification accuracy for occluded images, and achieves around 22.25% increase in classification accuracy under 20% occlusion on the ILSVRC2012 dataset.

Transfer Learning Based Image Visualization Using CNN

Image classification is a popular machine learning based applications of deep learning. Deep learning techniques are very popular because they can be effectively used in performing operations on image data in large-scale. In this paper CNN model was designed to better classify images. We make use of feature extraction part of inception v3 model for feature vector calculation and retrained the classification layer with these feature vector. By using the transfer learning mechanism the classification layer of the CNN model was trained with 20 classes of Caltech101 image dataset and 17 classes of Oxford 17 flower image dataset. After training, network was evaluated with testing dataset images from Oxford 17 flower dataset and Caltech101 image dataset. The mean testing precision of the neural network architecture with Caltech101 dataset was 98 % and with Oxford 17 Flower image dataset was 92.27 %.

Classification of Pedestrian using Convoluted Neural Network

We present our work based on classification of pedestrians into a single person and group of people using Convoluted Neural Network (CNN). Major work was done on classification-based feature extraction techniques before CNN is applied to it. CNN can classify objects without extracting the features. Here, we have set up a complete channel for pedestrian detection using sliding window approach and classification using a CNN network. Alex Net and ResNet are the two architectures used in CNN for implementing the classification algorithm. Performance is evaluated on the PET and Caltech dataset which consists of a number of people who are walking with a group or separately in the scene. We got the optimistic results in case of small dataset used for testing. We have also tested our algorithm over large dataset to verify its performance with the help of performance evaluation metrics.

Image retrieval based on convolutional neural network and linear discriminate analysis

Image retrieval is a hot research topic in the field of computer vision image processing, and the user queries the image database for similar images and produces a list of recommendations. The paper firstly sets forth the research status of image retrieval, then the convolution neural network is briefly introduced. Due to the traditional image retrieval and recommendation system use manual extraction of image features is relatively cumbersome, and the retrieval accuracy is not high research status, the paper proposes an image retrieval method based on the improved convolutional neural network and linear discriminate analysis. Caltech256 and CIFAR-10 datasets were trained using the model in this paper, experimental, results show that the proposed method can effectively improve the performance of retrieval.

Automated Labeling Process for Unknown Images in an open-world Scenario

Most of the recognition systems presume a controlled, well-defined research setting, where all possible classes that can appear during a test are known a priori. This environment is referred to as the ``closed-world'' model, while the ``open-world'' model implies that unknown classes can be incorporated into a recognition algorithm whilst being predicted. Therefore, recognition systems that operate in the real world have to deal with these unknown categories. Our objective was not only to detect data that originate from categories unseen during training, but to identify similarities between pieces of unknown data and then form new classes by automatically labeling them. Our Double Probability Model was extended by an image clustering algorithm, in which Kernel K-means was used. A new procedure, namely the Cluster Classification algorithm for the detection of unknowns and automated labeling, is proposed. These approaches facilitate the transition from open-set recognition to an open-world problem. The Fisher Vector (FV) was used for the mathematical representation of the images and then a Support Vector Machine introduced as a classifier. The measurement of similarity was based on the FV representations. Experiments were conducted on the Caltech101 and Caltech256 datasets of images and the Rand Index was evaluated over the unknown data. The results showed that our proposed Cluster Classification algorithm was able to yield almost the same Rand Index, even though the number of unknown categories increased.

New shape descriptor in the context of edge continuity

The object contour is a significant cue for identifying and categorising objects. The current work is motivated by indicative researches that attribute object contours to edge information. The spatial continuity exhibited by the edge pixels belonging to the object contour make these different from the noisy edge pixels belonging to the background clutter. In this study, the authors seek to quantify the object contour from a relative count of the adjacent edge pixels that are oriented in the four possible directions, and measure using exponential functions the continuity of each edge over the next adjacent pixel in that direction. The resulting computationally simple, low-dimensional feature set, called as ‘edge continuity features’, can successfully distinguish between object contours and at the same time discriminate intra-class contour variations, as proved by the high accuracies of object recognition achieved on a challenging subset of the Caltech-256 dataset. Grey-to-RGB template matching with City-block distance is implemented that makes the object recognition pipeline independent of the actual colour of the object, but at the same time incorporates colour edge information for discrimination. Comparison with the state-of-the-art validates the efficiency of the proposed approach.

Pedestrian detection using first- and second-order aggregate channel features

The content-based analysis of visual multimedia like images and videos are urgently needed to empower human society for the automation of difficult tasks. Pedestrian detection serves as a backbone for a multitude of image processing and machine learning algorithms and secures quite a lot of real-world applications. Keeping this fact in mind, here, we deal with the fabrication of suitable features to identify human/pedestrian instances from images with near accuracy. Accordingly, we introduce second-order aggregate channel features (SOACF) to enhance the performance of much-celebrated pedestrian detection algorithm which was mainly based on the first-order information in an image—aggregate channel features detector (ACF detector). We experimentally proved the complementary nature of ACF and SOACF. Designed to garner both these features together, instead of simple concatenation, or direct merging of the two detectors, we employed a weighted non-maximum suppression merging algorithm. The prospective detector not only performed well on INRIA, Caltech and KITTI pedestrian data set but also, mitigate the miss rate by $$\sim 4\%$$ in Caltech data set and $$\sim 2\%$$ in KITTI data set in comparison with ACF detector. Despite the fact that our in-house generated detector uses only a few channels, it surpasses many state-of-the-art methods based on baseline ACF detector. Moreover, the detection speed is 100 times faster than the topmost pedestrian detector based on ACF.

STDP-Based Pruning of Connections and Weight Quantization in Spiking Neural Networks for Energy-Efficient Recognition

Spiking neural networks (SNNs) with a large number of weights and varied weight distribution can be difficult to implement in emerging in-memory computing hardware due to the limitations on crossbar size (implementing dot product), the constrained number of conductance states in non-CMOS devices and the power budget. We present a sparse SNN topology where noncritical connections are pruned to reduce the network size, and the remaining critical synapses are weight quantized to accommodate for limited conductance states. Pruning is based on the power law weight-dependent spike timing dependent plasticity model; synapses between pre- and post-neuron with high spike correlation are retained, whereas synapses with low correlation or uncorrelated spiking activity are pruned. The weights of the retained connections are quantized to the available number of conductance states. The process of pruning noncritical connections and quantizing the weights of critical synapses is performed at regular intervals during training. We evaluated our sparse and quantized network on MNIST dataset and on a subset of images from Caltech-101 dataset. The compressed topology achieved a classification accuracy of 90.1% (91.6%) on the MNIST (Caltech-101) dataset with 3.1X (2.2X) and 4X (2.6X) improvement in energy and area, respectively. The compressed topology is energy and area efficient while maintaining the same classification accuracy of a 2-layer fully connected SNN topology.

Scene perception system for visually impaired based on object detection and classification using multimodal deep convolutional neural network

This paper represents a cost-effective scene perception system aimed towards visually impaired individual. We use an odroid system integrated with an USB camera and USB laser that can be attached on the chest. The system classifies the detected objects along with its distance from the user and provides a voice output. Experimental results provided in this paper use outdoor traffic scenes. The object detection and classification framework exploits a multi-modal fusion based faster RCNN using motion, sharpening and blurring filters for efficient feature representation.