Large Number Of Object Classes Research Articles

Decoding Brain Signals from Rapid-Event EEG for Visual Analysis Using Deep Learning.

The perception and recognition of objects around us empower environmental interaction. Harnessing the brain's signals to achieve this objective has consistently posed difficulties. Researchers are exploring whether the poor accuracy in this field is a result of the design of the temporal stimulation (block versus rapid event) or the inherent complexity of electroencephalogram (EEG) signals. Decoding perceptive signal responses in subjects has become increasingly complex due to high noise levels and the complex nature of brain activities. EEG signals have high temporal resolution and are non-stationary signals, i.e., their mean and variance vary overtime. This study aims to develop a deep learning model for the decoding of subjects' responses to rapid-event visual stimuli and highlights the major factors that contribute to low accuracy in the EEG visual classification task.The proposed multi-class, multi-channel model integrates feature fusion to handle complex, non-stationary signals. This model is applied to the largest publicly available EEG dataset for visual classification consisting of 40 object classes, with 1000 images in each class. Contemporary state-of-the-art studies in this area investigating a large number of object classes have achieved a maximum accuracy of 17.6%. In contrast, our approach, which integrates Multi-Class, Multi-Channel Feature Fusion (MCCFF), achieves a classification accuracy of 33.17% for 40 classes. These results demonstrate the potential of EEG signals in advancing EEG visual classification and offering potential for future applications in visual machine models.

Embedding Visual Hierarchy with Deep Networks for Large-Scale Visual Recognition.

In this paper, a layer-wise mixture model (LMM) is developed to support hierarchical visual recognition, where a Bayesian approach is used to automatically adapt the visual hierarchy to the progressive improvements of the deep network along the time. Our LMM algorithm can provide an end-to-end approach for jointly learning: (a) the deep network for achieving more discriminative deep representations for object classes and their inter-class visual similarities; (b) the tree classifier for recognizing large numbers of object classes hierarchically; and (c) the visual hierarchy adaptation for achieving more accurate assignment and organization of large numbers of object classes. By learning the tree classifier, the deep network and the visual hierarchy adaptation jointly in an end-to-end manner, our LMM algorithm can achieve higher accuracy rates on hierarchical visual recognition. Our experiments are carried on ImageNet1K and ImageNet10K image sets, which have demonstrated that our LMM algorithm can achieve very competitive results on the accuracy rates as compared with the baseline methods.

Integrating multi-level deep learning and concept ontology for large-scale visual recognition

To support large-scale visual recognition (i.e., recognizing thousands or even tens of thousands of object classes), a multi-level deep learning algorithm is developed to learn multiple deep networks and a tree classifier jointly, where a concept ontology is constructed to organize large numbers of object classes hierarchically in a coarse-to-fine fashion and determine the inter-related learning tasks automatically. Our multi-level deep learning algorithm can: (a) train multiple deep networks simultaneously to achieve more discriminative representations of both coarse-grained groups and fine-grained object classes at different levels of the concept ontology (i.e., learning multiple sets of deep features simultaneously for different tasks); (b) leverage multi-task learning to train more discriminative classifiers for the fine-grained object classes in the same group to enhance their separability significantly and enable inter-class knowledge transferring; and (c) learn multiple deep networks and the tree classifier jointly in an end-to-end fashion. Our experimental results on three image sets have demonstrated that our multi-level deep learning algorithm can achieve very competitive results on both the accuracy rates and the computational efficiency for large-scale visual recognition.

Contextual Co-occurrence Information for Object Representation and Categorization

Object categorization based on hierarchical context modeling has shown to be useful in large database of object categories, especially, when a large number of object classes needs to be recognized from a range of different scene categories. However, average precision of categorization is still low compared to other existing methods. This may reflect that the contribution of underlying relations between objects has not been fully considered. In this paper, we improve average precision of contextual object recognition by taking advantage of objects co-occurrence information. Our method consists of two main phases. In the first phase, object representation is derived by considering the frequency of objects appeared in each image. The second phase is focused on classification of objects by applying a decision tree algorithm. We use SUN09 database to evaluate our proposed method. This database consists of images spanning from different scene categories and object instances. Our experimental results demonstrate that our proposed method achieves a higher average precision in comparison to a recent similar method by encoding contextual information in an efficient way.

Cost-sensitive learning of hierarchical tree classifiers for large-scale image classification and novel category detection

In this paper, a cost-sensitive learning algorithm is developed to train hierarchical tree classifiers for large-scale image classification application (i.e., categorizing large-scale images into thousands of object classes). A visual tree is first constructed for organizing large numbers of object classes hierarchically and identifying inter-related learning tasks automatically. For the fine-grained object classes at the sibling leaf nodes, they share significant common visual properties but still contain subtle visual differences, thus a multi-task structural learning algorithm is developed to train their inter-related classifiers jointly to enhance their discrimination power. For the coarse-grained categories (i.e., groups of visually similar object classes) at the sibling non-leaf nodes, a hierarchical learning algorithm is developed to leverage tree structure (by adding two inter-level constraints) to train their inter-related classifiers jointly and control inter-level error propagation effectively. To achieve more robust detection of large numbers of object classes, a visual forest is learned by combining multiple visual trees (for different configurations) and their hierarchical tree classifiers. By penalizing various types of misclassification errors differently, a cost-sensitive learning approach is further developed to detect the appearances of new object classes accurately, and an incremental learning algorithm is developed to achieve more effective training of the discriminative classifiers for new object classes. Our experimental results have demonstrated that our cost-sensitive hierarchical learning algorithm can achieve very competitive results on both classification accuracy and computational efficiency as compared with other state-of-the-art techniques.

Training inter-related classifiers for automatic image classification and annotation

A structural learning algorithm is developed in this paper to achieve more effective training of large numbers of inter-related classifiers for supporting large-scale image classification and annotation. A visual concept network is constructed for characterizing the inter-concept visual correlations intuitively and determining the inter-related learning tasks automatically in the visual feature space rather than in the label space. By partitioning large numbers of object classes and image concepts into a set of groups according to their inter-concept visual correlations, the object classes and image concepts in the same group will share similar visual properties and their classifiers are strongly inter-related while the object classes and image concepts in different groups will contain various visual properties and their classifiers can be trained independently. By leveraging the inter-concept visual correlations for inter-related classifier training, our structural learning algorithm can train the inter-related classifiers jointly rather than independently, which can enhance their discrimination power significantly. Our experiments have also provided very positive results on large-scale image classification and annotation.

Contextually guided semantic labeling and search for three-dimensional point clouds

RGB-D cameras, which give an RGB image together with depths, are becoming increasingly popular for robotic perception. In this paper, we address the task of detecting commonly found objects in the three-dimensional (3D) point cloud of indoor scenes obtained from such cameras. Our method uses a graphical model that captures various features and contextual relations, including the local visual appearance and shape cues, object co-occurrence relationships and geometric relationships. With a large number of object classes and relations, the model’s parsimony becomes important and we address that by using multiple types of edge potentials. We train the model using a maximum-margin learning approach. In our experiments concerning a total of 52 3D scenes of homes and offices (composed from about 550 views), we get a performance of 84.06% and 73.38% in labeling office and home scenes respectively for 17 object classes each. We also present a method for a robot to search for an object using the learned model and the contextual information available from the current labelings of the scene. We applied this algorithm successfully on a mobile robot for the task of finding 12 object classes in 10 different offices and achieved a precision of 97.56% with 78.43% recall.1

Multi-taskmulti-labelmultiple instance learning

For automatic object detection tasks, large amounts of training images are usually labeled to achieve more reliable training of the object classifiers; this is cost-expensive since it requires hiring professionals to label large-scale training images. When a large number of object classes come into view, the issue of obtaining a large enough amount of the labeled training images becomes more critical. There are three potential solutions to reduce the burden for image labeling: (1) allowing people to provide the object labels loosely at the image level rather than at the object level (e.g., loosely-tagged images without identifying the exact object locations in the images); (2) harnessing large-scale collaboratively-tagged images that are available on the Internet; and, (3) developing new machine learning algorithms that can directly leverage large-scale collaboratively- or loosely-tagged images for achieving more effective training of a large number of object classifiers. Based on these observations, a multi-task multi-label multiple instance learning (MTML-MIL) algorithm is developed in this paper by leveraging both interobject correlations and large-scale loosely-labeled images for object classifier training. By seamlessly integrating multi-task learning, multi-label learning, and multiple instance learning, our MTML-MIL algorithm can achieve more accurate training of a large number of inter-related object classifiers (where an object network is constructed for determining the inter-related learning tasks directly in the feature space rather than in the label space). Our experimental results have shown that our MTML-MIL algorithm can achieve higher detection accuracy rates for automatic object detection.

Structured Max-Margin Learning for Inter-Related Classifier Training and Multilabel Image Annotation

In this paper, a structured max-margin learning algorithm is developed to achieve more effective training of a large number of inter-related classifiers for multilabel image annotation application. To leverage multilabel images for classifier training, each multilabel image is partitioned into a set of image instances (image regions or image patches) and an automatic instance label identification algorithm is developed to assign multiple labels (which are given at the image level) to the most relevant image instances. A K-way min-max cut algorithm is developed for automatic instance clustering and kernel weight determination, where multiple base kernels are seamlessly combined to address the issue of huge intra-concept visual diversity more effectively. Second, a visual concept network is constructed for characterizing the inter-concept visual similarity contexts more precisely in the high-dimensional multimodal feature space. The visual concept network is used to determine the inter-related learning tasks directly in the feature space rather than in the label space because feature space is the common space for classifier training and image classification. Third, a parallel computing platform is developed to achieve more effective learning of a large number of inter-related classifiers over the visual concept network. A structured max-margin learning algorithm is developed by incorporating the visual concept network, max-margin Markov networks and multitask learning to address the issue of huge inter-concept visual similarity more effectively. By leveraging the inter-concept visual similarity contexts for inter-related classifier training, our structured max-margin learning algorithm can significantly enhance the discrimination power of the inter-related classifiers. Our experiments have also obtained very positive results for a large number of object classes and image concepts.

Empirical Capacity of a Recognition Channel for Single- and Multipose Object Recognition Under the Constraint of PCA Encoding

The ability of practical recognition systems to recognize a large number of objects is constrained by a variety of factors that include choice of a feature extraction technique, quality of images, complexity and variability of underlying objects and of collected data. Given a feature extraction technique generating templates of objects from data and a resolution of the original images, the remaining factors can be attributed to distortions due to a recognition channel. We define the recognition channel as the environment that transforms reference templates of objects in a database into templates submitted for recognition. If templates in an object database are generated to be statistically independent and the noise in a query template is statistically independent of templates in the database, then the abilities of the recognition channel to recognize a large number of object classes can be characterized by a number called recognition capacity. In this paper, we evaluate the empirical recognition capacity of PCA-based object recognition systems. The encoded data (templates) and the additive noise in query templates are modeled to be Gaussian distributed with zero mean and estimated variances. We analyze both the case of a single encoded image and the case of encoded correlated multiple images. For this case, we propose a model that is orientation and elevation angle (pose) dependent. The fit of proposed models is judged using statistical goodness of fit tests. We define recognition rate as the ratio R=log(M)/n, where M is the number of objects to recognize and n is the length of PCA templates. The empirical capacity of PCA-based recognition systems is numerically evaluated. The empirical random coding exponent is also numerically evaluated and plotted as a function of the recognition rate. With these results, given a value of the recognition capacity and the length of templates (assume large), we can predict the number of distinct object classes that can be stored in an object library and be identified with probability of error close to zero.