Network For Object Recognition Research Articles

Coupled-learning convolutional neural networks for object recognition

Recently, convolutional neural networks (CNN) have been attracting considerable attention in various computer vision tasks. Motivated by neuroscience, CNN has several similar properties with the learning process of human brain. A prominent difference is that each CNN is an independent learning process while the effective interaction/communication between people can play important role in the human visual system. Inspired by this fact, we proposed a novel Coupled-learning Convolutional Neural Network (Co-CNN) for the task of object recognition, which boosts its discriminative capability by employing the dynamic interaction between neural networks. Contrary to existing network architectures posing the network optimization problem as an isolated learning process, the intuition behind the Co-CNN framework is that the coupled learning mechanism may prevent the algorithm away from over-fitting to one or more particular objective functions. The proposed Co-CNN framework has three unique characteristics: (1) Co-CNN, which is a novel deep network learning framework, can simultaneously optimize both neural networks with same/different structures. (2) The learned semantic information, which can be gradually mined from neural networks, is employed to guide the communication between neural networks. (3) Co-CNN well incorporates the coupled-learning mechanism into the process of learning neural networks, and then further improve the recognition performance of neural networks by adopting the learned semantic information. Comprehensive evaluations on five benchmark datasets (CIFAR-10, CIFAR-100, MNIST, SVHN and Imagenet) well demonstrate the significant superiority of our proposed Co-CNN framework over other existing algorithms.

Convolutional neural network-based encoding and decoding of visual object recognition in space and time

Representations learned by deep convolutional neural networks (CNNs) for object recognition are a widely investigated model of the processing hierarchy in the human visual system. Using functional magnetic resonance imaging, CNN representations of visual stimuli have previously been shown to correspond to processing stages in the ventral and dorsal streams of the visual system. Whether this correspondence between models and brain signals also holds for activity acquired at high temporal resolution has been explored less exhaustively. Here, we addressed this question by combining CNN-based encoding models with magnetoencephalography (MEG). Human participants passively viewed 1,000 images of objects while MEG signals were acquired. We modelled their high temporal resolution source-reconstructed cortical activity with CNNs, and observed a feed-forward sweep across the visual hierarchy between 75 and 200 ms after stimulus onset. This spatiotemporal cascade was captured by the network layer representations, where the increasingly abstract stimulus representation in the hierarchical network model was reflected in different parts of the visual cortex, following the visual ventral stream. We further validated the accuracy of our encoding model by decoding stimulus identity in a left-out validation set of viewed objects, achieving state-of-the-art decoding accuracy.

A study of the effect of noise and occlusion on the accuracy of convolutional neural networks applied to 3D object recognition

In this work, we carry out a study of the effect of adverse conditions, which characterize real-world scenes, on the accuracy of a Convolutional Neural Network applied to 3D object class recognition. Firstly, we discuss possible ways of representing 3D data to feed the network. In addition, we propose a set of representations to be tested. Those representations consist of a grid-like structure (fixed and adaptive) and a measure for the occupancy of each cell of the grid (binary and normalized point density). After that, we propose and implement a Convolutional Neural Network for 3D object recognition using Caffe. At last, we carry out an in-depth study of the performance of the network over a 3D CAD model dataset, the Princeton ModelNet project, synthetically simulating occlusions and noise models featured by common RGB-D sensors. The results show that the volumetric representations for 3D data play a key role on the recognition process and Convolutional Neural Network can be considerably robust to noise and occlusions if a proper representation is chosen.

Fireground location understanding by semantic linking of visual objects and building information models

This paper presents an outline for improved localization and situational awareness in fire emergency situations based on semantic technology and computer vision techniques. The novelty of our methodology lies in the semantic linking of video object recognition results from visual and thermal cameras with Building Information Models (BIM). The current limitations and possibilities of certain building information streams in the context of fire safety or fire incident management are addressed in this paper. Furthermore, our data management tools match higher-level semantic metadata descriptors of BIM and deep-learning based visual object recognition and classification networks. Based on these matches, estimations can be generated of camera, objects and event positions in the BIM model, transforming it from a static source of information into a rich, dynamic data provider. Previous work has already investigated the possibilities to link BIM and low-cost point sensors for fireground understanding, but these approaches did not take into account the benefits of video analysis and recent developments in semantics and feature learning research. Finally, the strengths of the proposed approach compared to the state-of-the-art is its (semi-)automatic workflow, generic and modular setup and multi-modal strategy, which allows to automatically create situational awareness, to improve localization and to facilitate the overall fire understanding.

Deep active object recognition by joint label and action prediction

An active object recognition system has the advantage of acting in the environment to capture images that are more suited for training and lead to better performance at test time. In this paper, we utilize deep convolutional neural networks for active object recognition by simultaneously predicting the object label and the next action to be performed on the object with the aim of improving recognition performance. We treat active object recognition as a reinforcement learning problem and derive the cost function to train the network for joint prediction of the object label and the action. A generative model of object similarities based on the Dirichlet distribution is proposed and embedded in the network for encoding the state of the system. The training is carried out by simultaneously minimizing the label and action prediction errors using gradient descent. We empirically show that the proposed network is able to predict both the object label and the actions on GERMS, a dataset for active object recognition. We compare the test label prediction accuracy of the proposed model with Dirichlet and Naive Bayes state encoding. The results of experiments suggest that the proposed model equipped with Dirichlet state encoding is superior in performance, and selects images that lead to better training and higher accuracy of label prediction at test time.

Inferring brain-computational mechanisms with models of activity measurements.

High-resolution functional imaging is providing increasingly rich measurements of brain activity in animals and humans. A major challenge is to leverage such data to gain insight into the brain's computational mechanisms. The first step is to define candidate brain-computational models (BCMs) that can perform the behavioural task in question. We would then like to infer which of the candidate BCMs best accounts for measured brain-activity data. Here we describe a method that complements each BCM by a measurement model (MM), which simulates the way the brain-activity measurements reflect neuronal activity (e.g. local averaging in functional magnetic resonance imaging (fMRI) voxels or sparse sampling in array recordings). The resulting generative model (BCM-MM) produces simulated measurements. To avoid having to fit the MM to predict each individual measurement channel of the brain-activity data, we compare the measured and predicted data at the level of summary statistics. We describe a novel particular implementation of this approach, called probabilistic representational similarity analysis (pRSA) with MMs, which uses representational dissimilarity matrices (RDMs) as the summary statistics. We validate this method by simulations of fMRI measurements (locally averaging voxels) based on a deep convolutional neural network for visual object recognition. Results indicate that the way the measurements sample the activity patterns strongly affects the apparent representational dissimilarities. However, modelling of the measurement process can account for these effects, and different BCMs remain distinguishable even under substantial noise. The pRSA method enables us to perform Bayesian inference on the set of BCMs and to recognize the data-generating model in each case.This article is part of the themed issue ‘Interpreting BOLD: a dialogue between cognitive and cellular neuroscience’.

SU‐D‐201‐05: On the Automatic Recognition of Patient Safety Hazards in a Radiotherapy Setup Using a Novel 3D Camera System and a Deep Learning Framework

Purpose:Patient safety hazards such as a wrong patient/site getting treated can lead to catastrophic results. The purpose of this project is to automatically detect potential patient safety hazards during the radiotherapy setup and alert the therapist before the treatment is initiated.Methods:We employed a set of co‐located and co‐registered 3D cameras placed inside the treatment room. Each camera provided a point‐cloud of fraxels (fragment pixels with 3D depth information). Each of the cameras were calibrated using a custom‐built calibration target to provide 3D information with less than 2 mm error in the 500 mm neighborhood around the isocenter. To identify potential patient safety hazards, the treatment room components and the patient's body needed to be identified and tracked in real‐time. For feature recognition purposes, we used a graph‐cut based feature recognition with principal component analysis (PCA) based feature‐to‐object correlation to segment the objects in real‐time. Changes in the object's position were tracked using the CamShift algorithm. The 3D object information was then stored for each classified object (e.g. gantry, couch). A deep learning framework was then used to analyze all the classified objects in both 2D and 3D and was then used to fine‐tune a convolutional network for object recognition. The number of network layers were optimized to identify the tracked objects with >95% accuracy.Results:Our systematic analyses showed that, the system was effectively able to recognize wrong patient setups and wrong patient accessories. The combined usage of 2D camera information (color + depth) enabled a topology‐preserving approach to verify patient safety hazards in an automatic manner and even in scenarios where the depth information is partially available.Conclusion:By utilizing the 3D cameras inside the treatment room and a deep learning based image classification, potential patient safety hazards can be effectively avoided.

Shape-based object recognition via Evidence Accumulation Inference

Shape-based object recognition is one of the most challenging problems in computer vision. Learning a structural representation using graphical models is a new trend in object recognition. This paper tries to apply graphical models to learn a shape representation and proposes a pipeline of shape-based object recognition. First, a Bayesian Network represents the shape knowledge of a type of object. Second, an Evidence Accumulation Inference with Bayesian Network is developed to search for the region of interest which is most likely to contain an object in an image. Finally, a spatial pyramid matching approach is used to verify the hypothesis to identify objects and to refine object locations. Our experiments corroborate that Evidence Accumulation Inference with Bayesian Network for object recognition is correct and show that the proposed pipeline achieves comparable results on well-known ETHZ shape classes and INRIA Horse dataset.

Atoms of recognition in human and computer vision

Discovering the visual features and representations used by the brain to recognize objects is a central problem in the study of vision. Recently, neural network models of visual object recognition, including biological and deep network models, have shown remarkable progress and have begun to rival human performance in some challenging tasks. These models are trained on image examples and learn to extract features and representations and to use them for categorization. It remains unclear, however, whether the representations and learning processes discovered by current models are similar to those used by the human visual system. Here we show, by introducing and using minimal recognizable images, that the human visual system uses features and processes that are not used by current models and that are critical for recognition. We found by psychophysical studies that at the level of minimal recognizable images a minute change in the image can have a drastic effect on recognition, thus identifying features that are critical for the task. Simulations then showed that current models cannot explain this sensitivity to precise feature configurations and, more generally, do not learn to recognize minimal images at a human level. The role of the features shown here is revealed uniquely at the minimal level, where the contribution of each feature is essential. A full understanding of the learning and use of such features will extend our understanding of visual recognition and its cortical mechanisms and will enhance the capacity of computational models to learn from visual experience and to deal with recognition and detailed image interpretation.

Deep Learning for Intelligent Substation Device Infrared Fault Image Analysis

As an important kind of data for device status evaluation, the increasing infrared image data in electrical system puts forward a new challenge to traditional manually processing mode. To overcome this problem, this paper proposes a feasible way to automatically process massive infrared fault images. We take advantage of the imaging characteristics of infrared fault images and detect fault regions together with its belonging device part by our proposed algorithm, which first segment images into superpixels, and then adopt the state-of-the-art convolutional and recursive neural network for intelligent object recognition. In the experiment, we compare several unsupervised pre-training methods considering the importance of a pre-train procedure, and discuss the proper parameters for the proposed network. The experimental results show the good performance of our algorithm, and its efficiency for infrared analysis.

BLSTM Recurrent Neural Network for Object Recognition

Multi-object relationship information can help eliminate some incorrect combinations or locations of objects. Moreover, it is favorable to extract scene information for object recognition. In this paper, we introduce a new way to generate image representation and propose a deep learning framework to fuse the contextual dependencies among objects and scene information in an image. It adopts a bidirectional long short-term memory recurrent neural network (BLSTM-RNN) to deal with the problem of variable-length sequence produced by local detectors in different images. Then it is applied to the existing tree context model for further recognition. Experimental results on SUN09 dataset show that our model outperforms the state-of the-art object localization methods.

HFirst: A Temporal Approach to Object Recognition.

This paper introduces a spiking hierarchical model for object recognition which utilizes the precise timing information inherently present in the output of biologically inspired asynchronous address event representation (AER) vision sensors. The asynchronous nature of these systems frees computation and communication from the rigid predetermined timing enforced by system clocks in conventional systems. Freedom from rigid timing constraints opens the possibility of using true timing to our advantage in computation. We show not only how timing can be used in object recognition, but also how it can in fact simplify computation. Specifically, we rely on a simple temporal-winner-take-all rather than more computationally intensive synchronous operations typically used in biologically inspired neural networks for object recognition. This approach to visual computation represents a major paradigm shift from conventional clocked systems and can find application in other sensory modalities and computational tasks. We showcase effectiveness of the approach by achieving the highest reported accuracy to date (97.5% ± 3.5%) for a previously published four class card pip recognition task and an accuracy of 84.9% ± 1.9% for a new more difficult 36 class character recognition task.

Object recognition in remote sensing images using sparse deep belief networks

Object recognition has been one of the hottest issues in the field of remote sensing image analysis. In this letter, a new pixel-wise learning method based on deep belief networks (DBNs) for object recognition is proposed. The method is divided into two stages, the unsupervised pre-training stage and the supervised fine-tuning stage. Given a training set of images, a pixel-wise unsupervised feature learning algorithm is utilized to train a mixed structural sparse restricted Boltzmann machine (RBM). After that, the outputs of this RBM are put into the next RBM as inputs. By stacking several layers of RBM, the deep generative model of DBNs is built. At the fine-tuning stage, a supervised layer is attached to the top of the DBN and labels of the data are put into this layer. The whole network is then trained using the back-propagation (BP) algorithm with sparse penalty. Finally, the deep model generates good joint distribution of images and their labels. Comparative experiments are conducted on our dataset acquired by QuickBird with 60 cm resolution and the recognition results demonstrate the accuracy and efficiency of our proposed method.

Channel-level acceleration of deep face representations

A major challenge in biometrics is performing the test at the client side, where hardware resources are often limited. Deep learning approaches pose a unique challenge: while such architectures dominate the field of face recognition with regard to accuracy, they require elaborate, multi-stage computations. Recently, there has been some work on compressing networks for the purpose of reducing run time and network size. However, it is not clear that these compression methods would work in deep face nets, which are, generally speaking, less redundant than the object recognition networks, i.e., they are already relatively lean. We propose two novel methods for compression: one based on eliminating lowly active channels and the other on coupling pruning with repeated use of already computed elements. Pruning of entire channels is an appealing idea, since it leads to direct saving in run time in almost every reasonable architecture.

Learning Block Group Spase Representation Combined with Convolutional Neural Networks for RGB-D Object Recognition

Learning Block Group Spase Representation Combined with Convolutional Neural Networks for RGB-D Object Recognition

Words and objects at the tip of the left temporal lobe in primary progressive aphasia

Eleven of 69 prospectively enrolled primary progressive aphasics were selected for this study because of peak atrophy sites located predominantly or exclusively within the anterior left temporal lobe. Cortical volumes in these areas were reduced to less than half of control values, whereas average volume elsewhere in the left hemisphere deviated from control values by only 8%. Failure to name objects emerged as the most consistent and severe deficit. Naming errors were attributed to pure retrieval failure if the object could not be named even when the denoting word was understood, the object recognized and the two accurately matched. Surprisingly many of the naming errors reflected pure retrieval failures, without discernible semantic or associative component. The remaining set of errors had associative components. These errors reflected the inability to define the word denoting the object more often than the inability to define the nature of the pictured object. In a separate task where the same object had to be linked to verbal or non-verbal associations, performance was abnormal only in the verbal format. Excessive taxonomic interference was observed for picture-word, but not picture-picture, matching tasks. This excessive interference reflected a blurring of intra- rather than inter-category distinctions as if the acuity of word-object associations had been diminished so that correspondences were easier to recognize at generic than specific levels. These dissociations between verbal and non-verbal markers of object knowledge indicate that the reduced neural mass at peak atrophy sites of the left temporal tip, accounting for half or more of the presumed premorbid volume, was unlikely to have contained domain-independent semantic representations of the type that would be expected in a strictly amodal hub. A more likely arrangement entails two highly interactive routes--a strongly left lateralized temporosylvian language network for verbal concepts, and a presumably more bilateral or right-sided inferotemporal/fusiform object recognition network, which remained relatively spared because peak atrophy sites were concentrated on the left. The current results also suggest that the left anterior temporal neocortex should be inserted into the language network where it is likely to play a major role in selecting verbal labels for objects and mediating the progression of word comprehension from generic to specific levels of precision.

Deep Learning

Hierarchical neural networks for object recognition have a long history. In recent years, novel methods for incrementally learning a hierarchy of features from unlabeled inputs were proposed as good starting point for supervised training. These deep learning methods—together with the advances of parallel computers—made it possible to successfully attack problems that were not practical before, in terms of depth and input size. In this article, we introduce the reader to the basic concepts of deep learning, discuss selected methods in detail, and present application examples from computer vision and speech recognition.

An Event-Driven Multi-Kernel Convolution Processor Module for Event-Driven Vision Sensors

Event-Driven vision sensing is a new way of sensing visual reality in a frame-free manner. This is, the vision sensor (camera) is not capturing a sequence of still frames, as in conventional video and computer vision systems. In Event-Driven sensors each pixel autonomously and asynchronously decides when to send its address out. This way, the sensor output is a continuous stream of address events representing reality dynamically continuously and without constraining to frames. In this paper we present an Event-Driven Convolution Module for computing 2D convolutions on such event streams. The Convolution Module has been designed to assemble many of them for building modular and hierarchical Convolutional Neural Networks for robust shape and pose invariant object recognition. The Convolution Module has multi-kernel capability. This is, it will select the convolution kernel depending on the origin of the event. A proof-of-concept test prototype has been fabricated in a 0.35 μm CMOS process and extensive experimental results are provided. The Convolution Processor has also been combined with an Event-Driven Dynamic Vision Sensor (DVS) for high-speed recognition examples. The chip can discriminate propellers rotating at 2 k revolutions per second, detect symbols on a 52 card deck when browsing all cards in 410 ms, or detect and follow the center of a phosphor oscilloscope trace rotating at 5 KHz.

Biologically plausible saliency mechanisms improve feedforward object recognition

The biological plausibility of statistical inference and learning, tuned to the statistics of natural images, is investigated. It is shown that a rich family of statistical decision rules, confidence measures, and risk estimates, can be implemented with the computations attributed to the standard neurophysiological model of V1. In particular, different statistical quantities can be computed through simple re-arrangement of lateral divisive connections, non-linearities, and pooling. It is then shown that a number of proposals for the measurement of visual saliency can be implemented in a biologically plausible manner, through such re-arrangements. This enables the implementation of biologically plausible feedforward object recognition networks that include explicit saliency models. The potential of combined attention and recognition is illustrated by replacing the first layer of the HMAX architecture with a saliency network. Various saliency measures are compared, to investigate whether (1) saliency can substantially benefit visual recognition and (2) the benefits depend on the specific saliency mechanisms implemented. Experimental evaluation shows that saliency does indeed enhance recognition, but the gains are not independent of the saliency mechanisms. Best results are obtained with top-down mechanisms that equate saliency to classification confidence.

SINNOR: Evaluating a simple image-based neural network for object (and face) recognition

SINNOR: Evaluating a simple image-based neural network for object (and face) recognition