Problem Of Object Recognition Research Articles

Application of neural networks in object recognition tasks for ADAS systems

Modern driver assistance systems (ADAS) require an environmental recognition function to inform the driver and for making management decisions. Neural networks are used to select and recognize objects in such systems. The paper presents the results of comparative analysis of various neural networks in object recognition problems. Experimental data showed that convolutional neural networks show the best results in recognition problems.

Energy Efficiency Enhancement for CNN-based Deep Mobile Sensing

Recently, deep learning has been used to tackle mobile sensing problems, and the inference phase of deep learning is preferred to be run on mobile devices for speedy responses. However, mobile devices are resource-constrained platforms for both computation and power. Moreover, an inference task with deep learning involves tens of billions of mathematical operations and tens of millions of parameter reads. Thus, it is a critical issue to reduce the energy consumption of deep learning inference algorithms. In this article, we survey various energy reduction approaches, and classify them into three categories: the compressing neural network model, minimizing the data transfer required in computation, and offloading workloads. Moreover, we simulate and compare three techniques of model compression, by applying them to an object recognition problem.

Granulated deep learning and Z-numbers in motion detection and object recognition

The article deals with the problems of motion detection, object recognition, and scene description using deep learning in the framework of granular computing and Z-numbers. Since deep learning is computationally intensive, whereas granular computing, on the other hand, leads to computation gain, a judicious integration of their merits is made so as to make the learning mechanism computationally efficient. Further, it is shown how the concept of z-numbers can be used to quantify the abstraction of semantic information in interpreting a scene, where subjectivity is of major concern, through recognition of its constituting objects. The system, thus developed, involves recognition of both static objects in the background and moving objects in foreground separately. Rough set theoretic granular computing is adopted where rough lower and upper approximations are used in defining object and background models. During deep learning, instead of scanning the entire image pixel by pixel in the convolution layer, we scan only the representative pixel of each granule. This results in a significant gain in computation time. Arbitrary-shaped and sized granules, as expected, perform better than regular-shaped rectangular granules or fixed-sized granules. The method of tracking is able to deal efficiently with various challenging cases, e.g., tracking partially overlapped objects and suddenly appeared objects. Overall, the granulated system shows a balanced trade-off between speed and accuracy as compared to pixel level learning in tracking and recognition. The concept of using Z-numbers, in providing a granulated linguistic description of a scene, is unique. This gives a more natural interpretation of object recognition in terms of certainty toward scene understanding.

A Novel Weakly-Supervised Approach for RGB-D-Based Nuclear Waste Object Detection

This paper addresses the problem of RGBD object recognition in real-world applications, where large amounts of annotated training data are typically unavailable. To overcome this problem, we propose a novel, weakly-supervised learning architecture (DCNN-GPC) which combines parametric models (a pair of Deep Convolutional Neural Networks (DCNN) for RGB and D modalities) with non-parametric models (Gaussian Process Classification). Our system is initially trained using a small amount of labeled data, and then automatically prop- agates labels to large-scale unlabeled data. We first run 3D- based objectness detection on RGBD videos to acquire many unlabeled object proposals, and then employ DCNN-GPC to label them. As a result, our multi-modal DCNN can be trained end-to-end using only a small amount of human annotation. Finally, our 3D-based objectness detection and multi-modal DCNN are integrated into a real-time detection and recognition pipeline. In our approach, bounding-box annotations are not required and boundary-aware detection is achieved. We also propose a novel way to pretrain a DCNN for the depth modality, by training on virtual depth images projected from CAD models. We pretrain our multi-modal DCNN on public 3D datasets, achieving performance comparable to state-of-the-art methods on Washington RGBS Dataset. We then finetune the network by further training on a small amount of annotated data from our novel dataset of industrial objects (nuclear waste simulants). Our weakly supervised approach has demonstrated to be highly effective in solving a novel RGBD object recognition application which lacks of human annotations.

Design of visual communication based on deep learning approaches

Aiming at the problem of object recognition caused by small object scale, multi-interaction (occlusion), and strong hiding characteristics in the scene analysis task, an object-region-enhanced network based on deep learning was proposed. The network integrated two core modules designed for the task: object area enhancement strategy and black-hole-filling strategy. The former directly corresponded the object region with high semantic confidence to the local region of the specific category channel of the convolutional feature image. Weighted features were used to improve contextual relationships, and difficult object regions were identified. The latter avoided the mistake of identifying some difficult areas as additional background classes by masking additional background classes. The results showed that the modular design scheme improved the overall parsing performance of the model by replacing the modules, and the two strategies were applied to other existing scenario parsing networks. A unified framework is proposed for handling scene resolution tasks. Benefiting from the modular design approach, the proposed algorithm improves overall performance by replacing convolution or detection modules. Object enhancement and black hole filling are applied to other systems to improve the system’s ability to parse objects. Object area enhancement methods are used to recall objects that are not recognized in a standard split network. Black hole fill techniques can be used to resolve pixels that are incorrectly categorized into additional background classes that do not exist. Therefore, a variety of contextual semantic fusion strategies have certain reference value in the theoretical level of computer vision. More critically, this method has certain reference significance for the design and development of robust and practical application systems.

Towards an automatic filament detector with a Faster R-CNN on MAST-U

In the present magnetically confined plasmas, the prediction of particle loading on material surfaces is a primary concern in view of the protection of plasma facing components for next step devices. Thus, an understanding of filament dynamics is needed. In this context, this work aims to develop an automatic detector for filaments arising in the MAST-U plasma. The identification of the filaments has been done starting from 2D images acquired with a fast visible camera. Therefore, it can be faced as an image object recognition problem. Currently, the object recognition is a key output of deep learning and machine learning algorithms. In this paper, a database of several thousands of images generated by a synthetic diagnostic, which reproduces the statistical properties of experimental filaments in terms of position, size and intensity has been used. The synthetic images are pre-processed by mapping them onto the toroidal midplane of the machine. Then a Faster R-CNN is customized to the problem of identifying the filaments. In particular, in order to enhance the performance of the detector, a suitable definition of the target-boxes defining the filament positions and sizes is adopted with good results.

DPRNet: Deep 3D Point Based Residual Network for Semantic Segmentation and Classification of 3D Point Clouds

Point clouds are an important type of geometric data obtained from a variety of 3D sensors. They do not have an explicit neighborhood structure and therefore several researchers often perform a voxelization step to obtain structured 3D neighborhood. This, however, comes with certain disadvantages, e.g., it makes the data unnecessarily voluminous, enforces additional computation effort and can potentially introduce quantization errors that may not only hinder in extracting implicit 3D shape information but also in capturing the essential data invariances for the required segmentation and recognition task. In this context, this paper addresses the highly challenging problem of semantic segmentation and 3D object recognition using raw unstructured 3D point cloud data. Specifically, the deep network architecture has been proposed which consists of a cascaded combination of 3D point-based residual networks for simultaneous semantic scene segmentation and object classification. It exploits the 3D point-based convolutions for representational learning from raw unstructured 3D point cloud data. The proposed architecture has a simple design, easier implementation, and the performance which is better than the existing state-of-the architectures particularly for semantic scene segmentation over three public datasets. The implementation and evaluation are made public here https://github.com/saira05/DPRNet.

DETERMINATION OF RELEVANCE OF VISUAL OBJECT IMAGES BY APPLICATION OF STATISTICAL ANALYSIS OF REGARDING FRAGMENT REPRESENTATION OF THEIR DESCRIPTIONS

The problem of visual object recognition in the space features of descriptors of special image points is solved with the application of statistical distributions for a system of fragments of image description. The efficiency of application of the distribution variants for single- and multi-bit sizes of the fragments of binary descriptors in the calculation of the level of relevance of descriptions in the process of object recognition is analyzed. Using software simulation, an experimental evaluation of the computation time of the relevance value in the constructed feature space is carried out compared to the traditional voting approach of the set of descriptors. The efficiency of the proposed approach to data representation in terms of a significant increase in the performance rate with ensuring a sufficient level of recognition quality is confirmed.

Real-World ISAR Object Recognition and Relation Discovery Using Deep Relation Graph Learning

Real-world inverse synthetic aperture radar (ISAR) object recognition is the most critical and challenging problem in computer vision tasks. In this paper, an efficient real-world ISAR object recognition and relation discovery method are proposed, based on deep relation graph learning. It not only handles the real-world object recognition problem efficiently, but also exploits the inter-modal relationships among features, attributes, and classes with semantic knowledge. First, dilated deformable convolutional neural network, including dilated deformable convolution and dilated deformable location-aware RoI pooling, is introduced to greatly improve CNNs' sampling and transformation ability, and increase the output feature maps' resolutions significantly. And a related multi-modal regions ranking strategy is proposed. Second, deep graph attribute-association learning is proposed to jointly estimate a large number of multi-heterogeneous attributes, and leverage features, attributes, and semantic knowledge to learn their relations. Third, multi-scale relational-regularized convolutional sparse learning is proposed to further improve the accuracy and speed of the whole system. The extensive experiments are performed on two real-world ISAR datasets, showing our proposed method outperforms the state-of-the-art methods.

Exploring to learn visual saliency: The RL-IAC approach

The problem of object localization and recognition on autonomous mobile robots is still an active topic. In this context, we tackle the problem of learning a model of visual saliency directly on a robot. This model, learned and improved on-the-fly during the robot’s exploration provides an efficient tool for localizing relevant objects within their environment. The proposed approach includes two intertwined components. On the one hand, we describe a method for learning and incrementally updating a model of visual saliency from a depth-based object detector. This model of saliency can also be exploited to produce bounding box proposals around objects of interest. On the other hand, we investigate an autonomous exploration technique to efficiently learn such a saliency model. The proposed exploration, called Reinforcement Learning-Intelligent Adaptive Curiosity (RL-IAC) is able to drive the robot’s exploration so that samples selected by the robot are likely to improve the current model of saliency. We then demonstrate that such a saliency model learned directly on a robot outperforms several state-of-the-art saliency techniques, and that RL-IAC can drastically decrease the required time for learning a reliable saliency model.

Motion Planning and Reactive Control Algorithms for Object Manipulation in Uncertain Conditions

This work proposes the application of several smart strategies for object manipulation tasks. A real-time flexible motion planning method was developed to be adapted to typical in-store logistics scenarios. The solution combines and optimizes some state-of-the-art techniques to solve object recognition and localization problems with a new hybrid pipeline. The algorithm guarantees good robustness and accuracy for object detection through depth images. A standard planner plans collision-free trajectories throughout the whole task while a proposed reactive motion control is active. Distributed proximity sensors were adopted to locally modify the planned trajectory when unexpected or misplaced obstacles intervene in the scene. To implement a robust grasping phase, a novel slipping control algorithm was used. It dynamically computes the grasp force by adapting it to the actual object physical properties so as to prevent slipping. Experimental results carried out in a typical supermarket scenario demonstrate the effectiveness of the presented methods.

Indoor object recognition in RGBD images with complex-valued neural networks for visually-impaired people

We present a new multi-modal technique for assisting visually-impaired people in recognizing objects in public indoor environment. Unlike common methods which aim to solve the problem of multi-class object recognition in a traditional single-label strategy, a comprehensive approach is developed here allowing samples to take more than one label at a time. We jointly use appearance and depth cues, specifically RGBD images, to overcome issues of traditional vision systems using a new complex-valued representation. Inspired by complex-valued neural networks (CVNNs) and multi-label learning techniques, we propose two methods in order to associate each input RGBD image to a set of labels corresponding to the object categories recognized at once. The first one, ML-CVNN, is formalized as a ranking strategy where we make use of a fully complex-valued RBF network and extend it to be able to solve multi-label problems using an adaptive clustering method. The second method, L-CVNNs, deals with problem transformation strategy where instead of using a single network to formalize the classification problem as a ranking solution for the whole label set, we propose to construct one CVNN for each label where the predicted labels will be later aggregated to construct the resulting multi-label vector. Extensive experiments have been carried on two newly collected multi-labeled RGBD datasets prove the efficiency of the proposed techniques.

Evolutionary Hierarchical Sparse Extreme Learning Autoencoder Network for Object Recognition

Extreme learning machine (ELM), characterized by its fast learning efficiency and great generalization ability, has been applied to various object recognition tasks. When extended to the stacked autoencoder network, which is a typical symmetrical representation learning model architecture, ELM manages to realize hierarchical feature extraction and classification, which is what deep neural networks usually do, but with much less training time. Nevertheless, the input weights and biases of the hidden nodes in ELM are generated according to a random distribution and may lead to the occurrence of non-optimal and redundant parameters that deteriorate discriminative features, which will have a bad influence on the final classification effect. In this paper, a novel sparse autoencoder derived from ELM and differential evolution is proposed and integrated into a hierarchical hybrid autoencoder network to accomplish the end-to-end learning with raw visible light camera sensor images and applied to several typical object recognition problems. Experimental results show that the proposed method is able to obtain competitive or better performance than current relevant methods with acceptable or less time consumption.

A Hybrid Geometric Spatial Image Representation for scene classification.

The recent development in the technology has increased the complexity of image contents and demand for image classification becomes more imperative. Digital images play a vital role in many applied domains such as remote sensing, scene analysis, medical care, textile industry and crime investigation. Feature extraction and image representation is considered as an important step in scene analysis as it affects the image classification performance. Automatic classification of images is an open research problem for image analysis and pattern recognition applications. The Bag-of-Features (BoF) model is commonly used to solve image classification, object recognition and other computer vision-based problems. In BoF model, the final feature vector representation of an image contains no information about the co-occurrence of features in the 2D image space. This is considered as a limitation, as the spatial arrangement among visual words in image space contains the information that is beneficial for image representation and learning of classification model. To deal with this, researchers have proposed different image representations. Among these, the division of image-space into different geometric sub-regions for the extraction of histogram for BoF model is considered as a notable contribution for the extraction of spatial clues. Keeping this in view, we aim to explore a Hybrid Geometric Spatial Image Representation (HGSIR) that is based on the combination of histograms computed over the rectangular, triangular and circular regions of the image. Five standard image datasets are used to evaluate the performance of the proposed research. The quantitative analysis demonstrates that the proposed research outperforms the state-of-art research in terms of classification accuracy.

Image classification by addition of spatial information based on histograms of orthogonal vectors.

The Bag-of-Visual-Words (BoVW) model is widely used for image classification, object recognition and image retrieval problems. In BoVW model, the local features are quantized and 2-D image space is represented in the form of order-less histogram of visual words. The image classification performance suffers due to the order-less representation of image. This paper presents a novel image representation that incorporates the spatial information to the inverted index of BoVW model. The spatial information is added by calculating the global relative spatial orientation of visual words in a rotation invariant manner. For this, we computed the geometric relationship between triplets of identical visual words by calculating an orthogonal vector relative to each point in the triplets of identical visual words. The histogram of visual words is calculated on the basis of the magnitude of these orthogonal vectors. This calculation provides the unique information regarding the relative position of visual words when they are collinear. The proposed image representation is evaluated by using four standard image benchmarks. The experimental results and quantitative comparisons demonstrate that the proposed image representation outperforms the existing state-of-the-art in terms of classification accuracy.

MSANet: multimodal self-augmentation and adversarial network for RGB-D object recognition

This paper researches on the problem of object recognition using RGB-D data. Although deep convolutional neural networks have so far made progress in this area, they are still suffering a lot from lack of large-scale manually labeled RGB-D data. Labeling large-scale RGB-D dataset is a time-consuming and boring task. More importantly, such large-scale datasets often exist a long tail, and those hard positive examples of the tail can hardly be recognized. To solve these problems, we propose a multimodal self-augmentation and adversarial network (MSANet) for RGB-D object recognition, which can augment the data effectively at two levels while keeping the annotations. Toward the first level, series of transformations are leveraged to generate class-agnostic examples for each instance, which supports the training of our MSANet. Toward the second level, an adversarial network is proposed to generate class-specific hard positive examples while learning to classify them correctly to further improve the performance of our MSANet. Via the above schemes, the proposed approach wins the best results on several available RGB-D object recognition datasets, e.g., our experimental results indicate a 1.5% accuracy boost on benchmark Washington RGB-D object dataset compared with the current state of the art.

Behavior of vegetation/soil indices in shaded and sunlit pixels and evaluation of different shadow compensation methods using UAV high-resolution imagery over vineyards.

In high-resolution imagery, shadows may cause problems in object segmentation and recognition due to their low reflectance. For instance, the spectral reflectance of shadows and water are similar, particularly in the visible band. In precision agriculture, the vegetation condition in terms of plant water use, plant water stress, and chlorophyll content can be estimated using vegetation indices. Normalized Difference Vegetation Index (NDVI), Leaf Area Index (LAI) and Enhanced Vegetation Index (EVI) are widely used vegetation indices for characterizing the condition of the vegetation. In addition, many soil indices have been developed for describing soil characteristics, such as Soil-Adjusted Vegetation Index (SAVI). However, shadows can have an influence on the performance of these vegetation and soil indices.Moreover, enhancing spatial resolution heightens the impact of shadows in the imagery. In this study, the behavior of vegetation and soil indices are evaluated using four sets of high-resolution imagery captured by the Utah State University AggieAir unmanned aerial vehicle (UAV) system. These indices were obtained from flights conducted in 2014, 2015, and 2016 over a commercial vineyard located in California for the USDA Agricultural Research Service Grape Remote sensing Atmospheric Profile and Evapotranspiration Experiment (GRAPEX) Program. Different shadow restoration methods are used to alleviate the impact of shadows in information products that might be developed from the high-resolution imagery. The histogram pattern of vegetation and soil indices before and after shadow compensation, are compared using sanalysi of variance (ANOVA). The results of this study indicate how shadows can affect the vegetation/soil indices and whether shadow compensation methods are able to remove the statistical difference between sunlit and shadowed vegetation/soil indices.

Multi‐image matching for object recognition

One of the central problems in object recognition is to develop appropriate representations for the objects in images. The authors present a novel approach for image representation that is based on graphs. In the proposed image graph, each node represents a patch and edges are added between neighbouring nodes. First, class‐specific match‐set graphs are generated by matching the image graphs that are in the same categories, and the multi‐image matching problem is solved by applying a seed‐expansion strategy. Then, the matches between the match‐set graphs and an image graph are considered to be the object patches in the image. Finally, the features extracted from these patches are used for the image representation. Extensive experiments are conducted to demonstrate that their approach can obtain state‐of‐the‐art results on several challenging datasets.

Intelligent image classification-based on spatial weighted histograms of concentric circles

As digital images play a vital role in multimedia content, the automatic classification of images is an open research problem. The Bag of Visual Words (BoVW) model is used for image classification, retrieval and object recognition problems. In the BoVW model, a histogram of visual words is computed without considering the spatial layout of the 2-D image space. The performance of BoVW suffers due to a lack of information about spatial details of an image. Spatial Pyramid Matching (SPM) is a popular technique that computes the spatial layout of the 2-D image space. However, SPM is not rotation-invariant and does not allow a change in pose and view point, and it represents the image in a very high dimensional space. In this paper, the spatial contents of an image are added and the rotations are dealt with efficiently, as compared to approaches that incorporate spatial contents. The spatial information is added by constructing the histogram of circles, while rotations are dealt with by using concentric circles. A weighed scheme is applied to represent the image in the form of a histogram of visual words. Extensive evaluation of benchmark datasets and the comparison with recent classification models demonstrate the effectiveness of the proposed approach. The proposed representation outperforms the state-of-the-art methods in terms of classification accuracy.

Texture Filters and Fractal Dimension on Image Segmentation

Texture analysis is important in several image segmentation and classification problems. Different image textures manifest themselves by dissimilarity in both the property values and the spatial interrelationships of their component texture primitives. We use this fact in a texture discrimination system. This paper focuses on how to apply texture operators based on co-occurrence matrix, texture filters and fractal dimension to the problem of object recognition and image segmentation.