Object Classification Performance Research Articles

Hierarchical Sparse Coding of Objects in Deep Convolutional Neural Networks.

Recently, deep convolutional neural networks (DCNNs) have attained human-level performances on challenging object recognition tasks owing to their complex internal representation. However, it remains unclear how objects are represented in DCNNs with an overwhelming number of features and non-linear operations. In parallel, the same question has been extensively studied in primates' brain, and three types of coding schemes have been found: one object is coded by the entire neuronal population (distributed coding), or by one single neuron (local coding), or by a subset of neuronal population (sparse coding). Here we asked whether DCNNs adopted any of these coding schemes to represent objects. Specifically, we used the population sparseness index, which is widely-used in neurophysiological studies on primates' brain, to characterize the degree of sparseness at each layer in representative DCNNs pretrained for object categorization. We found that the sparse coding scheme was adopted at all layers of the DCNNs, and the degree of sparseness increased along the hierarchy. That is, the coding scheme shifted from distributed-like coding at lower layers to local-like coding at higher layers. Further, the degree of sparseness was positively correlated with DCNNs' performance in object categorization, suggesting that the coding scheme was related to behavioral performance. Finally, with the lesion approach, we demonstrated that both external learning experiences and built-in gating operations were necessary to construct such a hierarchical coding scheme. In sum, our study provides direct evidence that DCNNs adopted a hierarchically-evolved sparse coding scheme as the biological brain does, suggesting the possibility of an implementation-independent principle underling object recognition.

VB-Net: Voxel-Based Broad Learning Network for 3D Object Classification

Point clouds have been widely used in three-dimensional (3D) object classification tasks, i.e., people recognition in unmanned ground vehicles. However, the irregular data format of point clouds and the large number of parameters in deep learning networks affect the performance of object classification. This paper develops a 3D object classification system using a broad learning system (BLS) with a feature extractor called VB-Net. First, raw point clouds are voxelized into voxels. Through this step, irregular point clouds are converted into regular voxels which are easily processed by the feature extractor. Then, a pre-trained VoxNet is employed as a feature extractor to extract features from voxels. Finally, those features are used for object classification by the applied BLS. The proposed system is tested on the ModelNet40 dataset and ModelNet10 dataset. The average recognition accuracy was 83.99% and 90.08%, respectively. Compared to deep learning networks, the time consumption of the proposed system is significantly decreased.

Single shot object detection with refined feature

Object classification and localization are two significant aspects of object detector based on the Single Shot MultiBox Detector (SSD). In general, the more feature maps there are, the better the object classification performance will be. However, when the information of excessive feature maps are sparse and unnecessary, the performance of object detection is slightly improved or maybe precisely opposite, which is instead harmful to the production of object localization. The performance of object detectors is not only related to the number of feature maps but also relies partly on the bounding box regression and Non-Maximum Suppression (NMS). In this paper, a detector is constructed based on SSD, called Detection with Refined Feature (DRF), involving center map and scale map, the detection loss is reshaped. Our motivation is to improve the accuracy of classification and localization by searching for central points and predicting the scales of the object points. Center map is used to predict the Intersection over Union (IoU) between the prediction box and ground truth box, while scale map considers the relationships among the different scales. Experimental results on both Pascal VOC and MS COCO 2014 instance datasets demonstrate the effectiveness of DRF. Using Darknet53, we achieve an 86.4% mean Average Precision (mAP) on Pascal VOC2007 and an 87.4% mAP on Pascal VOC2007 and VOC2012. On MS COCO, the DRF with ResNet50 still achieves moderate improvement.

Object-Based Image Analysis Using Harmonic Analysis on A High-Spatial Resolution Satellite Image

ABSTRACT The ability to extract biophysical and structural properties of features in digital images is an important goal in remote sensing. Delineating the boundaries of image features, and building meaningful numerical descriptors of them based on their geometric shape, are both challenging tasks that are necessary to identify shape pattern in image data. This study seeks to evaluate and compare the performance of Fourier harmonic descriptors against dimensionless ratios of image-object shape for characterizing objects in image classification training samples. We took 150 random samples of three different object types – forest canopies, residential houses, and buildings (50 samples each) – from a high spatial-resolution satellite image (WorldView-2) covering a portion of the southern end of California’s Great Central Valley. To identify patterns in object shapes and perform object classification, we followed three steps. First, we performed image segmentation through an object-based image analysis (OBIA) method based on the Watershed Transform. We then carried out shape characterization using Fourier harmonics to measure variation in the silhouette of different object types. Finally, we compared and evaluated the performance of dimensionless ratios with Fourier descriptors by classifying different object shapes and assessing the accuracy of each method. Multinomial regression models were fitted to compare the accuracy and error of the two methods. Classification accuracy assessment was addressed utilizing hypothesis tests and significance of the likelihood ratio (LR) test and Akaike’s Information Criterion (AIC). Dimensionless ratios and Fourier harmonics had similar, moderate accuracy of 70% and 73.33%, respectively, but only four harmonics were required to achieve the best model fit, and six were needed for dimensionless ratios. Harmonic analysis provides quantitative descriptions of object shapes allowing pattern characterizations that can improve a supervised classification.

MorphoCluster: Efficient Annotation of Plankton Images by Clustering.

In this work, we present MorphoCluster, a software tool for data-driven, fast, and accurate annotation of large image data sets. While already having surpassed the annotation rate of human experts, volume and complexity of marine data will continue to increase in the coming years. Still, this data requires interpretation. MorphoCluster augments the human ability to discover patterns and perform object classification in large amounts of data by embedding unsupervised clustering in an interactive process. By aggregating similar images into clusters, our novel approach to image annotation increases consistency, multiplies the throughput of an annotator, and allows experts to adapt the granularity of their sorting scheme to the structure in the data. By sorting a set of 1.2 M objects into 280 data-driven classes in 71 h (16 k objects per hour), with 90% of these classes having a precision of 0.889 or higher. This shows that MorphoCluster is at the same time fast, accurate, and consistent; provides a fine-grained and data-driven classification; and enables novelty detection.

Real-time 3D object proposal generation and classification using limited processing resources

The task of detecting 3D objects is important in various robotic applications. The existing deep learning-based detection techniques have achieved impressive performances. However, these techniques are limited to being run on a graphics processing unit (GPU) in a real-time environment. To achieve real-time 3D object detection with limited computational resources, we propose an efficient detection method based on 3D proposal generation and classification. The proposal generation is based mainly on point segmentation, while proposal classification is performed by a lightweight convolution neural network (CNN). KITTI datasets are then used to validate our method. It takes only 0.082 s for our method to process one point block with one core of the central processing unit (CPU). In addition to efficiency, the experimental results also demonstrate the capability of the proposed method of producing a competitive performance in object recall and classification.

A Review on Deep Learning Approaches for 3D Data Representations in Retrieval and Classifications

Deep learning approach has been used extensively in image analysis tasks. However, implementing the methods in 3D data is a bit complex because most of the previously designed deep learning architectures used 1D or 2D as input. In this work, the performance of deep learning methods on different 3D data representations has been reviewed. Based on the categorization of the different 3D data representations proposed in this paper, the importance of choosing a suitable 3D data representation which depends on simplicity, usability, and efficiency has been highlighted. Furthermore, the origin and contents of the major 3D datasets were discussed in detail. Due to growing interest in 3D object retrieval and classification tasks, the performance of different 3D object retrieval and classification on ModelNet40 dataset were compared. According to the findings in this work, multi views methods surpass voxel-based methods and with increased layers and enough data augmentation the performance can still be increased. Therefore, it can be concluded that deep learning together with a suitable 3D data representation gives an effective approach for improving the performance of 3D shape analysis. Finally, some possible directions for future researches were suggested.

Robust 3-Dimension Point Cloud Mapping in Dynamic Environment Using Point-Wise Static Probability-Based NDT Scan-Matching

Graph-based simultaneous localization and mapping (SLAM) is one of the methods to generate point cloud maps which are used for various applications in autonomous vehicles. Graph-based SLAM represents the pose of the vehicle as a node and the odometry between two different nodes as an edge. Among the edge generating methods, scan matching, light detection and ranging (LiDAR) based method, can provide an accurate pose between two nodes based on the high distance accuracy of the LiDAR. However, the point cloud in real driving situations contains numerous moving objects, which degrade the scan-matching performance. Therefore, this article defines the static probability which means the likelihood that an acquired point is from a static object, and proposes the weighted normal distribution transformation (NDT), which is achieved by modifying NDT. Weighted NDT is a scan-matching algorithm which can reflect the static probability of each point as a weight. The odometry from the weighted NDT is utilized for graph construction to generate a robust point cloud map even in a dynamic environment. Finally, the proposed algorithm was compared with the existing object removal algorithms in two areas: dynamic object classification and scan-matching performance. Based on the scan-matching results, the accuracy of the point cloud map generated by the proposed algorithm was evaluated with a reference map using high-performance global navigation satellite system (GNSS). It was confirmed that the proposed algorithm has higher classification accuracy and lower scan-matching error compared with other dynamic object removal methods. The proposed algorithm was able to generate a point cloud map, despite the presence of many dynamic objects, that was similar to a map generated in the absence of dynamic objects in the same environment.

Rotation-Invariant Feature Learning for Object Detection in VHR Optical Remote Sensing Images by Double-Net

Rotation-invariant feature extraction is crucial for object detection in Very High Resolution (VHR) optical remote sensing images. Although convolutional neural networks (CNNs) are good at extracting the translation-invariant features and have been widely applied in computer vision, it is still a challenging problem for CNNs to extract rotation-invariant features in VHR optical remote sensing images. In this paper we present a novel Double-Net with sample pairs from the same class as inputs to improve the performance of object detection and classification in VHR optical remote sensing images. Specifically, the proposed Double-Net contains multiple channels of CNNs in which each channel refers to a specific rotation direction and all CNNs share identical weights. Based on the output features of all channels, multiple instance learning algorithm is employed to extract the final rotation-invariant features. Experimental results on two publicly available benchmark datasets, namely Mnist-rot-12K and NWPU VHR-10, demonstrate that the presented Double-Net outperforms existing approaches on the performance of rotation-invariant feature extraction, which is especially effective under the situation of small training samples.

Using a Variable-Friction Robot Hand to Determine Proprioceptive Features for Object Classification During Within-Hand-Manipulation.

Interactions with an object during within-hand manipulation (WIHM) constitutes an assortment of gripping, sliding, and pivoting actions. In addition to manipulation benefits, the re-orientation and motion of the objects within-the-hand also provides a rich array of additional haptic information via the interactions to the sensory organs of the hand. In this article, we utilize variable friction (VF) robotic fingers to execute a rolling WIHM on a variety of objects, while recording 'proprioceptive' actuator data, which is then used for object classification (i.e., without tactile sensors). Rather than hand-picking a select group of features for this task, our approach begins with 66 general features, which are computed from actuator position and load profiles for each object-rolling manipulation, based on gradient changes. An Extra Trees classifier performs object classification while also ranking each feature's importance. Using only the six most-important 'Key Features' from the general set, a classification accuracy of 86% was achieved for distinguishing the six geometric objects included in our data set. Comparatively, when all 66 features are used, the accuracy is 89.8%.

Transfer learning of deep neural network representations for fMRI decoding

BackgroundDeep neural networks have revolutionised machine learning, with unparalleled performance in object classification. However, in brain imaging (e.g., fMRI), the direct application of Convolutional Neural Networks (CNN) to decoding subject states or perception from imaging data seems impractical given the scarcity of available data. New methodIn this work we propose a robust method to transfer information from deep learning (DL) features to brain fMRI data with the goal of decoding. By adopting Reduced Rank Regression with Ridge Regularisation we establish a multivariate link between imaging data and the fully connected layer (fc7) of a CNN. We exploit the reconstructed fc7 features by performing an object image classification task on two datasets: one of the largest fMRI databases, taken from different scanners from more than two hundred subjects watching different movie clips, and another with fMRI data taken while watching static images. ResultsThe fc7 features could be significantly reconstructed from the imaging data, and led to significant decoding performance. Comparison with existing methodsThe decoding based on reconstructed fc7 outperformed the decoding based on imaging data alone. ConclusionIn this work we show how to improve fMRI-based decoding benefiting from the mapping between functional data and CNN features. The potential advantage of the proposed method is twofold: the extraction of stimuli representations by means of an automatic procedure (unsupervised) and the embedding of high-dimensional neuroimaging data onto a space designed for visual object discrimination, leading to a more manageable space from dimensionality point of view.

Feed-forward visual processing suffices for coarse localization but fine-grained localization in an attention-demanding context needs feedback processing.

It is well known that simple visual tasks, such as object detection or categorization, can be performed within a short period of time, suggesting the sufficiency of feed-forward visual processing. However, more complex visual tasks, such as fine-grained localization may require high-resolution information available at the early processing levels in the visual hierarchy. To access this information using a top-down approach, feedback processing would need to traverse several stages in the visual hierarchy and each step in this traversal takes processing time. In the present study, we compared the processing time required to complete object categorization and localization by varying presentation duration and complexity of natural scene stimuli. We hypothesized that performance would be asymptotic at shorter presentation durations when feed-forward processing suffices for visual tasks, whereas performance would gradually improve as images are presented longer if the tasks rely on feedback processing. In Experiment 1, where simple images were presented, both object categorization and localization performance sharply improved until 100 ms of presentation then it leveled off. These results are a replication of previously reported rapid categorization effects but they do not support the role of feedback processing in localization tasks, indicating that feed-forward processing enables coarse localization in relatively simple visual scenes. In Experiment 2, the same tasks were performed but more attention-demanding and ecologically valid images were used as stimuli. Unlike in Experiment 1, both object categorization performance and localization precision gradually improved as stimulus presentation duration became longer. This finding suggests that complex visual tasks that require visual scrutiny call for top-down feedback processing.

Class-specific differential detection in diffractive optical neural networks improves inference accuracy

Diffractive deep neural networks have been introduced earlier as an optical machine learning framework that uses task-specific diffractive surfaces designed by deep learning to all-optically perform inference, achieving promising performance for object classification and imaging. Here we demonstrate systematic improvements in diffractive optical neural networks based on a differential measurement technique that mitigates the non-negativity constraint of light intensity. In this scheme, each class is assigned to a separate pair of photodetectors, behind a diffractive network, and the class inference is made by maximizing the normalized signal difference between the detector pairs. Moreover, by utilizing the inherent parallelization capability of optical systems, we reduced the signal coupling between the positive and negative detectors of each class by dividing their optical path into two jointly-trained diffractive neural networks that work in parallel. We further made use of this parallelization approach, and divided individual classes among multiple jointly-trained differential diffractive neural networks. Using this class-specific differential detection in jointly-optimized diffractive networks, our simulations achieved testing accuracies of 98.52%, 91.48% and 50.82% for MNIST, Fashion-MNIST and grayscale CIFAR-10 datasets, respectively. Similar to ensemble methods practiced in machine learning, we also independently-optimized multiple differential diffractive networks that optically project their light onto a common detector plane, and achieved testing accuracies of 98.59%, 91.06% and 51.44% for MNIST, Fashion-MNIST and grayscale CIFAR-10, respectively. Through these systematic advances in designing diffractive neural networks, the reported classification accuracies set the state-of-the-art for an all-optical neural network design.

An experimental study on breast lesion detection and classification from ultrasound images using deep learning architectures

BackgroundComputer-aided diagnosis (CAD) in the medical field has received more and more attention in recent years. One important CAD application is to detect and classify breast lesions in ultrasound images. Traditionally, the process of CAD for breast lesions classification is mainly composed of two separated steps: i) locate the lesion region of interests (ROI); ii) classify the located region of interests (ROI) to see if they are benign or not. However, due to the complex structure of breast and the existence of noise in the ultrasound images, traditional handcrafted feature based methods usually can not achieve satisfactory result.MethodsWith the recent advance of deep learning, the performance of object detection and classification has been boosted to a great extent. In this paper, we aim to systematically evaluate the performance of several existing state-of-the-art object detection and classification methods for breast lesions CAD. To achieve that, we have collected a new dataset consisting of 579 benign and 464 malignant lesion cases with the corresponding ultrasound images manually annotated by experienced clinicians. We evaluate different deep learning architectures and conduct comprehensive experiments on our newly collected dataset.ResultsFor the lesion regions detecting task, Single Shot MultiBox Detector with the input size as 300×300 (SSD300) achieves the best performance in terms of average precision rate (APR), average recall rate (ARR) and F1 score. For the classification task, DenseNet is more suitable for our problems.ConclusionsOur experiments reveal that better and more efficient detection and convolutional neural network (CNN) frameworks is one important factor for better performance of detecting and classification task of the breast lesion. Another significant factor for improving the performance of detecting and classification task, which is transfer learning from the large-scale annotated ImageNet to classify breast lesion.

Ultra-rapid object categorization in real-world scenes with top-down manipulations.

Humans are able to achieve visual object recognition rapidly and effortlessly. Object categorization is commonly believed to be achieved by interaction between bottom-up and top-down cognitive processing. In the ultra-rapid categorization scenario where the stimuli appear briefly and response time is limited, it is assumed that a first sweep of feedforward information is sufficient to discriminate whether or not an object is present in a scene. However, whether and how feedback/top-down processing is involved in such a brief duration remains an open question. To this end, here, we would like to examine how different top-down manipulations, such as category level, category type and real-world size, interact in ultra-rapid categorization. We have constructed a dataset comprising real-world scene images with a built-in measurement of target object display size. Based on this set of images, we have measured ultra-rapid object categorization performance by human subjects. Standard feedforward computational models representing scene features and a state-of-the-art object detection model were employed for auxiliary investigation. The results showed the influences from 1) animacy (animal, vehicle, food), 2) level of abstraction (people, sport), and 3) real-world size (four target size levels) on ultra-rapid categorization processes. This had an impact to support the involvement of top-down processing when rapidly categorizing certain objects, such as sport at a fine grained level. Our work on human vs. model comparisons also shed light on possible collaboration and integration of the two that may be of interest to both experimental and computational vision researches. All the collected images and behavioral data as well as code and models are publicly available at https://osf.io/mqwjz/.

An Adaptive End-to-End Classification Approach for Mobile Laser Scanning Point Clouds Based on Knowledge in Urban Scenes

It is fundamental for 3D city maps to efficiently classify objects of point clouds in urban scenes. However, it is still a large challenge to obtain massive training samples for point clouds and to sustain the huge training burden. To overcome it, a knowledge-based approach is proposed. The knowledge-based approach can explore discriminating features of objects based on people’s understanding of the surrounding environment, which exactly replaces the role of training samples. To implement the approach, a two-step segmentation procedure is carried out in this paper. In particular, Fourier Fitting is applied for second adaptive segmentation to separate points of multiple objects lying within a single group of the first segmentation. Then height difference and three geometrical eigen-features are extracted. In comparison to common classification methods, which need massive training samples, only basic knowledge of objects in urban scenes is needed to build an end-to-end match between objects and extracted features in the proposed approach. In addition, the proposed approach has high computational efficiency because of no heavy training process. Qualitative and quantificational experimental results show the proposed approach has promising performance for object classification in various urban scenes.

Deep convolutional networks do not classify based on global object shape.

Deep convolutional networks (DCNNs) are achieving previously unseen performance in object classification, raising questions about whether DCNNs operate similarly to human vision. In biological vision, shape is arguably the most important cue for recognition. We tested the role of shape information in DCNNs trained to recognize objects. In Experiment 1, we presented a trained DCNN with object silhouettes that preserved overall shape but were filled with surface texture taken from other objects. Shape cues appeared to play some role in the classification of artifacts, but little or none for animals. In Experiments 2–4, DCNNs showed no ability to classify glass figurines or outlines but correctly classified some silhouettes. Aspects of these results led us to hypothesize that DCNNs do not distinguish object’s bounding contours from other edges, and that DCNNs access some local shape features, but not global shape. In Experiment 5, we tested this hypothesis with displays that preserved local features but disrupted global shape, and vice versa. With disrupted global shape, which reduced human accuracy to 28%, DCNNs gave the same classification labels as with ordinary shapes. Conversely, local contour changes eliminated accurate DCNN classification but caused no difficulty for human observers. These results provide evidence that DCNNs have access to some local shape information in the form of local edge relations, but they have no access to global object shapes.

Point source classification on astronomical photometric images using artificial neural network

Gravitational-wave Optical Transient Observer (GOTO) is a set of telescopes primarily for optical follow-up observation of gravitational wave. It is designed to have as much sky coverage as 18 square-degree per pointing. This property allows the telescopes to compensate for an area of uncertainty produced by gravitational wave detector such as LIGO. The secondary objective of GOTO is to perform an all-sky survey. Each night, it could cover as much as 2700 square-degree. Such large area of observation produces as large number of objects. It was estimated to be able to collect 12 million objects per night. Processing such quantity of data manually is extremely difficult. One of the most challenging task is to perform object classification. A solution to this problem is to have machine perform classification, which is possible via machine learning algorithm. In this work, we use supervise machine learning performed on simulated data to create a point source classifier. We will focus on application and performance of artificial neural network in the work.

NormalNet: A voxel-based CNN for 3D object classification and retrieval

Abstract A common approach to tackle 3D object recognition tasks is to project 3D data to multiple 2D images. Projection only captures the outline of the object, and discards the internal information that may be crucial for the recognition. In this paper, we stay in 3D and concentrate on tapping the potential of 3D representations. We present NormalNet, a voxel-based convolutional neural network (CNN) designed for 3D object recognition. The network uses normal vectors of the object surfaces as input, which demonstrate stronger discrimination capability than binary voxels. We propose a reflection–convolution–concatenation (RCC) module to realize the conv layers, which extracts distinguishable features for 3D vision tasks while reducing the number of parameters significantly. We further improve the performance of NormalNet by combining two networks, which take normal vectors and voxels as input respectively. We carry out a series of experiments that validate the design of the network and achieve competitive performance in 3D object classification and retrieval tasks.

Deep Learning: Individual Maize Segmentation From Terrestrial Lidar Data Using Faster R-CNN and Regional Growth Algorithms.

The rapid development of light detection and ranging (Lidar) provides a promising way to obtain three-dimensional (3D) phenotype traits with its high ability of recording accurate 3D laser points. Recently, Lidar has been widely used to obtain phenotype data in the greenhouse and field with along other sensors. Individual maize segmentation is the prerequisite for high throughput phenotype data extraction at individual crop or leaf level, which is still a huge challenge. Deep learning, a state-of-the-art machine learning method, has shown high performance in object detection, classification, and segmentation. In this study, we proposed a method to combine deep leaning and regional growth algorithms to segment individual maize from terrestrial Lidar data. The scanned 3D points of the training site were sliced row and row with a fixed 3D window. Points within the window were compressed into deep images, which were used to train the Faster R-CNN (region-based convolutional neural network) model to learn the ability of detecting maize stem. Three sites of different planting densities were used to test the method. Each site was also sliced into many 3D windows, and the testing deep images were generated. The detected stem in the testing images can be mapped into 3D points, which were used as seed points for the regional growth algorithm to grow individual maize from bottom to up. The results showed that the method combing deep leaning and regional growth algorithms was promising in individual maize segmentation, and the values of r, p, and F of the three testing sites with different planting density were all over 0.9. Moreover, the height of the truly segmented maize was highly correlated to the manually measured height (R2> 0.9). This work shows the possibility of using deep leaning to solve the individual maize segmentation problem from Lidar data.