Object Classification Tasks Research Articles

Adaptive Pyramid Context Fusion for Point Cloud Perception

Deep learning for 3-D point cloud perception has been a very active research topic in recent years. A current trend is toward the combination of the semantically strong and the fine-grained information from different scales of intermediate representations to boost network generalization power and robustness against scale variation. One prominent challenge is how to effectively conduct the allocation of multiple scales of information. In this letter, we propose a module, named adaptive pyramid context fusion (APCF), to adaptively capture scales of contextual information from a multiscale feature pyramid for the point cloud. The APCF module reweights and aggregates the features from different levels in the feature pyramid via a softmax attention strategy. The allocation of information is adaptively conducted level by level from bottom to up first and then from top to bottom. To ensure both effectiveness and efficiency, we propose a multiscale context-aware network APCF-Net through applying our proposed APCF to the PointConv architecture. Experiments demonstrate that APCF-Net surpasses its vanilla counterpart by a large margin both in effectiveness and efficiency. Especially, APCF-Net outperforms state-of-the-art approaches on 3-D object classification and semantic segmentation task, with the overall accuracy of 93.3% on ModelNet40 and mIoU of 63.1% on ScanNet V2 online test.

Fast Classification and Action Recognition With Event-Based Imaging

The neuromorphic event cameras, which capture the optical changes of a scene, have drawn increasing attention due to their high speed and low power consumption. However, the event data are noisy, sparse, and nonuniform in the spatial-temporal domain with extremely high temporal resolution, making it challenging to process for traditional deep learning algorithms. To enable convolutional neural network models for event vision tasks, most methods encode events into point-cloud or voxel representations, but their performance still has much room for improvement. Additionally, as event cameras can only detect changes in the scene, relative movements can lead to misalignment, i.e., the same pixel may refer to different real-world points at different times. To this end, this work proposes the aligned compressed event tensor (ACE) as a novel event data representation, and a framework called branched event net (BET) for event-based vision under both static and dynamic scenes. We apply them on various datasets for object classification and action recognition tasks, and show that they surpass state-of-the-art methods by significant margins. Specifically, our method achieves 98.88% accuracy for the DVS128 action recognition task, and outperforms the second best method by large margins of 4.85%, 9.56% and 2.33% on N-Caltech101, DVSAction and NeuroIV datasets, respectively. Furthermore, the proposed ACE-BET is efficient, and achieves the fastest inference speed among various methods being tested.

Deep Transfer Learning for Wall Bulge Endpoints Regression for Autonomous Decoration Robots

Wall bulge maintenance and repairing is an essential task for autonomous decoration robots. The problem of the wall bulge endpoints regression refers to identifying the position of the wall bulge endpoints in spatial coordinates. This problem is of significant importance for autonomous decoration robots as these robots target automatic maintenance and repairing of wall bulges, they must automatically recognize where to start and stop the repairing process. Training deep convolutional neural networks for supervised computer vision tasks requires a large number of annotated images. Since gathering annotated images for this task is difficult, laborious, and time-consuming, we proposed a model for detecting the wall bulge endpoints position based on deep transfer learning. Our proposed model is capable of classifying the wall bulge into one of four classes according to its orientation. Deep transfer learning transfers the knowledge acquired by deep learning models trained for a specific task and domain to another different but related task and domain. Our proposed model is mainly based on deep convolutional neural networks pre-trained on large datasets for tasks of object classification and detection. We transfer the knowledge acquired by the model from these tasks to solve both problems in our new task.

An Efficient BoF Representation for Object Classification

The Bag-of-features (BoF) approach has proved to yield better performance in a patch-based object classification system owing to its simplicity. However, often the very large number of patch-based descriptors (such as scale-invariant feature transform and speeded up robust features, extracted from images to create a BoF vector) leads to huge computational cost and an increased storage requirement. This paper demonstrates a two-staged approach to creating a discriminative and compact BoF representation for object classification. As a preprocessing stage to the codebook construction, ambiguous patch-based descriptors are eliminated using an entropy-based and one-pass feature selection approach, to retain high-quality descriptors. As a post-processing stage to the codebook construction, a subset of codewords which is not activated enough in images are eliminated from the initially constructed codebook based on statistical measures. Finally, each patch-based descriptor of an image is assigned to the closest codeword to create a histogram representation. One-versus-all support vector machine is applied to classify the histogram representation. The proposed methods are evaluated on benchmark image datasets. Testing results show that the proposed methods enables the codebook to be more discriminative and compact in moderate sized visual object classification tasks.

Model of the Weak Reset Process in HfOx Resistive Memory for Deep Learning Frameworks

The implementation of current deep learning training algorithms is power-hungry, owing to data transfer between memory and logic units. Oxide-based RRAMs are outstanding candidates to implement in-memory computing, which is less power-intensive. Their weak RESET regime, is particularly attractive for learning, as it allows tuning the resistance of the devices with remarkable endurance. However, the resistive change behavior in this regime suffers many fluctuations and is particularly challenging to model, especially in a way compatible with tools used for simulating deep learning. In this work, we present a model of the weak RESET process in hafnium oxide RRAM and integrate this model within the PyTorch deep learning framework. Validated on experiments on a hybrid CMOS/RRAM technology, our model reproduces both the noisy progressive behavior and the device-to-device (D2D) variability. We use this tool to train Binarized Neural Networks for the MNIST handwritten digit recognition task and the CIFAR-10 object classification task. We simulate our model with and without various aspects of device imperfections to understand their impact on the training process and identify that the D2D variability is the most detrimental aspect. The framework can be used in the same manner for other types of memories to identify the device imperfections that cause the most degradation, which can, in turn, be used to optimize the devices to reduce the impact of these imperfections.

A Comprehensive Review of Image Segmentation Techniques

Image segmentation is a wide research topic; a huge amount of research has been performed in this context. Image segmentation is a crucial procedure for most object detection, image recognition, feature extraction, and classification tasks depend on the quality of the segmentation process. Image segmentation is the dividing of a specific image into a numeral of homogeneous segments; therefore, the representation of an image into simple and easy forms increases the effectiveness of pattern recognition. The effectiveness of approaches varies according to the conditions of objects arrangement, lighting, shadow, and other factors. However, there is no generic approach for successfully segmenting all images, where some approaches have been proven to be more effective than others. The major goal of this study is to provide summarize of the disadvantages and the advantages of each of the reviewed approaches of image segmentation.

Multilayer residual sparsifying transform (MARS) model for low-dose CT image reconstruction.

Signal models based on sparse representations have received considerable attention in recent years. On the other hand, deep models consisting of a cascade of functional layers, commonly known as deep neural networks, have been highly successful for the task of object classification and have been recently introduced to image reconstruction. In this work, we develop a new image reconstruction approach based on a novel multilayer model learned in an unsupervised manner by combining both sparse representations and deep models. The proposed framework extends the classical sparsifying transform model for images to a Multilayer residual sparsifying transform (MARS) model, wherein the transform domain data are jointly sparsified over layers. We investigate the application of MARS models learned from limited regular-dose images for low-dose CT reconstruction using penalized weighted least squares (PWLS) optimization. We propose new formulations for multilayer transform learning and image reconstruction. We derive an efficient block coordinate descent algorithm to learn the transforms across layers, in an unsupervised manner from limited regular-dose images. The learned model is then incorporated into the low-dose image reconstruction phase. Low-dose CT experimental results with both the XCAT phantom and Mayo Clinic data show that the MARS model outperforms conventional methods such as filtered back-projection and PWLS methods based on the edge-preserving (EP) regularizer in terms of two numerical metrics (RMSE and SSIM) and noise suppression. Compared with the single-layer learned transform (ST) model, the MARS model performs better in maintaining some subtle details. This work presents a novel data-driven regularization framework for CT image reconstruction that exploits learned multilayer or cascaded residual sparsifying transforms. The image model is learned in an unsupervised manner from limited images. Our experimental results demonstrate the promising performance of the proposed multilayer scheme over single-layer learned sparsifying transforms. Learned MARS models also offer better image quality than typical nonadaptive PWLS methods.

Cross-Racial Automatic Age Estimation from Facial Images using Deep Learning

This paper presents a deep learning approach for age estimation of human beings using their facial images. The different racial groups based on skin colour have been incorporated in the annotations of the images in the dataset, while ensuring an adequate distribution of subjects across the racial groups so as to achieve an accurate Automatic Facial Age Estimation (AFAE). The principle of transfer learning is applied to the ResNet50 Convolutional Neural Network (CNN) initially pretrained for the task of object classification and finetuning it’s hyperparameters to propose an AFAE system that can be used to automate ages of humans across multiple racial groups. The mean absolute error of 4.25 years is obtained at the end of the research which proved the effectiveness and superiority of the proposed method.

Segmentation and Selective Feature Extraction for Human Detection to the Direction of Action Recognition

Detection as well as classification of different object for machine vision application is a challenging task. Similar to the other object detection and classification task, human detection concept provides a major role for the ad- vancement in the design of an automatic visual surveillance system (AVSS). For the future automation system if it is possible to include human detection and tracking, human action recognition, usual as well as unusual event recognition etc. concept for future AVSS, it will be a greater success in the transformable world. In this paper we have proposed a proper human detection and tracking technique for human action recognition toward the design of AVSS. Here we use median filter for noise removal, graph cut for segment the human images, mathematical morphology to refine the segmentation mask, extract selective feature points by sing HOG, classify human objects by using SVM with polynomial ker- nel and finally particle filter for tracking those of detected human. Due to the above mentioned combinations our system can independent to the variations of lightening conditions, color, shape, size, clothing etc. and can handle the occlusion. Our system can easily detect and track human in different indoor as well as outdoor environ- ment with a automatic multiple human detection rate of 97:61% and total multiple human detection and tracking accuracy is about 92% for AVSS. Due to the use of HOG to extract features af- ter graph cut segmentation operation, our system requires less memory for store the trained data therefore processing speed as well as accuracy of detection and tracking will be better than other techniques which can be suitable for action classification task.

DGANet: A Dilated Graph Attention-Based Network for Local Feature Extraction on 3D Point Clouds

Feature extraction on point clouds is an essential task when analyzing and processing point clouds of 3D scenes. However, there still remains a challenge to adequately exploit local fine-grained features on point cloud data due to its irregular and unordered structure in a 3D space. To alleviate this problem, a Dilated Graph Attention-based Network (DGANet) with a certain feature for learning ability is proposed. Specifically, we first build a local dilated graph-like region for each input point to establish the long-range spatial correlation towards its corresponding neighbors, which allows the proposed network to access a wider range of geometric information of local points with their long-range dependencies. Moreover, by integrating the dilated graph attention module (DGAM) implemented by a novel offset–attention mechanism, the proposed network promises to highlight the differing importance on each edge of the constructed local graph to uniquely learn the discrepancy feature of geometric attributes between the connected point pairs. Finally, all the learned edge attention features are further aggregated, allowing the most significant geometric feature representation of local regions by the graph–attention pooling to fully extract local detailed features for each point. The validation experiments using two challenging benchmark datasets demonstrate the effectiveness and powerful generation ability of our proposed DGANet in both 3D object classification and segmentation tasks.

Voxel-based three-view hybrid parallel network for 3D object classification

Three-dimensional models are widely used in the fields of multimedia, computer graphics, virtual reality, entertainment, design, and manufacturing because of the rich information that preserves the surface, color and texture of real objects. Therefore, effective 3D object classification technology has become an urgent need. Previous methods usually directly convert classic 2D convolution into 3D form and apply it to objects with binary voxel representation, which may lose internal information that is essential for recognition. In this paper, we propose a novel voxel-based three-view hybrid parallel network for 3D shape classification. This method first obtains the depth projection views of the three-dimensional model from the front view, the top view and the side view, so as to preserve the spatial information of the three-dimensional model to the greatest extent, and output its predicted probability value for the category of the three-dimensional model, and then combining the three-view parallel network with voxel sub-network performs weight fusion, and then uses Softmax for classification. We conducted a series of experiments to verify the design of the network and achieved competitive performance in the 3D object classification tasks of ModelNet10 and ModelNet40.

Hypergraph wavelet neural networks for 3D object classification

Recently, hypergraph learning has shown great potential in a variety of classification tasks. However, existing hypergraph neural networks lack flexibility in modeling and extracting high-order relationships among data. To solve this problem, we propose a novel framework called hypergraph wavelet neural networks (HGWNN) to explore the high-order correlation in 3D data. Firstly, considering the non-uniformity of most data sets in the real world, we propose a “data-driven” hypergraph construction scheme, which is more efficient than some commonly used hypergraph construction methods. Secondly, in order to efficiently learn deep embeddings from the constructed hypergraph, we propose a hypergraph wavelet convolution operator. It enables efficient information aggregation by fully exploiting the localization property of wavelets. This convolution operator is suitable for both non-uniform and uniform hypergraphs. Finally, we design a new hypergraph regularizer based on the sparse prior of wavelet coefficients to promote local smoothness and avoid network overfitting. We have conducted experiments on object classification tasks on two 3D benchmark datasets: the National Taiwan University (NTU) 3D model dataset and the ModelNet40 dataset. Experimental results demonstrate the effectiveness of the proposed method compared with the state-of-the-art methods.

An Inverse Node Graph-Based Method for the Urban Scene Segmentation of 3D Point Clouds

Urban object segmentation and classification tasks are critical data processing steps in scene understanding, intelligent vehicles and 3D high-precision maps. Semantic segmentation of 3D point clouds is the foundational step in object recognition. To identify the intersecting objects and improve the accuracy of classification, this paper proposes a segment-based classification method for 3D point clouds. This method firstly divides points into multi-scale supervoxels and groups them by proposed inverse node graph (IN-Graph) construction, which does not need to define prior information about the node, it divides supervoxels by judging the connection state of edges between them. This method reaches minimum global energy by graph cutting, obtains the structural segments as completely as possible, and retains boundaries at the same time. Then, the random forest classifier is utilized for supervised classification. To deal with the mislabeling of scattered fragments, higher-order CRF with small-label cluster optimization is proposed to refine the classification results. Experiments were carried out on mobile laser scan (MLS) point dataset and terrestrial laser scan (TLS) points dataset, and the results show that overall accuracies of 97.57% and 96.39% were obtained in the two datasets. The boundaries of objects were retained well, and the method achieved a good result in the classification of cars and motorcycles. More experimental analyses have verified the advantages of the proposed method and proved the practicability and versatility of the method.

Exploring influential factors of shape classifier comprehension and production in Mandarin-speaking children

Mandarin classifiers are a complex system, but little is known about how Mandarin-speaking children manage to learn the system. Based on the extant literature, we explored potential factors influencing the comprehension and production of Mandarin shape classifiers, including classifier-based semantic categorization and errors pertaining to the semantic strategies, input frequency of classifier-noun combinations, and vocabulary knowledge. In total, 138 typically developing monolingual Mandarin-speaking children between ages 4;1 and 6;5 completed an object categorization task, shape classifier comprehension and production tasks, and a vocabulary test. The results showed that classifier-based categorization did not significantly relate to classifier knowledge, but children’s comprehension errors were mostly selecting an object that is perceptually similar to the target object. Estimated input frequency of classifier-noun combinations was significantly related to classifier comprehension, and there was differential accuracy for different classifier-noun combinations, which may indicate item-by-item learning of individual classifier-noun pairings. Mandarin-speaking children may take a combined approach by sorting semantic features for different classifiers and learning individual classifier-noun combinations. The interplay of the two approaches can be very complex and should be further investigated in future studies. Vocabulary knowledge was significantly related to classifier comprehension and production, indicating common traits between classifier learning and noun learning.

Radar Transformer: An Object Classification Network Based on 4D MMW Imaging Radar.

Automotive millimeter-wave (MMW) radar is essential in autonomous vehicles due to its robustness in all weather conditions. Traditional commercial automotive radars are limited by their resolution, which makes the object classification task difficult. Thus, the concept of a new generation of four-dimensional (4D) imaging radar was proposed. It has high azimuth and elevation resolution and contains Doppler information to produce a high-quality point cloud. In this paper, we propose an object classification network named Radar Transformer. The algorithm takes the attention mechanism as the core and adopts the combination of vector attention and scalar attention to make full use of the spatial information, Doppler information, and reflection intensity information of the radar point cloud to realize the deep fusion of local attention features and global attention features. We generated an imaging radar classification dataset and completed manual annotation. The experimental results show that our proposed method achieved an overall classification accuracy of 94.9%, which is more suitable for processing radar point clouds than the popular deep learning frameworks and shows promising performance.

Joint Learning of Multiple Latent Domains and Deep Representations for Domain Adaptation.

In domain adaptation, the automatic discovery of multiple latent source domains has succeeded by capturing the intrinsic structure underlying the source data. Different from previous works that mainly rely on shallow models for domain discovery, we propose a novel unified framework based on deep neural networks to jointly address latent domain prediction from source data and deep representation learning from both source and target data. Within this framework, an iterative algorithm is proposed to alternate between 1) utilizing a new probabilistic hierarchical clustering method to separate the source domain into latent clusters and 2) training deep neural networks by using the domain membership as the supervision to learn deep representations. The key idea behind this joint learning framework is that good representations can help to improve the prediction accuracy of latent domains and, in turn, domain prediction results can provide useful supervisory information for feature learning. During the training of the deep model, a domain prediction loss, a domain confusion loss, and a task-specific classification loss are effectively integrated to enable the learned feature to distinguish between different latent source domains, transfer between source and target domains, and become semantically meaningful among different classes. Trained in an end-to-end fashion, our framework outperforms the state-of-the-art methods for latent domain discovery, as validated by extensive experiments on both object classification and human action-recognition tasks.

CF2PN: A Cross-Scale Feature Fusion Pyramid Network Based Remote Sensing Target Detection

In the wake of developments in remote sensing, the application of target detection of remote sensing is of increasing interest. Unfortunately, unlike natural image processing, remote sensing image processing involves dealing with large variations in object size, which poses a great challenge to researchers. Although traditional multi-scale detection networks have been successful in solving problems with such large variations, they still have certain limitations: (1) The traditional multi-scale detection methods note the scale of features but ignore the correlation between feature levels. Each feature map is represented by a single layer of the backbone network, and the extracted features are not comprehensive enough. For example, the SSD network uses the features extracted from the backbone network at different scales directly for detection, resulting in the loss of a large amount of contextual information. (2) These methods combine with inherent backbone classification networks to perform detection tasks. RetinaNet is just a combination of the ResNet-101 classification network and FPN network to perform the detection tasks; however, there are differences in object classification and detection tasks. To address these issues, a cross-scale feature fusion pyramid network (CF2PN) is proposed. First and foremost, a cross-scale fusion module (CSFM) is introduced to extract sufficiently comprehensive semantic information from features for performing multi-scale fusion. Moreover, a feature pyramid for target detection utilizing thinning U-shaped modules (TUMs) performs the multi-level fusion of the features. Eventually, a focal loss in the prediction section is used to control the large number of negative samples generated during the feature fusion process. The new architecture of the network proposed in this paper is verified by DIOR and RSOD dataset. The experimental results show that the performance of this method is improved by 2–12% in the DIOR dataset and RSOD dataset compared with the current SOTA target detection methods.

Comparison of neuronal responses in primate inferior-temporal cortex and feed-forward deep neural network model with regard to information processing of faces.

Feed-forward deep neural networks have better performance in object categorization tasks than other models of computer vision. To understand the relationship between feed-forward deep networks and the primate brain, we investigated representations of upright and inverted faces in a convolutional deep neural network model and compared them with representations by neurons in the monkey anterior inferior-temporal cortex, area TE. We applied principal component analysis to feature vectors in each model layer to visualize the relationship between the vectors of the upright and inverted faces. The vectors of the upright and inverted monkey faces were more separated through the convolution layers. In the fully-connected layers, the separation among human individuals for upright faces was larger than for inverted faces. The Spearman correlation between each model layer and TE neurons reached a maximum at the fully-connected layers. These results indicate that the processing of faces in the fully-connected layers might resemble the asymmetric representation of upright and inverted faces by the TE neurons. The separation of upright and inverted faces might take place by feed-forward processing in the visual cortex, and separations among human individuals for upright faces, which were larger than those for inverted faces, might occur in area TE.

Integration of complex wavelet transform and Zernike moment for multi‐class classification

Multiclass object classification is a crucial problem in computer vision research and have different emerging applications such as video surveillance. The task of multiclass object classification has more challenges because of highly variable nature and real time processing requirement of data. For tackling the multiclass object classification task, several existing methods adopt one feature or combination of features to classify objects. In this work, we propose a new combination of features-based algorithm for object classification. In the combination, the two features: (1) Daubechies complex wavelet transform (DCxWT) and (2) Zernike moments (ZM) have been used. The shift-invariance and symmetry properties of DCxWT facilitate the object classification in the wavelet domain. Specifically, the shift-invariance property of DCxWT is effective for translated object representation whereas the symmetry property yields perfect reconstruction for retaining object boundaries (i.e., edges). Moreover, translation and rotation-invariance properties of ZM are especially beneficial for the representation of varying pose and orientation of the objects. For these reasons, the composite of the two features brings about significant synthesized benefits over each single feature and the other widely used features. The multi-class support vector machine classifier is used for classifying different objects. The proposed method has been tested on standard datasets as well as our own dataset prepared by authors of this paper. Experimental results demonstrated the significant outperformance of the proposed method through quantitative evaluations and also suggest that the proposed hybridization of features is preferable for the classification problem.

L3DOC: Lifelong 3D Object Classification.

3D object classification has been widely applied in both academic and industrial scenarios. However, most state-of-the-art algorithms rely on a fixed object classification task set, which cannot tackle the scenario when a new 3D object classification task is coming. Meanwhile, the existing lifelong learning models can easily destroy the learned tasks performance, due to the unordered, large-scale, and irregular 3D geometry data. To address these challenges, we propose a Lifelong 3D Object Classification (i.e., L3DOC) model, which can consecutively learn new 3D object classification tasks via imitating "human learning". More specifically, the core idea of our model is to capture and store the cross-task common knowledge of 3D geometry data in a 3D neural network, named as point-knowledge, through employing layer-wise point-knowledge factorization architecture. Afterwards, a task-relevant knowledge distillation mechanism is employed to connect the current task to previous relevant tasks and effectively prevent catastrophic forgetting. It consists of a point-knowledge distillation module and a transforming-space distillation module, which transfers the accumulated point-knowledge from previous tasks and soft-transfers the compact factorized representations of the transforming-space, respectively. To our best knowledge, the proposed L3DOC algorithm is the first attempt to perform deep learning on 3D object classification tasks in a lifelong learning way. Extensive experiments on several point cloud benchmarks illustrate the superiority of our L3DOC model over the state-of-the-art lifelong learning methods.