Object Classification Performance Research Articles

Multiple-output network for simultaneous target classification and moving direction estimation in automotive radar systems

To ensure the stable operation of autonomous driving, a rich understanding of road conditions is indispensable. Therefore, in this paper, we propose a method that uses a single deep learning (DL) network structure to classify commonly encountered on-road objects, such as vehicles, cyclists, and pedestrians, and simultaneously estimating their moving direction. First, we obtain information about the target, such as range, velocity, azimuth angle, and elevation angle, using a four-dimensional imaging radar. Next, we convert the detection results into point cloud data and represent them within a three-dimensional spatial coordinate system. Then, the point cloud data is projected onto the XY-plane to classify the target’s class and estimate the moving direction of the target. In the XY-plane, we apply a preprocessing step using density-based spatial clustering to remove noise from the detection results, cluster the targets, and convert this processed data into image data. Subsequently, we conduct training on a multiple-output DL network designed to simultaneously perform object classification and predict the moving direction using the image data. Finally, the performance evaluation of the proposed method results in an object classification accuracy of 96.10%, and the root mean square error for estimated moving directions is 5.54∘, 3.89∘, and 15.35∘ for vehicles, cyclists, and pedestrians, respectively, with a runtime of 0.1 s, demonstrating its efficiency.

Deep learning for 3D object recognition: A survey

With the growing availability of extensive 3D datasets and the rapid progress in computational power, deep learning (DL) has emerged as a highly promising approach for learning from 3D data, addressing critical tasks like object detection, segmentation, and recognition. Despite the unique challenges in processing geometry data with deep neural networks, recent advancements in DL for 3D object recognition have shown remarkable success, with various methods proposed to tackle different issues. This paper aims to stimulate future research by providing a comprehensive review of recent progress in DL techniques for 3D object recognition, which are systematically categorized based on their learning behavior. We discuss the advantages, limitations, and application of each approach, highlighting their performance in 3D object classification on benchmark datasets such as ModelNet, ScanObjectNN, and Sydney Urban Object. The survey offers insightful observations and inspires future research directions.

Soft Tactile Sensing for Object Classification and Fine Grasping Adjustment Using a Pneumatic Hand With an Inflatable Palm

Power grasps and fine manipulations with a robotic hand usually require dexterity in the fingers and a rich tactile function. In this paper, we propose a novel soft hand with a tactile sensing function and investigate how tactile information can be used to perform object classification and fine grasping adjustments. We first developed a pneumatically driven soft hand with an inflatable palm and bendable fingers, where a TacTip -type sensor and several flex sensors were integrated. The inflatable palm can not only significantly improve grasping but also object sensing. Based on the perspective projection model of a camera, a 3D reconstruction algorithm for the deformed surface of a soft palm is presented. A spatial characterization of geometric information regarding objects based on sensing information from the palm and fingers was deduced and analyzed. A classifier was designed for object classification. Finally, a control method for the fine adjustment of grasps based on tactile sensing information was presented and verified using a series of grasps of different objects with a soft hand. The object classification and contact state sensed by the soft tactile sensors can be used for grasp planning and manipulation control with the soft hand.

Classification of architectural and MEP BIM objects for building performance evaluation

Numerous factors in current workflows reduce the completeness, accuracy, and reliability of the information exchanged between Building Information Modelling (BIM) authoring tools and building performance simulation and analysis software. Automated classification of BIM objects could ameliorate most of the interoperability problems. However, architectural, structural, mechanical, electrical, and plumbing (MEP), and other building objects have a wide range of shapes, features, and properties, which makes identifying them reliably and accurately with any single AI tool difficult. This study explored methods and techniques for automated classification of architectural and MEP BIM objects that can significantly reduce or eliminate the manual effort required to correct data exchange and interoperability issues in pre-processing building models for performance evaluation. A series of ensemble models using a stacking mechanism of four classification tools (semantic keyword search, rule-based inferencing, machine learning using object geometry features, and deep learning using visual shape features) were implemented and tested. A BIM object dataset with 3,410 instances belonging to 40 classes was compiled for training and testing the models. The dataset contains diverse architecture and MEP objects with high-quality selected instances and is available for public use. We found that an ensemble learning approach that exploits the various types of information available in instance models may exhibit superior object classification performance than any individual tool. The ensemble model applying all four tools achieved 91.0 % accuracy (F1 score 0.88), suggesting that automated classification using ensemble learning is an effective strategy.

Advanced deep learning framework for underwater object detection with multibeam forward-looking sonar

Underwater object detection (UOD) is an essential activity in maintaining and monitoring underwater infrastructure, playing an important role in their efficient and low-risk asset management. In underwater environments, sonar, recognized for overcoming the limitations of optical imaging in low-light and turbid conditions, has increasingly gained popularity for UOD. However, due to the low resolution and limited foreground-background contrast in sonar images, existing sonar-based object detection algorithms still face challenges regarding precision and transferability. To solve these challenges, this article proposes an advanced deep learning framework for UOD that uses the data from multibeam forward-looking sonar. The framework is adapted from the network architecture of YOLOv7, one of the state-of-the-art vision-based object detection algorithms, by incorporating unique optimizations in three key aspects: data preprocessing, feature fusion, and loss functions. These improvements are extensively tested on a dedicated public dataset, showing superior object classification performance compared to the selected existing sonar-based methods. Through experiments conducted on an underwater remotely operated vehicle, the proposed framework validates significant enhancements in target classification, localization, and transfer learning capabilities. Since the engineering structures have similar geometric shapes to the objects tested in this study, the proposed framework presents potential applicability to underwater structural inspection and monitoring, and autonomous asset management.

Architecture of the brain's visual system enhances network stability and performance through layers, delays, and feedback.

In the visual system of primates, image information propagates across successive cortical areas, and there is also local feedback within an area and long-range feedback across areas. Recent findings suggest that the resulting temporal dynamics of neural activity are crucial in several vision tasks. In contrast, artificial neural network models of vision are typically feedforward and do not capitalize on the benefits of temporal dynamics, partly due to concerns about stability and computational costs. In this study, we focus on recurrent networks with feedback connections for visual tasks with static input corresponding to a single fixation. We demonstrate mathematically that a network's dynamics can be stabilized by four key features of biological networks: layer-ordered structure, temporal delays between layers, longer distance feedback across layers, and nonlinear neuronal responses. Conversely, when feedback has a fixed distance, one can omit delays in feedforward connections to achieve more efficient artificial implementations. We also evaluated the effect of feedback connections on object detection and classification performance using standard benchmarks, specifically the COCO and CIFAR10 datasets. Our findings indicate that feedback connections improved the detection of small objects, and classification performance became more robust to noise. We found that performance increased with the temporal dynamics, not unlike what is observed in core vision of primates. These results suggest that delays and layered organization are crucial features for stability and performance in both biological and artificial recurrent neural networks.

A two-stage semi-supervised object detection method for SAR images with missing labels based on meta pseudo-labels

Recently, deep learning has been applied to analyze satellite images. However, applying fully supervised object detection (FSOD) is impractical because it is challenging to detect and annotate objects that are relatively small in high-resolution satellite images. In addition, most data owned by public institutions, such as military reconnaissance videos, do not contain sufficient object class label information. Therefore, the application of semi-supervised objection detection (SSOD) is more practical. The SSOD performance is often determined by generated pseudo-labels. Therefore, this study proposes a 2-stage SSOD model to generate accurate pseudo-labels. In the first stage, annotations, including the location information of unlabeled data, are detected using a modified faster R-CNN model. In the second stage, pseudo-labels of objects are additionally suggested through the meta pseudo-label method. Finally, reliable pseudo-labels are generated by comparing and combining the pseudo-labels generated in each stage, improving the SSOD model performance. Extensive experiments were conducted using the xView3 dataset provided by the U.S. Defense Innovation Unit (DIU). The proposed method performed approximately 11% better than the FSOD model, which does not learn pseudo-labels for datasets with missing labels according to the evaluation metrics proposed by the DIU covering both object detection and classification performance.

A Deep Learning-Based Hyperspectral Object Classification Approach via Imbalanced Training Samples Handling

Object classification in hyperspectral images involves accurately categorizing objects based on their spectral characteristics. However, the high dimensionality of hyperspectral data and class imbalance pose significant challenges to object classification performance. To address these challenges, we propose a framework that incorporates dimensionality reduction and re-sampling as preprocessing steps for a deep learning model. Our framework employs a novel subgroup-based dimensionality reduction technique to extract and select the most informative features with minimal redundancy. Additionally, the data are resampled to achieve class balance across all categories. The reduced and balanced data are then processed through a hybrid CNN model, which combines a 3D learning block and a 2D learning block to extract spectral–spatial features and achieve satisfactory classification accuracy. By adopting this hybrid approach, we simplify the model while improving performance in the presence of noise and limited sample size. We evaluated our proposed model on the Salinas scene, Pavia University, and Kennedy Space Center benchmark hyperspectral datasets, comparing it to state-of-the-art methods. Our object classification technique achieves highly promising results, with overall accuracies of 99.98%, 99.94%, and 99.46% on the three datasets, respectively. This proposed approach offers a compelling solution to overcome the challenges of high dimensionality and class imbalance in hyperspectral object classification.

Implementation of Field-Programmable Gate Array Platform for Object Classification Tasks Using Spike-Based Backpropagated Deep Convolutional Spiking Neural Networks.

This paper investigates the performance of deep convolutional spiking neural networks (DCSNNs) trained using spike-based backpropagation techniques. Specifically, the study examined temporal spike sequence learning via backpropagation (TSSL-BP) and surrogate gradient descent via backpropagation (SGD-BP) as effective techniques for training DCSNNs on the field programmable gate array (FPGA) platform for object classification tasks. The primary objective of this experimental study was twofold: (i) to determine the most effective backpropagation technique, TSSL-BP or SGD-BP, for deeper spiking neural networks (SNNs) with convolution filters across various datasets; and (ii) to assess the feasibility of deploying DCSNNs trained using backpropagation techniques on low-power FPGA for inference, considering potential configuration adjustments and power requirements. The aforementioned objectives will assist in informing researchers and companies in this field regarding the limitations and unique perspectives of deploying DCSNNs on low-power FPGA devices. The study contributions have three main aspects: (i) the design of a low-power FPGA board featuring a deployable DCSNN chip suitable for object classification tasks; (ii) the inference of TSSL-BP and SGD-BP models with novel network architectures on the FPGA board for object classification tasks; and (iii) a comparative evaluation of the selected spike-based backpropagation techniques and the object classification performance of DCSNNs across multiple metrics using both public (MNIST, CIFAR10, KITTI) and private (INHA_ADAS, INHA_KLP) datasets.

Classification of objects within agriculture landscape from high resolution aerial imagery using maximum likelihood discriminant rules

In this study, the performance of object classification based on four discriminant functions, namely linear, quadratic, diagonal linear and diagonal quadratic is investigated and compared. High-resolution aerial imagery captured from a UAV-based remote sensing platform is used for this purpose. Initially, K-means clustering of 9 clusters is used to assist in the selection of training pixels for the subsequent supervised classification implementation. The classification is experimented with using a training set size of 10 through 100 pixels for each of the discriminant functions. The outcome of the classification shows that training set size 40 and 10 are to be the optimal training set sizes for linear and quadratic discriminant function, and diagonal linear and quadratic discriminant function respectively. Overall, the linear discriminant function is found to have the highest overall accuracy of 91% followed by diagonal linear, quadratic, and diagonal quadratic discriminant function with overall accuracies of 82%, 79%, and 73% respectively.

DMAU-Net: An Attention-Based Multiscale Max-Pooling Dense Network for the Semantic Segmentation in VHR Remote-Sensing Images

High-resolution remote-sensing images cover more feature information, including texture, structure, shape, and other geometric details, while the relationships among target features are more complex. These factors make it more complicated for classical convolutional neural networks to obtain ideal results when performing a feature classification on remote-sensing images. To address this issue, we proposed an attention-based multiscale max-pooling dense network (DMAU-Net), which is based on U-Net for ground object classification. The network is designed with an integrated max-pooling module that incorporates dense connections in the encoder part to enhance the quality of the feature map, and thus improve the feature-extraction capability of the network. Equally, in the decoding, we introduce the Efficient Channel Attention (ECA) module, which can strengthen the effective features and suppress the irrelevant information. To validate the ground object classification performance of the multi-pooling integration network proposed in this paper, we conducted experiments on the Vaihingen and Potsdam datasets provided by the International Society for Photogrammetry and Remote Sensing (ISPRS). We compared DMAU-Net with other mainstream semantic segmentation models. The experimental results show that the DMAU-Net proposed in this paper effectively improves the accuracy of the feature classification of high-resolution remote-sensing images. The feature boundaries obtained by DMAU-Net are clear and regionally complete, enhancing the ability to optimize the edges of features.

Development of Potato Mass Estimation System Based on Deep Learning

Potato is one of the world’s four major food crops as an important resource cultivated in about 150 countries. As precision agriculture has recently attracted increasing attention for its role in improving productivity, interest in yield monitoring is also increasing. Yield monitoring is a precision agriculture technology, and it can help farmhouse business management in the future by contributing to variable fertilization and supply and demand control. The present study was carried out to develop and evaluate a system that uses machine vision and deep learning technologies to estimate potato mass to monitor potato yield. The system performs object classification using the YOLOv5 algorithm to sort out potatoes among various foreign substances, object tracking using the DeepSORT algorithm to track the sorted potatoes, and volume calculation using the lengths of the major axis and minor axis of the tracked potatoes. The results of analyzing the function of the developed yield monitoring system showed an object detection rate of 95.2% and a weight measurement error of 9%, indicating that the computation load must be reduced through algorithm optimization to improve the accuracy and that error correction needs to be performed based on the potato position within the view angle.

Applications of object detection networks in high-power laser systems and experiments

Abstract The recent advent of deep artificial neural networks has resulted in a dramatic increase in performance for object classification and detection. While pre-trained with everyday objects, we find that a state-of-the-art object detection architecture can very efficiently be fine-tuned to work on a variety of object detection tasks in a high-power laser laboratory. In this paper, three exemplary applications are presented. We show that the plasma waves in a laser–plasma accelerator can be detected and located on the optical shadowgrams. The plasma wavelength and plasma density are estimated accordingly. Furthermore, we present the detection of all the peaks in an electron energy spectrum of the accelerated electron beam, and the beam charge of each peak is estimated accordingly. Lastly, we demonstrate the detection of optical damage in a high-power laser system. The reliability of the object detector is demonstrated over 1000 laser shots in each application. Our study shows that deep object detection networks are suitable to assist online and offline experimental analysis, even with small training sets. We believe that the presented methodology is adaptable yet robust, and we encourage further applications in Hz-level or kHz-level high-power laser facilities regarding the control and diagnostic tools, especially for those involving image data.

Constant rank factorisations of smooth maps, with applications to sonar

AbstractSonar systems are frequently used to classify objects at a distance by using the structure of the echoes of acoustic waves as a proxy for the object’s shape and composition. Traditional synthetic aperture processing is highly effective in solving classification problems when the conditions are favourable but relies on accurate knowledge of the sensor’s trajectory relative to the object being measured. This article provides several new theoretical tools that decouple object classification performance from trajectory estimation in synthetic aperture sonar processing. The key insight is that decoupling the trajectory from classification-relevant information involves factoring a function into the composition of two functions. The article presents several new general topological invariants for smooth functions based on their factorisations over function composition. These invariants specialise to the case when a sonar platform trajectory is deformed by a non-small perturbation. The mathematical results exhibited in this article apply well beyond sonar classification problems. This article is written in a way that supports full mathematical generality.

Automated identification of citrus diseases in orchards using deep learning

Citrus disease identification is vital to ensure the quality and quantity of production and minimise the damage in orchards. This study proposed a deep learning-based algorithm to perform automated identification on five common types of citrus diseases in orchards. The proposed algorithm consisted of a detection network to detect the citrus fruit in the complicated background and a classification network to classify them into the corresponding types. Several state-of-the-art network architectures were studied in terms of their object detection and classification performance, and they were evaluated on a dataset of 1524 images taken in field conditions from different orchards in distinct time intervals, scales, angles, and lighting conditions. Based on the experimental results, the algorithm eventually adopted an optimised YOLO-V4 model for detection and the EfficientNet model for classification, and the overall algorithm obtained the accuracy and F1 score of 0.890 and 0.872, respectively. In conclusion, the proposed algorithm is capable of automated citrus disease identification in orchards featuring high efficiency and precision.

Understanding transformation tolerant visual object representations in the human brain and convolutional neural networks

Forming transformation-tolerant object representations is critical to high-level primate vision. Despite its significance, many details of tolerance in the human brain remain unknown. Likewise, despite the ability of convolutional neural networks (CNNs) to exhibit human-like object categorization performance, whether CNNs form tolerance similar to that of the human brain is unknown. Here we provide the first comprehensive documentation and comparison of three tolerance measures in the human brain and CNNs. We measured fMRI responses from human ventral visual areas to real-world objects across both Euclidean and non-Euclidean feature changes. In single fMRI voxels in higher visual areas, we observed robust object response rank-order preservation across feature changes. This is indicative of functional smoothness in tolerance at the fMRI meso-scale level that has never been reported before. At the voxel population level, we found highly consistent object representational structure across feature changes towards the end of ventral processing. Rank-order preservation, consistency, and a third tolerance measure, cross-decoding success (i.e., a linear classifier's ability to generalize performance across feature changes) showed an overall tight coupling. These tolerance measures were in general lower for Euclidean than non-Euclidean feature changes in lower visual areas, but increased over the course of ventral processing for all feature changes. These characteristics of tolerance, however, were absent in eight CNNs pretrained with ImageNet images with varying network architecture, depth, the presence/absence of recurrent processing, or whether a network was pretrained with the original or stylized ImageNet images that encouraged shape processing. CNNs do not appear to develop the same kind of tolerance as the human brain over the course of visual processing.

Comparing deep learning models for low-light natural scene image enhancement and their impact on object detection and classification: Overview, empirical evaluation, and challenges

Low-light image (LLI) enhancement is an important image processing task that aims at improving the illumination of images taken under low-light conditions. Recently, a remarkable progress has been made in utilizing deep learning (DL) approaches for LLI enhancement. This paper provides a concise and comprehensive review and comparative study of the most recent DL models used for LLI enhancement. To our knowledge, this is the first comparative study dedicated to DL-based models for LLI enhancement. We address LLI enhancement in two ways: (i) standalone, as a separate task, and (ii) end-to-end, as a pre-processing stage embedded within another high-level computer vision task, namely object detection and classification. The paper consists of six logical parts. First, we provide an overview of the background and literature in LLI enhancement. Second, we describe the test data and experimental setup of the study. Third, we present a quantitative and qualitative comparison of the visual and perceptual quality achieved by 10 of the most recent DL-based LLI enhancement models. Fourth, we present a comparative analysis for object detection and classification performance achieved by 4 different object detection models applied on LLIs and their enhanced counterparts. Fifth, we perform a feature analysis of DL feature maps extracted from normal, low-light, and enhanced images, and perform the occlusion experiment to better understand the effect of LLI enhancement on the object detection and classification task. Finally, we provide our conclusions and highlight future steps and potential directions.

DiM-PCNet:3D Point Clouds Classification with Multi-scale and Multi-level Feature Net

Nowadays, point clouds are frequently gathered by 3D scanners such as Lidar and Kinect, which produces thousands of point cloud models. Point cloud processing is vital to 3D vision, especially 3D object recognition, positioning, and navigation technology. Addressing uneven data density caused by coordinate frame transformations and the inherent problem of insufficient context connection in point clouds, the DiM-PCNet (Multi-scale and Multi-level Point Clouds Classification Network, and the Di is a prefix to represent double M in Multi-scale and Multi-level) is proposed in this paper. DiM-PCNet is provided for object classification with multi-scale and multi-level features. We encode the point cloud in multi-scale and fully fuse the features with the raw point cloud for keeping the context relationship. In DiM-PCNet, we sample the point clouds from eight parts for multi-scales feature extraction. The multi-scales features are fully fused by multi-level pyramid models. The multi-scale and multi-level strategies are applied in DiM-PCNet, in which the abundant and important features of point clouds are extracted and utilized in the 3D object classification. It is worth noting that the DiM-PCNet feature block can be embedded into the segmentation net, where the accuracy achieved is 87.1%. We conducted experiments on ShapeNet and ModelNet40 and the experimental results show that DiM-PCNet achieves state-of-the-art performance in 3D object classification. The experiment shows competitive performance on robustness and segmentation tasks.

Soft and flexible large-strain piezoresistive sensors: On implementing proprioception, object classification and curvature estimation systems in adaptive, human-like robot hands

Soft and flexible large strain piezoresistive sensors can be widely used in robotic applications due to their simplicity and compact and lightweight structure. In this study, combining conductive filler of carbon black with elastomers, a soft elastomer strain sensor that can reach very large elongations is presented. These sensors have good linearity and extensibility. Stress relaxation does affect the accuracy of the sensor, but this can be minimized. In this work, we demonstrate the application of this sensor into the structure of an adaptive, human-like robot hand to perform object classification and object curvature estimation employing hand proprioception (measuring the bending angles of the fingers). Experiments demonstrated that the sensors could assist the hand with the ability to classify the object’s basic shape using a single grasp.

DFT-Net: Deep Feature Transformation Based Network for Object Categorization and Part Segmentation in 3-Dimensional Point Clouds.

Unlike 2-dimensional (2D) images, direct 3-dimensional (3D) point cloud processing using deep neural network architectures is challenging, mainly due to the lack of explicit neighbor relationships. Many researchers attempt to remedy this by performing an additional voxelization preprocessing step. However, this adds additional computational overhead and introduces quantization error issues, limiting an accurate estimate of the underlying structure of objects that appear in the scene. To this end, in this article, we propose a deep network that can directly consume raw unstructured point clouds to perform object classification and part segmentation. In particular, a Deep Feature Transformation Network (DFT-Net) has been proposed, consisting of a cascading combination of edge convolutions and a feature transformation layer that captures the local geometric features by preserving neighborhood relationships among the points. The proposed network builds a graph in which the edges are dynamically and independently calculated on each layer. To achieve object classification and part segmentation, we ensure point order invariance while conducting network training simultaneously—the evaluation of the proposed network has been carried out on two standard benchmark datasets for object classification and part segmentation. The results were comparable to or better than existing state-of-the-art methodologies. The overall score obtained using the proposed DFT-Net is significantly improved compared to the state-of-the-art methods with the ModelNet40 dataset for object categorization.