Robust Object Detection Research Articles

Synthetic Data Augmentation Using Multiscale Attention CycleGAN for Aircraft Detection in Remote Sensing Images

Deep learning approaches require enough training samples to perform well, but it is a challenge to collect enough real training data and label them manually. In this letter, we propose a practical framework for automatically generating content-rich synthetic images with ground-truth annotations. By rendering 3-D CAD models, we generate two synthetic aircraft image data sets with wide distribution (Syn N and Syn U). For improving the quality of synthetic images, we propose a multiscale attention module which enhances the Cycle-Consistent Adversarial Network (CycleGAN) in spatial and channel dimensions. Then, we compare the synthetic images before and after translation qualitatively and quantitatively. Experiments on Northwestern Polytechnical University (NWPU) very high resolution (VHR)-10, University of Chinese Academy of Sciences, orientation robust object detection in aerial images (UCAS-AOD), and benchmark for object DetectIon in Optical Remote sensing images (DIOR) data sets demonstrate that synthetic data augmentation can improve the performance of aircraft detection in remote sensing images, especially when real data are insufficient. Synthetic data are available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://weix-liu.github.io/</uri> .

Feature Enhancement Network for Object Detection in Optical Remote Sensing Images

Automatic and robust object detection in remote sensing images is of vital significance in real-world applications such as land resource management and disaster rescue. However, poor performance arises when the state-of-the-art natural image detection algorithms are directly applied to remote sensing images, which largely results from the variations in object scale, aspect ratio, indistinguishable object appearances, and complex background scenario. In this paper, we propose a novel Feature Enhancement Network (FENet) for object detection in optical remote sensing images, which consists of a Dual Attention Feature Enhancement (DAFE) module and a Context Feature Enhancement (CFE) module. Specifically, the DAFE module is introduced to highlight the network to focus on the distinctive features of the objects of interest and suppress useless ones by jointly recalibrating the spatial and channel feature responses. The CFE module is designed to capture global context cues and selectively strengthen class-aware features by leveraging image-level contextual information that indicates the presence or absence of the object classes. To this end, we employ a context encoding loss to regularize the model training which promotes the object detector to understand the scene better and narrows the probable object categories in prediction. We achieve our proposed FENet by unifying DAFE and CFE into the framework of Faster R-CNN. In the experiments, we evaluate our proposed method on two large-scale remote sensing image object detection datasets including DIOR and DOTA and demonstrate its effectiveness compared with the baseline methods.

A Fully Spiking Hybrid Neural Network for Energy-Efficient Object Detection.

This paper proposes a Fully Spiking Hybrid Neural Network (FSHNN) for energy-efficient and robust object detection in resource-constrained platforms. The network architecture is based on a Spiking Convolutional Neural Network using leaky-integrate-fire neuron models. The model combines unsupervised Spike Time-Dependent Plasticity (STDP) learning with back-propagation (STBP) learning methods and also uses Monte Carlo Dropout to get an estimate of the uncertainty error. FSHNN provides better accuracy compared to DNN based object detectors while being more energy-efficient. It also outperforms these object detectors, when subjected to noisy input data and less labeled training data with a lower uncertainty error.

POAT-Net: Parallel Offset-Attention Assisted Transformer for 3D Object Detection for Autonomous Driving

3D object detection is playing a key role in the perception process of autonomous driving and industrial robots automation. Inherent characteristics of point cloud raise an enormous challenge to both spatial representation and association analysis. Unordered point cloud spatial data structure and density variations caused by gradually varying distances to LiDAR make accurate and robust 3D object detection even more difficult. In this paper, we present a novel transformer network POAT-Net for 3D point cloud object detection. Transformer is credited with the great success in Natural Language Processing (NLP) and exhibiting inspiring potentials in point cloud processing. Our method POAT-Net is inherently insensitive to element permutations within the unordered point cloud. The associations between local points contribute significantly to 3D object detection or other 3D tasks. Parallel offset-attention is leveraged to highlight and capture subtle associations between local points. To overcome the non-uniform density distribution of different objects, we exploit Normalized multi-resolution Grouping (NMRG) strategy to enhance the non-uniform density adaptive ability for POAT-Net. Quantitative experimental results on KITTI3D dataset demonstrate our method achieves the state-of-the-art performance.

Multi-class Object Detection Model in Satellite Images Using Convolutional Neural Network

The accurate multi-detection of objects in satellite images has become very essential due to the high criminal activities that posed security threat to humanity all over the world. However, there are significant limitations of traditional methods of multi-object detection such as matching based techniques and object based image analysis. Although Convolutional neural network and image processing techniques has been proved to be essential fields in so many applications of computer vision specifically multi-object detection, multi-object classification, object retrieval, object recognition and object segmentation in a digital image or video, however, multi-object detection especially in satellite images suffer from problems such as shadow, camouflage, and occlusion. The aim of this research work was to design a robust multi-class object detection model in satellite images using image processing techniques and convolutional neural network with a particular concern on image preprocessing, image denoising and image enhancement to enable address the issue of noise in satellite images. The Satellite image that are propose for this model is LandSat-8, because it is free access for research and have a tract record in terms of consistency. The proposed model applied supervised learning algorithm for training different samples of labeled data for the model to enable the system detect vegetation, water bodies, road networks and building. This research will enable the government to know the positions as well as the coordinates of every thick forest, drainage, road networks and buildings in the forest for security reasons. It is at the heart of this research to pave away for the full implementation of this model using either MATLAB or Python Programming.

Generation of Background Model Image Using Foreground Model

Proper consideration of the temporal domain and the spatial domain is essential to perform robust foreground object detection in visual surveillance. However, there are difficulties in considering long-term temporal information with CNN-based methods. To solve this limitation, classical algorithms and some deep learning-based algorithms have used a background model image. However, acquiring a sophisticated background model image is also one of the complex problems. Most of the algorithms take a lot of time to initialize the background model image and generate many errors in the presence of a static foreground. This paper proposes an algorithm for generating a background model image using a deep-learning-based segmenter to solve this problem. The proposed method shows a 66.25% lower mean square error (MSE) than the background subtraction (BGS) algorithm and 79.25% lower than the latest deep learning algorithm in the SBI dataset. In addition, in the deep learning-based segmenter that uses a background image as input, replacing the background image of BGS algorithm with the background image of the proposed method shows a 38.63% reduction in the false detection rate (PWC).

VAE로 새성한 배경 이미지를 이용한 전경 물체 분할

In visual surveillance, the robust detection of foreground objects under diverse environmental changes is the main goal. In the case of traditional algorithms, they usually obtain a background model image through the statistical analysis, which is used for finding foreground objects by comparing with a current image. Recently, many deep learning-based visual surveillance algorithms have been proposed, and they show improved performance than traditional algorithms. However, they usually show a good performance when test images are similar to training environments. Retraining is required to have an improved result in scenes which are different from training environments. In this paper, we aim to have an improved deep learning-based visual surveillance algorithm which also an improvement in new scenes. We use two types of images as the input of the U-Net, which produces a foreground segmentation map as output. A background model image which is generated by VAE is used one type of input. The other type of input to the network is multiple original images. Also, we train the presented network using multiple scenes while most conventional deep learning-based visual surveillance algorithms newly train a network per scene. Experimental results using various open datasets show the feasibility of the presented algorithm.

RAMP-CNN: A Novel Neural Network for Enhanced Automotive Radar Object Recognition

Millimeter-wave radars are being increasingly integrated into commercial vehicles to support new advanced driver-assistance systems by enabling robust and high-performance object detection, localization, as well as recognition - a key component of new environmental perception. In this paper, we propose a novel radar multiple-perspectives convolutional neural network (RAMP-CNN) that extracts the location and class of objects based on further processing of the range-velocity-angle (RVA) heatmap sequences. To bypass the complexity of 4D convolutional neural networks (NN), we propose to combine several lower-dimension NN models within our RAMP-CNN model that nonetheless approaches the performance upper-bound with lower complexity. The extensive experiments show that the proposed RAMP-CNN model achieves better average recall and average precision than prior works in all testing scenarios. Besides, the RAMP-CNN model is validated to work robustly under nighttime, which enables low-cost radars as a potential substitute for pure optical sensing under severe conditions.

Lipschitz Constrained Neural Networks for Robust Object Detection at Sea

Autonomous ships rely on sensory data to perceive objects of interest in their environment. Deep Learning based object detection in the image domain commonly used to solve this issue. The robustness of such approaches in non-ideal conditions is, however, still to be proven. In this work state of the art methods are applied on the RetinaNet architecture attempting to create a more robust object detection network given noisy input data. The GroupSort activation function and Spectral Normalization is used and the results are compared to the standard RetinaNet network. Our findings show that these modifications perform better and ensure robustness under moderate noise levels, than the standard RetinaNet network.

Target detection using features for sonar images

Robust object detection in sonar images is an important task for underwater exploration, navigation and mapping. Current methods make assumptions about the shape, highlight or shadow of an object, which may be invalid for some environments or targets. We focus on the area of feature extraction-based detection, which does not rely on information about the shape of the target, towards a robust framework for target detection for a variety of seabed structures and target types. The proposed framework first estimates the seabed type from the spatial distribution of features to determine the set of optimal parameters, and then obtains a set of features which are filtered according to intensity and distribution to yield a detection decision. The proposed method also provides a means to determine the seabed type, and a machine-learning based methodology to choose the feature detectors' parameters to match the evaluated seabed type. We report the performance of a variety of feature detectors for a simulated environment and of one feature detector for real sonar images. Results show the importance of choosing the parameters of the feature extractors based on the current environmental conditions and the proposed method obtains a favourable tradeoff between detection and false alarm rates.

A Two-Phase Cross-Modality Fusion Network for Robust 3D Object Detection.

A two-phase cross-modality fusion detector is proposed in this study for robust and high-precision 3D object detection with RGB images and LiDAR point clouds. First, a two-stream fusion network is built into the framework of Faster RCNN to perform accurate and robust 2D detection. The visible stream takes the RGB images as inputs, while the intensity stream is fed with the intensity maps which are generated by projecting the reflection intensity of point clouds to the front view. A multi-layer feature-level fusion scheme is designed to merge multi-modal features across multiple layers in order to enhance the expressiveness and robustness of the produced features upon which region proposals are generated. Second, a decision-level fusion is implemented by projecting 2D proposals to the space of the point cloud to generate 3D frustums, on the basis of which the second-phase 3D detector is built to accomplish instance segmentation and 3D-box regression on the filtered point cloud. The results on the KITTI benchmark show that features extracted from RGB images and intensity maps complement each other, and our proposed detector achieves state-of-the-art performance on 3D object detection with a substantially lower running time as compared to available competitors.

Dataset of annotated food crops and weed images for robotic computer vision control.

Weed management technologies that can identify weeds and distinguish them from crops are in need of artificial intelligence solutions based on a computer vision approach, to enable the development of precisely targeted and autonomous robotic weed management systems. A prerequisite of such systems is to create robust and reliable object detection that can unambiguously distinguish weed from food crops. One of the essential steps towards precision agriculture is using annotated images to train convolutional neural networks to distinguish weed from food crops, which can be later followed using mechanical weed removal or selected spraying of herbicides. In this data paper, we propose an open-access dataset with manually annotated images for weed detection. The dataset is composed of 1118 images in which 6 food crops and 8 weed species are identified, altogether 7853 annotations were made in total. Three RGB digital cameras were used for image capturing: Intel RealSense D435, Canon EOS 800D, and Sony W800. The images were taken on food crops and weeds grown in controlled environment and field conditions at different growth stages

TANet: Robust 3D Object Detection from Point Clouds with Triple Attention

In this paper, we focus on exploring the robustness of the 3D object detection in point clouds, which has been rarely discussed in existing approaches. We observe two crucial phenomena: 1) the detection accuracy of the hard objects, e.g., Pedestrians, is unsatisfactory, 2) when adding additional noise points, the performance of existing approaches decreases rapidly. To alleviate these problems, a novel TANet is introduced in this paper, which mainly contains a Triple Attention (TA) module, and a Coarse-to-Fine Regression (CFR) module. By considering the channel-wise, point-wise and voxel-wise attention jointly, the TA module enhances the crucial information of the target while suppresses the unstable cloud points. Besides, the novel stacked TA further exploits the multi-level feature attention. In addition, the CFR module boosts the accuracy of localization without excessive computation cost. Experimental results on the validation set of KITTI dataset demonstrate that, in the challenging noisy cases, i.e., adding additional random noisy points around each object, the presented approach goes far beyond state-of-the-art approaches. Furthermore, for the 3D object detection task of the KITTI benchmark, our approach ranks the first place on Pedestrian class, by using the point clouds as the only input. The running speed is around 29 frames per second.

Bio-inspired head detection framework based on online learning algorithm

Online learning algorithms have been widely used to address vision-related issues such as object detection and tracking. However, a robust online learning object detection system that can continuously improve performance through self-learning continues to elude designers. This study proposes a novel online learning framework, which combines detection and verification modules to train a scene-specific head detector on a fly. For the detection module, a proposed online bootstrap cascade classifier is employed as the object detector of the framework. The cascade decision strategy is used to integrate a number of weak online classifiers. The resulting system contains sufficient weak classifiers and maintains a low computation cost. During the online learning process, the complexity of the cascade structure adapts to the difficulty of the detection task. For the verification module, a simple yet effective particle filter tracking algorithm, based on information fusion, is used to automatically label online learning samples produced by detection responses. With this method, the object detector improves detection performance by autonomously learning the samples. The online head detection framework is ported to the NVIDIA Jetson TK1 embedded platform, which enables the platform to recognize different head postures through self-learning. Experimental results on three video datasets demonstrate the effectiveness of the framework.

Multi-Scale Feature Integrated Attention-Based Rotation Network for Object Detection in VHR Aerial Images.

Accurate and robust detection of multi-class objects in very high resolution (VHR) aerial images has been playing a significant role in many real-world applications. The traditional detection methods have made remarkable progresses with horizontal bounding boxes (HBBs) due to CNNs. However, HBB detection methods still exhibit limitations including the missed detection and the redundant detection regions, especially for densely-distributed and strip-like objects. Besides, large scale variations and diverse background also bring in many challenges. Aiming to address these problems, an effective region-based object detection framework named Multi-scale Feature Integration Attention Rotation Network (MFIAR-Net) is proposed for aerial images with oriented bounding boxes (OBBs), which promotes the integration of the inherent multi-scale pyramid features to generate a discriminative feature map. Meanwhile, the double-path feature attention network supervised by the mask information of ground truth is introduced to guide the network to focus on object regions and suppress the irrelevant noise. To boost the rotation regression and classification performance, we present a robust Rotation Detection Network, which can generate efficient OBB representation. Extensive experiments and comprehensive evaluations on two publicly available datasets demonstrate the effectiveness of the proposed framework.

Robust one-stage object detection with location-aware classifiers

Recent progress on one-stage detectors focuses on improving the quality of bounding boxes, while they pay less attention to the classification head. In this work, we focus on investigating the influence of the classification head. To understand the behavior of the classifier in one-stage detectors, we resort to the methods of the Explainable deep learning area. We visualize its learned representations via activation maps and analyze its robustness to image scene context. Based on the analysis, we observe that the classifier limits the performance of the detector due to its limited receptive field and the lack of object locations. Then, we design a simple but efficient location-aware multi-dilation module (LAMD) to enhance the weak classifier. We conduct extensive experiments on the COCO benchmark to validate the effectiveness of LAMD. The results suggest that our LAMD can achieve consistent improvements and leads to robust detection across various one-stage detectors with different backbones.

Multi-Sensor Fusion in Dynamic Environment using Evidential Grid Mapping

The performance of autonomous agents in both commercial and consumer applications increases along with their situational awareness. Tasks such as obstacle avoidance, agent to agent interaction, and path planning are directly dependent upon their ability to convert sensor readings into scene understanding. Central to this is the ability to detect and recognize objects. Many object detection methodologies operate on a single modality such as vision or LiDAR. Camera-based object detection models benefit from an abundance of feature-rich information for classifying different types of objects. LiDAR-based object detection models use sparse point clouds, where each point contains accurate 3D position of object surfaces. Camera-based methods lack accurate object to lens distance measurements, while LiDAR-based methods lack dense feature-rich details. By utilizing information from both camera and LiDAR sensors, advanced object detection and identification is possible. In this work, we introduce a deep learning framework for fusing these modalities and produce a robust real-time 3D bounding box object detection network. We demonstrate qualitative and quantitative analysis of the proposed fusion model on the popular KITTI dataset.

Metrology Impact of Advanced Driver Assistance Systems

With the growing demand for robust object detection algorithms in self-driving systems, it is important to consider the varying lighting and weather conditions in which cars operate all year round. The goal of our work is to gain a deeper understanding of meaningful strategies for selecting and merging training data from currently available databases and self-annotated videos in the context of automotive night scenes. We retrain an existing Convolutional Neural Network (YOLOv3) to study the influence of different training dataset combinations on the final object detection results in nighttime and low-visibility traffic scenes. Our evaluation shows that a suitable selection of training data from the GTSRD, VIPER, and BDD databases in conjunction with selfrecorded night scenes can achieve an mAP of 63,5% for ten object classes, which is an improvement of 16,7% when compared to the performance of the original YOLOv3 network on the same test set.

Robust object detection in images corrupted by impulse noise

Robust object detection in images corrupted by impulse noise

Robust Object Detection in Aerial Imagery Based on Multi-Scale Detector and Soft Densely Connected

Object detection in aerial images is vital for autonomous guidance, navigation and control, and situational awareness. However, there are still many challenges facing researchers in this filed, including the target scales, the perspectives in taking pictures, and the highly complex background. The present paper introduces a robust object detector which is optimized for handling with multi-scale objects and the overhead capturing perspective object instances in aerial images. Firstly, in the feature extraction stage, an effective multi-scale detector (MSD) is designed to search for objects with different scales in feature maps. After that, when detecting a small target from a cluttered background, both the shallow and deep layer features are densely connected by the deconvolution after tackling the issues of low dimensionality in deep layers and inadequate representation of small objects. In the experiments part, we analyze the impacts of the above mentioned components on the model and make a comparison between the method at issue and other state-of-the-art approaches on two publicly-available datasets captured by satellites and high-altitude UAVs. The results show that the proposed method, which is applicable to a wider range of aerial images, is more effective and robust.