Deep Object Detection Research Articles

A Deep Learning Framework Performance Evaluation to Use YOLO in Nvidia Jetson Platform

Deep learning-based object detection technology can efficiently infer results by utilizing graphics processing units (GPU). However, when using general deep learning frameworks in embedded systems and mobile devices, processing functionality is limited. This allows deep learning frameworks such as TensorFlow-Lite (TF-Lite) and TensorRT (TRT) to be optimized for different hardware. Therefore, this paper introduces a performance inference method that fuses the Jetson monitoring tool with TensorFlow and TRT source code on the Nvidia Jetson AGX Xavier platform. In addition, central processing unit (CPU) utilization, GPU utilization, object accuracy, latency, and power consumption of the deep learning framework were compared and analyzed. The model is You Look Only Once Version4 (YOLOv4), and the dataset uses Common Objects in Context (COCO) and PASCAL Visual Object Classes (VOC). We confirmed that using TensorFlow results in high latency. We also confirmed that TensorFlow-TensorRT (TF-TRT) and TRT using Tensor Cores provide the most efficiency. However, it was confirmed that TF-Lite showed the lowest performance because it utilizes a GPU limited to mobile devices. Through this paper, we think that when developing deep learning-related object detection technology on the Nvidia Jetson platform or desktop environment, services and research can be efficiently conducted through measurement results.

Dilated convolution based RCNN using feature fusion for Low-Altitude aerial objects

The low-altitude aerial objects are hard to detect by existing deep learning-based object detectors because of the scale variance, small size, and occlusion-related problems. Deep learning-based detectors do not consider contextual information about the scale information of small-sized objects in low-altitude aerial images. This paper proposes a new system using the concept of receptive fields and fusion of feature maps to improve the efficiency of deep object detectors in low-altitude aerial images. A Dilated ResNet Module (DRM) is proposed, motivated from the trident networks, which works on dilated convolutions to study the contextual data for specifically small-sized objects. Applicability of this component builds the model strong towards scale variations in low-altitude aerial objects. Then, Feature Fusion Module (FFM) is created to offer semantic intelligence for better detection of low-altitude aerial objects. We have chosen vastly deployed faster RCNN as the base detector for the proposal of our technique. The dilated convolution-based RCNN using feature fusion (DCRFF) system is implemented on a benchmark low-altitude UAV based-object detection dataset, VisDrone, which contains multiple object categories of pedestrians, vehicles in crowded scenes. The experiments exhibit the enactment of the given detector on chosen low-altitude aerial object dataset. The proposed system of DCRFF achieves 35.04% mAP on the challenging VisDrone dataset, indicating an average improvement of 2% when compared.

A comparative assessment of deep object detection models for blood smear analysis

A blood smear is a common type of blood test where blood sample is taken from a patient, smear is made from the sample followed by observation of red blood cells, white blood cells and platelets. A pathologist carefully observes the sample and manually counts the number of RBC, WBC and platelets. This entire process from creating a smear to manually counting each element is tedious and susceptible to human errors. That is why, with the advancement of deep learning, various object detection techniques have become useful for automating the process and mitigating human errors in blood smear analysis. This work presents a comparative assessment of three different object detection models namely Faster R-CNN, EfficientDet D3 and CenterNet Hourglass, and presents their respective inference results. The three models have been compared using the COCO evaluation metrics to identify the best model performance for the given task. It is observed that out of the three models, the Faster R-CNN model performs the best in detecting WBCs and platelets in microscopic blood smear images with an average precision of 99.4%. Critical tasks like medical image processing require accurate predictions to prevent unintended ramifications. Therefore, while slower in terms of inference time, Faster R-CNN is the go-to model where accuracy is the priority. The work is also compared with the existing work in this domain to prove its efficiency.

Intelligent Image Saliency Detection Method Based on Convolution Neural Network Combining Global and Local Information

Image saliency object detection can rapidly extract useful information from image scenes and further analyze it. At present, the traditional saliency target detection technology still has the edge of outstanding target that cannot be well preserved. Convolutional neural network (CNN) can extract highly general deep features from the images and effectively express the essential feature information of the images. This paper designs a model which applies CNN in deep saliency object detection tasks. It can efficiently optimize the edges of foreground objects and realize highly efficient image saliency detection through multilayer continuous feature extraction, refinement of layered boundary, and initial saliency feature fusion. The experimental result shows that the proposed method can achieve more robust saliency detection to adjust itself to complex background environment.

Deep object detection framework for automated quality inspection in assembly operations

The recent advance of flexible production systems requires fast and objective quality inspection of products. Computer vision based deep Convolutional Neural Networks (CNNs), are suitable for such applications since they provide automated, non-destructive, and cost-effective techniques to accomplish the requirements, hence eliminating the human operators or other inspections. In this paper a deep learning object detection framework is presented, able to detect correct, misaligned, and missing objects in complex scenes of the production line. Furthermore, the proposed architecture provides interfaces that allow the seamless integration of the model with varying manufacturing systems.

HM-Net: A Regression Network for Object Center Detection and Tracking on Wide Area Motion Imagery

Wide Area Motion Imagery (WAMI) yields high resolution images with a large number of extremely small objects. Target objects have large spatial displacements throughout consecutive frames. This nature of WAMI images makes object tracking and detection challenging. In this paper, we present our deep neural network-based combined object detection and tracking model, namely, Heat Map Network (HM-Net). HM-Net is significantly faster than state-of-the-art frame differencing and background subtraction-based methods, without compromising detection and tracking performances. HM-Net follows object center-based joint detection and tracking paradigm. Simple heat map-based predictions support unlimited number of simultaneous detections. The proposed method uses two consecutive frames and the object detection heat map obtained from the previous frame as input, which helps HM-Net monitor spatio-temporal changes between frames and keeps track of previously predicted objects. Although reuse of prior object detection heat map acts as a vital feedback-based memory element, it can lead to unintended surge of false positive detections. To increase robustness of the method against false positives and to eliminate low confidence detections, HM-Net employs novel feedback filters and advanced data augmentations. HM-Net outperforms state-of-the-art WAMI moving object detection and tracking methods on WPAFB dataset with its 96.2% F1 and 94.4% mAP detection scores, while achieving a 61.8% mAP tracking score on the same dataset.

Автоматическая разметка данных для сегментации изображений документов с использованием глубоких нейронных сетей

The article proposes a new method for automatic data annotation for solving the problem of document image segmentation using deep object detection neural networks. The format of marked PDF files is considered as the initial data for markup. The peculiarity of this format is that it includes hidden marks that describe the logical and physical structure of the document. To extract them, a tool has been developed that simulates the operation of a stack-based printing machine according to the PDF format specification. For each page of the document, an image and annotation are generated in PASCAL VOC format. The classes and coordinates of the bounding boxes are calculated during the interpretation of the labeled PDF file based on the labels. To test the method, a collection of marked up PDF files was formed from which images of document pages and annotations for three segmentation classes (text, table, figure) were automatically obtained. Based on these data, a neural network of the EfficientDet D2 architecture was trained. The model was tested on manually labeled data from the same domain, which confirmed the effectiveness of using automatically generated data for solving applied problems.

Robust Object Detection via Adversarial Novel Style Exploration.

Deep object detection models trained on clean images may not generalize well on degraded images due to the well-known domain shift issue. This hinders their application in real-life scenarios such as video surveillance and autonomous driving. Though domain adaptation methods can adapt the detection model from a labeled source domain to an unlabeled target domain, they struggle in dealing with open and compound degradation types. In this paper, we attempt to address this problem in the context of object detection by proposing a robust object Detector via Adversarial Novel Style Exploration (DANSE). Technically, DANSE first disentangles images into domain-irrelevant content representation and domain-specific style representation under an adversarial learning framework. Then, it explores the style space to discover diverse novel degradation styles that are complementary to those of the target domain images by leveraging a novelty regularizer and a diversity regularizer. The clean source domain images are transferred into these discovered styles by using a content-preserving regularizer to ensure realism. These transferred source domain images are combined with the target domain images and used to train a robust degradation-agnostic object detection model via adversarial domain adaptation. Experiments on both synthetic and real benchmark scenarios confirm the superiority of DANSE over state-of-the-art methods.

PDE: A Real-Time Object Detection and Enhancing Model under Low Visibility Conditions

Deep object detection models are important tools that can accurately detect objects and frame them for the user in real time. However, in low visibility conditions, such as fog or low light conditions, the captured images are underexposed and blurred, which negatively affects the recognition accuracy and is not well visible to humans. In addition, the image enhancement model is complex and time-consuming. Using the image enhancement model before the object recognition model cannot meet the real-time requirements. Therefore, we propose the Parallel Detection and Enhancement model (PDE), which detects objects and enhances poorly visible images in parallel and in real time. Specifically, we introduce the specially designed tiny prediction head along with coordinated attention and multi-stage concatenation modules to better detect underexposed and blurred objects. For the parallel image enhancement model, we adaptively develop improved weighting evaluation models for each “3D Lookup Table” module. As a result, PDE achieves better detection accuracy for poorly visible objects and more user-friendly reference in real time. Experimental results show that PDE has significantly better object recognition performance than the state-of-the-art on real foggy (8.9%) and low-light (20.6%) datasets.

Object-wise Secure Image Display Method for Screen Capture Protection

In this study, we propose an image copyright protection technology to prevent analog hole attacks using a deep object detection network. Instead of applying a virtual fence to the whole image, only a region of copyright is protected with fences. The region of copyright interest is called the protection zone and is generated by merging bounding boxes found with a YOLO network. The protection zone is adjusted to satisfy the fence generation rules, and then the virtual fence is finally applied. The main goal of this study is to attenuate the image degradation caused by virtual fences while preserving copyright protection capability. Since the virtual fence method uses the afterimage effect of the human visual system, it is necessary to generate images perceived by a human eye at every moment in order to calculate the objective image quality compared to the original unprotected image. Perceived images are generated at every 0.001 seconds using the mathematical model of the afterimage effect. The protection zone is found using object bounding boxes detected using a YOLO deep neural network. We applied the proposed method to four animation images, and the simulation results show that it achieves image quality that is about 3dB better than that of other methods.

Adversarial point cloud perturbations against 3D object detection in autonomous driving systems

Deep learning models have been demonstrated vulnerable to adversarial attacks even with imperceptible perturbations. As such, the reliability of existing deep neural networks-based autonomous driving systems can suffer. However, deep 3D models have applications in various Cyber-Physical Systems (CPSs) with safety-critical requirements, particularly autonomous driving systems. In this paper, the robustness of deep 3D object detection models under adversarial point cloud perturbations has been investigated. A novel method is developed to generate 3D adversarial examples from point cloud perturbations, which are common due to the intrinsic characteristics of the data captured by 3D sensors, e.g., LiDAR. The generation of adversarial samples is supervised by a dual loss, which constitutes an adversarial loss and a perturbation loss. The adversarial loss produces a point cloud with the property of aggressiveness, while the perturbation loss enforces the produced point cloud subject to visual imperception. We demonstrate that the method can successfully attack 3D object detection models in most cases, and expose their vulnerability to physical-world attacks in the form of point cloud perturbations. We perform a thorough evaluation of popular deep 3D object detectors in an adversarial setting on the KITTI vision benchmark. Experimental results show that current deep 3D object detection models are susceptible to adversarial attacks in the context of autonomous driving, and their performances are degraded by a large margin in the presence of adversarial point clouds generated by the proposed method.

Detecting Salient Image Objects Using Color Histogram Clustering for Region Granularity.

Salient object detection represents a novel preprocessing stage of many practical image applications in the discipline of computer vision. Saliency detection is generally a complex process to copycat the human vision system in the processing of color images. It is a convoluted process because of the existence of countless properties inherent in color images that can hamper performance. Due to diversified color image properties, a method that is appropriate for one category of images may not necessarily be suitable for others. The selection of image abstraction is a decisive preprocessing step in saliency computation and region-based image abstraction has become popular because of its computational efficiency and robustness. However, the performances of the existing region-based salient object detection methods are extremely hooked on the selection of an optimal region granularity. The incorrect selection of region granularity is potentially prone to under- or over-segmentation of color images, which can lead to a non-uniform highlighting of salient objects. In this study, the method of color histogram clustering was utilized to automatically determine suitable homogenous regions in an image. Region saliency score was computed as a function of color contrast, contrast ratio, spatial feature, and center prior. Morphological operations were ultimately performed to eliminate the undesirable artifacts that may be present at the saliency detection stage. Thus, we have introduced a novel, simple, robust, and computationally efficient color histogram clustering method that agglutinates color contrast, contrast ratio, spatial feature, and center prior for detecting salient objects in color images. Experimental validation with different categories of images selected from eight benchmarked corpora has indicated that the proposed method outperforms 30 bottom-up non-deep learning and seven top-down deep learning salient object detection methods based on the standard performance metrics.

A novel method for finding grasping handles in a clutter using RGBD Gaussian mixture models

AbstractThe paper proposes a novel method to detect graspable handles for picking objects from a confined and cluttered space, such as the bins of a rack in a retail warehouse. The proposed method combines color and depth curvature information to create a Gaussian mixture model that can segment the target object from its background and imposes the geometrical constraints of a two-finger gripper to localize the graspable regions. This helps in overcoming the limitations of a poorly trained deep network object detector and provides a simple and efficient method for grasp pose detection that does not require a priori knowledge about object geometry and can be implemented online with near real-time performance. The efficacy of the proposed approach is demonstrated through simulation as well as real-world experiment.

Multitarget Tracking Using Siamese Neural Networks

In this article, we detect and track visual objects by using Siamese network or twin neural network. The Siamese network is constructed to classify moving objects based on the associations of object detection network and object tracking network, which are thought of as the two branches of the twin neural network. The proposed tracking method was designed for single-target tracking, which implements multitarget tracking by using deep neural networks and object detection. The contributions of this article are stated as follows. First, we implement the proposed method for visual object tracking based on multiclass classification using deep neural networks. Then, we attain multitarget tracking by combining the object detection network and the single-target tracking network. Next, we uplift the tracking performance by fusing the outcomes of the object detection network and object tracking network. Finally, we speculate on the object occlusion problem based on IoU and similarity score, which effectively diminish the influence of this issue in multitarget tracking.

Subtask Attention Based Object Detection in Remote Sensing Images

Object detection in remote sensing images (RSIs) is one of the basic tasks in the field of remote sensing image automatic interpretation. In recent years, the deep object detection frameworks of natural scene images (NSIs) have been introduced into object detection on RSIs, and the detection performance has improved significantly because of the powerful feature representation. However, there are still many challenges concerning the particularities of remote sensing objects. One of the main challenges is the missed detection of small objects which have less than five percent of the pixels of the big objects. Generally, the existing algorithms choose to deal with this problem by multi-scale feature fusion based on a feature pyramid. However, the benefits of this strategy are limited, considering that the location of small objects in the feature map will disappear when the detection task is processed at the end of the network. In this study, we propose a subtask attention network (StAN), which handles the detection task directly on the shallow layer of the network. First, StAN contains one shared feature branch and two subtask attention branches of a semantic auxiliary subtask and a detection subtask based on the multi-task attention network (MTAN). Second, the detection branch uses only low-level features considering small objects. Third, the attention map guidance mechanism is put forward to optimize the network for keeping the identification ability. Fourth, the multi-dimensional sampling module (MdS), global multi-view channel weights (GMulW) and target-guided pixel attention (TPA) are designed for further improvement of the detection accuracy in complex scenes. The experimental results on the NWPU VHR-10 dataset and DOTA dataset demonstrated that the proposed algorithm achieved the SOTA performance, and the missed detection of small objects decreased. On the other hand, ablation experiments also proved the effects of MdS, GMulW and TPA.

Co-Training for Deep Object Detection: Comparing Single-Modal and Multi-Modal Approaches.

Top-performing computer vision models are powered by convolutional neural networks (CNNs). Training an accurate CNN highly depends on both the raw sensor data and their associated ground truth (GT). Collecting such GT is usually done through human labeling, which is time-consuming and does not scale as we wish. This data-labeling bottleneck may be intensified due to domain shifts among image sensors, which could force per-sensor data labeling. In this paper, we focus on the use of co-training, a semi-supervised learning (SSL) method, for obtaining self-labeled object bounding boxes (BBs), i.e., the GT to train deep object detectors. In particular, we assess the goodness of multi-modal co-training by relying on two different views of an image, namely, appearance (RGB) and estimated depth (D). Moreover, we compare appearance-based single-modal co-training with multi-modal. Our results suggest that in a standard SSL setting (no domain shift, a few human-labeled data) and under virtual-to-real domain shift (many virtual-world labeled data, no human-labeled data) multi-modal co-training outperforms single-modal. In the latter case, by performing GAN-based domain translation both co-training modalities are on par, at least when using an off-the-shelf depth estimation model not specifically trained on the translated images.

Video salient object detection using dual-stream spatiotemporal attention

Video salient object detection plays an important role in many exciting applications in different areas. However, the existing deep learning-based video salient object detection methods still struggle in scenes of large salient object variabilities and great background scene diversity between and within frames. In this paper, we propose a dual-stream spatiotemporal attention network (DSSANet) for saliency detection in videos. It creatively introduces a multiplex attention mechanism to effectively extract and fuse spatiotemporal features of video salient object over frames in the video, thereby improving saliency detection performance. The DSSANet consists of: (1) A context feature path leverages a novel attention-augmented convolutional LSTM to effectively model the long-range dependency of the great temporal variation in the salient object over frames. (2) A content feature path creatively leverages an attention-based 1D dilated convolution to effectively model the local pixel correlation structure of each pixel in the salient object and the surrounding objects. (3) A refinement fusion module fuses these two features from their paths and further refines the fused feature by an attention-based feature selection. By integrating these three parts, DSSANet accurately detects the salient object from the video. The extensive experiments are performed on four public datasets and demonstrate the effectiveness of DSSANet and the superiority to five state-of-the-art video salient object detection methods.

Aluminum Casting Inspection using Deep Object Detection Methods and Simulated Ellipsoidal Defects

Aluminum Casting Inspection using Deep Object Detection Methods and Simulated Ellipsoidal Defects

A Survey on Deep Learning Based Methods and Datasets for Monocular 3D Object Detection

Owing to recent advancements in deep learning methods and relevant databases, it is becoming increasingly easier to recognize 3D objects using only RGB images from single viewpoints. This study investigates the major breakthroughs and current progress in deep learning-based monocular 3D object detection. For relatively low-cost data acquisition systems without depth sensors or cameras at multiple viewpoints, we first consider existing databases with 2D RGB photos and their relevant attributes. Based on this simple sensor modality for practical applications, deep learning-based monocular 3D object detection methods that overcome significant research challenges are categorized and summarized. We present the key concepts and detailed descriptions of representative single-stage and multiple-stage detection solutions. In addition, we discuss the effectiveness of the detection models on their baseline benchmarks. Finally, we explore several directions for future research on monocular 3D object detection.

Fooling deep neural detection networks with adaptive object-oriented adversarial perturbation

Deep learning has shown superiority in dealing with complicated and professional tasks (e.g., computer vision, audio, and language processing). However, research works have confirmed that Deep Neural Networks (DNNs) are vulnerable to carefully crafted adversarial perturbations, which cause DNNs confusion on specific tasks. In object detection domain, the background has little contributions to object classification, and the crafted adversarial perturbations added to the background do not improve the adversary effect in fooling deep neural detection models yet induce substantial distortions in generated examples. Based on such situation, we introduce an adversarial attack algorithm named Adaptive Object-oriented Adversarial Method (AO2AM). It aims to fool deep neural object detection networks with the adversarial examples by applying the adaptive cumulation of object-based gradients and adding the adaptive object-based adversarial perturbations merely onto objects rather than the whole frame of input images. AO2AM can effectively make the representations of generated adversarial samples close to the decision boundary in the latent space, and force deep neural detection networks to yield inaccurate locations and false classification in the process of object detection. Compared with existing adversarial attack methods which generate adversarial perturbations acting on the global scale of the original inputs, the adversarial examples produced by AO2AM can effectively fool deep neural object detection networks and maintain a high structural similarity with corresponding clean inputs. Performing adversarial attacks on Faster R-CNN, AO2AM gains attack success rate (ASR) over 98.00% on pre-processed Pascal VOC 2007&2012 (Val), and reaches SSIM over 0.870. In Fooling SSD, AO2AM receives SSIM exceeding 0.980 on L2 norm constraint. On SSIM and Mean Attack Ratio, AO2AM outperforms adversarial attack methods based on global scale perturbations.