Conventional Object Detection Research Articles

Flame Detection Using Appearance-Based Pre-Processing and Convolutional Neural Network

It is important for fire detectors to operate quickly in the event of a fire, but existing conventional fire detectors sometimes do not work properly or there are problems where non-fire or false reporting occurs frequently. Therefore, in this study, HSV color conversion and Harris Corner Detection were used in the image pre-processing step to reduce the incidence of false detections. In addition, among the detected corners, the vicinity of the corner point facing the upper direction was extracted as a region of interest (ROI), and the fire was determined using a convolutional neural network (CNN). These methods were designed to detect the appearance of flames based on top-pointing properties, which resulted in higher accuracy and higher precision than when input images were still used in conventional object detection algorithms. This also reduced the false detection rate for non-fires, enabling high-precision fire detection.

Event-Based Pedestrian Detection Using Dynamic Vision Sensors

Pedestrian detection has attracted great research attention in video surveillance, traffic statistics, and especially in autonomous driving. To date, almost all pedestrian detection solutions are derived from conventional framed-based image sensors with limited reaction speed and high data redundancy. Dynamic vision sensor (DVS), which is inspired by biological retinas, efficiently captures the visual information with sparse, asynchronous events rather than dense, synchronous frames. It can eliminate redundant data transmission and avoid motion blur or data leakage in high-speed imaging applications. However, it is usually impractical to directly apply the event streams to conventional object detection algorithms. For this issue, we first propose a novel event-to-frame conversion method by integrating the inherent characteristics of events more efficiently. Moreover, we design an improved feature extraction network that can reuse intermediate features to further reduce the computational effort. We evaluate the performance of our proposed method on a custom dataset containing multiple real-world pedestrian scenes. The results indicate that our proposed method raised its pedestrian detection accuracy by about 5.6–10.8%, and its detection speed is nearly 20% faster than previously reported methods. Furthermore, it can achieve a processing speed of about 26 FPS and an AP of 87.43% when implanted on a single CPU so that it fully meets the requirement of real-time detection.

Application of conventional UAV-based high-throughput object detection to the early diagnosis of pine wilt disease by deep learning

The early diagnosis of pine wilt disease (PWD) is crucial to its management. Substantial effort has been made to develop an accurate early diagnosis method. However, none of the existing methods are suitable for large-scale rapid screening in the field. In this study, an unmanned aerial vehicle (UAV) was used to collect a large number of images from a pine tree canopy in an early stage of infection for the generation of a training dataset. Owing to the inability to develop object detection models using nine regular machine learning classifiers, two advanced deep learning algorithms were employed, namely the Faster Region-based Convolutional Network (Faster R-CNN) and You Only Look Once version 3 (YOLOv3). Model performances were compared based on the precision (mAP), size, and processing speed. All four models possessed similar precision (0.602–0.64), but the YOLO-based models had a smaller size and faster processing speed than the Faster R-CNN-adapted models. The population of infected trees (in the early or late stage of infection) was predicted under different treatments (retaining or removing the dead trees from the forest) using these four models to explore their application. The results indicate that retaining dead trees after chopping them down results in fewer dead trees but more early infected trees the following year. We propose a cost-effective and high-throughput method for the early diagnosis of PWD in the field using UAV-based image processing and object detection (uses YOLOv3) based on deep learning.

Few-Shot Object Detection via Sample Processing

Few-shot object detection (FSOD) eliminates the dependence on tremendous instances with manual annotations in conventional object detection. We deem that the scarcity of positive samples is the main reason that restricts the performance of FSOD detectors. In this paper, a novel FSOD model via sample processing, namely, FSSP, is proposed to detect objects accurately with only a few annotated samples, which is based on the structural design of the Siamese network and uses YOLOv3-SPP as the baseline. Central to FSSP are our designed self-attention (SAM) and positive-sample augmentation (PSA) modules. The former attempts to better extract the representative features of hard samples, and the latter expands the number and enriches the scale distribution of positive samples, inhibiting the growth of negative samples. For the fine-tuning phase, we modify the classification loss function to increase the punishment for hard samples. Experiments conducted on the PASCAL VOC and MS COCO datasets confirm that the proposed FSSP achieves competitive detection performance compared with state-of-the-art detectors.

Instance-Level Relative Saliency Ranking With Graph Reasoning.

Conventional salient object detection models cannot differentiate the importance of different salient objects. Recently, two works have been proposed to detect saliency ranking by assigning different degrees of saliency to different objects. However, one of these models cannot differentiate object instances and the other focuses more on sequential attention shift order inference. In this paper, we investigate a practical problem setting that requires simultaneously segment salient instances and infer their relative saliency rank order. We present a novel unified model as the first end-to-end solution, where an improved Mask R-CNN is first used to segment salient instances and a saliency ranking branch is then added to infer the relative saliency. For relative saliency ranking, we build a new graph reasoning module by combining four graphs to incorporate the instance interaction relation, local contrast, global contrast, and a high-level semantic prior, respectively. A novel loss function is also proposed to effectively train the saliency ranking branch. Besides, a new dataset and an evaluation metric are proposed for this task, aiming at pushing forward this field of research. Finally, experimental results demonstrate that our proposed model is more effective than previous methods. We also show an example of its practical usage on adaptive image retargeting.

RGB-D Salient Object Detection With Ubiquitous Target Awareness.

Conventional RGB-D salient object detection methods aim to leverage depth as complementary information to find the salient regions in both modalities. However, the salient object detection results heavily rely on the quality of captured depth data which sometimes are unavailable. In this work, we make the first attempt to solve the RGB-D salient object detection problem with a novel depth-awareness framework. This framework only relies on RGB data in the testing phase, utilizing captured depth data as supervision for representation learning. To construct our framework as well as achieving accurate salient detection results, we propose a Ubiquitous Target Awareness (UTA) network to solve three important challenges in RGB-D SOD task: 1) a depth awareness module to excavate depth information and to mine ambiguous regions via adaptive depth-error weights, 2) a spatial-aware cross-modal interaction and a channel-aware cross-level interaction, exploiting the low-level boundary cues and amplifying high-level salient channels, and 3) a gated multi-scale predictor module to perceive the object saliency in different contextual scales. Besides its high performance, our proposed UTA network is depth-free for inference and runs in real-time with 43 FPS. Experimental evidence demonstrates that our proposed network not only surpasses the state-of-the-art methods on five public RGB-D SOD benchmarks by a large margin, but also verifies its extensibility on five public RGB SOD benchmarks.

Object Detection-Tracking Algorithm for Unmanned Surface Vehicles Based on a Radar-Photoelectric System

Object tracking is an important basis for the autonomous navigation of unmanned surface vehicles. However, several problems still must be addressed for a wide applicating of object tracking in unmanned surface vehicles. First, if multiple objects of the same classification exist in the same field of view, then stable extraction of an object is difficult. Second, in an environment with a complex background and large changes in object shape, the tracking accuracy is low, and object tracking errors and tracking loss can easily occur. Third, much time is required to detect a high-resolution real-time video stream, not meeting the delay requirement of the photoelectric servo stable tracking. To resolve these problems, this paper proposes an object detection-tracking algorithm based on a radar-photoelectric system. The algorithm combines an object detection algorithm with an object tracking algorithm and involves the following steps. First, a first-frame object extraction algorithm is used to extract the tracking object from the first frame. Second, a region of interest (ROI)-prediction algorithm is used to predict ROIs and detect objects in these ROIs. This algorithm can effectively solve the above problems in marine tests. When multiple objects of the same classification exist in the same field of view, the algorithm can extract the radar-guided object stably. When faced with a complex background and a large change in object shape, the algorithm substantially improves the accuracy and robustness of object tracking. Compared with the conventional object detection algorithm, the time consumption of this algorithm is reduced by 25.8%.

TasselNetV2+: A Fast Implementation for High-Throughput Plant Counting From High-Resolution RGB Imagery.

Plant counting runs through almost every stage of agricultural production from seed breeding, germination, cultivation, fertilization, pollination to yield estimation, and harvesting. With the prevalence of digital cameras, graphics processing units and deep learning-based computer vision technology, plant counting has gradually shifted from traditional manual observation to vision-based automated solutions. One of popular solutions is a state-of-the-art object detection technique called Faster R-CNN where plant counts can be estimated from the number of bounding boxes detected. It has become a standard configuration for many plant counting systems in plant phenotyping. Faster R-CNN, however, is expensive in computation, particularly when dealing with high-resolution images. Unfortunately high-resolution imagery is frequently used in modern plant phenotyping platforms such as unmanned aerial vehicles, engendering inefficient image analysis. Such inefficiency largely limits the throughput of a phenotyping system. The goal of this work hence is to provide an effective and efficient tool for high-throughput plant counting from high-resolution RGB imagery. In contrast to conventional object detection, we encourage another promising paradigm termed object counting where plant counts are directly regressed from images, without detecting bounding boxes. In this work, by profiling the computational bottleneck, we implement a fast version of a state-of-the-art plant counting model TasselNetV2 with several minor yet effective modifications. We also provide insights why these modifications make sense. This fast version, TasselNetV2+, runs an order of magnitude faster than TasselNetV2, achieving around 30 fps on image resolution of 1980 × 1080, while it still retains the same level of counting accuracy. We validate its effectiveness on three plant counting tasks, including wheat ears counting, maize tassels counting, and sorghum heads counting. To encourage the use of this tool, our implementation has been made available online at https://tinyurl.com/TasselNetV2plus.

Improved salient object detection using hybrid Convolution Recurrent Neural Network

Salient object detection is a critical and active field that aims at the detection of objects in a video, however, it draws increased attention among researchers. With increasing dynamic video data, the performance of saliency object detection method has been degrading with conventional object detection methods. The challenges lie with blurry moving targets, rapid movement of objects and background occlusion or dynamic background change on foreground regions in video frames. Such challenges result in poor saliency detection. In this paper, we design a deep learning model to address the issues, which uses a novel framework by combining the idea of Convolutional Neural Network (CNN) with Recurrent Neural Network (RNN) for video saliency detection. The proposed method aims at developing a spatiotemporal model that exploits temporal, spatial and local constraint cues to achieve global optimization. The task of finding the salient objects in benchmark dynamic video datasets is then carried out by capturing the temporal, spatial and local constraint features with the Convolution Recurrent Neural Network (CRNN). The CRNN is evaluated on benchmark datasets against conventional video salient object detection methods in terms of precision, F-measure, mean absolute error (MAE) and computational load. The experiments reveal that the CRNN model achieves improved performance than other state-of-the-art saliency models in terms of increased speed and reduced computational load.

A New Approach for Navigation and Traffic Signs Indication Using Map Integrated Augmented Reality for Self-Driving Cars

Self-driving vehicles are one of the emerging technologies. This technology has potential to save lives and make lives comfortable. However, the technology used in self driving cars has to perform series of task for building perceptions. This has some certain prerequisites related to road infrastructure and is affected by daylight and weather conditions of the place. If these prerequisites are not satisfied then it could affect the performance of the vehicle and can be considered as compromise with safety of the users. This research work is focused on trying to find a new approach using which the underdeveloped countries will also be able to implement self driving cars in their county. The objective of this paper is to propose a new approach to supplement the technology used in the self-driving cars for perception. Using this approach the countries who don’t satisfy the prerequisites would be eligible to implement them without compromising the safety.The proposed approach uses the technology Augmented Reality to create and augment artificial objects of navigational signs and traffic signals based on vehicles location to reality. Later the augmented scene is fed into the conventional Deep learning object detection algorithm to detect the navigational artificial objected along with other real objects. This approach help navigate the vehicle even if the road infrastructure does not have very good sign indications and marking.The approach was tested locally by creating a local navigational system and a smartphone based augmented reality app. The approach performed better than the conventional method as the objects were clearer in the frame which made it each for the object detection to detect them.

An Efficient Lens Structures Segmentation Method on AS-OCT Images.

Lens structures segmentation on anterior segment optical coherence tomography (AS-OCT) images is a fundamental task for cataract grading analysis. In this paper, in order to reduce the computational cost while keeping the segmentation accuracy, we propose an efficient segmentation method for lens structures segmentation. At first, we adopt an efficient semantic segmentation network in the work, and used it to extract the lens area image instead of the conventional object detection method, and then used it once again to segment the lens structures. Finally, we introduce the curve fitting processing (CFP) on the segmentation results. Experiment results show that our method has good performance on accuracy and processing speed, and could be applied to CASIA II device for practical applications.

Optic Disc Segmentation in Fundus Images with Deep Learning Object Detector

The Optic Disc (OD) is an important anatomical landmark in the fundus image to diagnose a myriad of diseases, such as glaucoma and Diabetic Retinopathy (DR) and to locate structures such as the macula and the main vascular arcade. However, locating and segmenting the OD are not easy tasks. Previous methods have employed a deep Convolutional Neural Network (CNN) without any need for hand-crafted features. Among these methods, RetinaNet has recently attracted attention as a simple one-stage object detector that performs quickly and efficiently while achieving state-of-the-art results. RetinaNet has proven its efficiency in multiple conventional object detection tasks with a larger training set that contains a sufficient number of diverse cases which are beyond reach in medical tasks. Thus, we propose an OD segmentation model from fundus images based on RetinaNet extension with DenseNet that addresses the vanishing gradient problem, enhances feature propagation, performs deep supervision, strengthens feature reuse and reduces the number of parameters. The experimental results using three publicly available databases show the efficacy of deep object detection network and the dense connectivity when applied to fundus images, which is a promising step in providing a segmentation to detect patients in the early stages of the disease.

A novel deformable body partition model for MMW suspicious object detection and dynamic tracking

In recent years, millimeter wave (MMW) imaging techniques have developed rapidly and been widely used in public security field. Traditional security techniques on MMW images mostly based on manual analysis and simple image processing, which cannot achieve real-time and accurate performance. This paper proposes a deep learning-based deformable body partition (DBP) model for detecting suspicious objects hidden in human body on MMW image dataset. Considering the characteristics of MMW security images, we combine the location information of concealed object with the corresponding human body part for object detection. According to the human body structure, 17 deformable regions are partitioned with 12 body keypoints to address body region misalignment problems of different people. By training convolutional neural network (CNN) with different body regions, the suspicious objects can be detected, together with their location information. To make full use of spatio-temporal context information of MMW image sequences, we design an object dynamic tracking method to lock the concealed targets moving from different angles’ MMW images of the same still body. Compared with conventional global-based object detection methods, the proposed method can not only accurately detect suspicious objects but also output the location information of them. Importantly, DBP model is more adaptive than the fixed coordinate-based body partition method and can automatically change the size of each body region according to each person’s body shape. Moreover, our object dynamic tracking method can utilize the positional relationship of suspicious objects in image sequences to reduce the object searching area of each image. Experimental results prove the effectiveness of the proposed method. The detection accuracy and speed on MMW images are favorable in practical application.

Object Detection in Optical Remote Sensing Images Based on Residual Network

Conventional object detection algorithm based on deep learning only make use of deep feature which would become indiscriminative for small target in optical remote sensing images when the network deepens. In this paper, a multi-layer feature fusion method based on residual learning is proposed, which combine the shallow feature with deep feature to classify object comprehensively. First, a ResNet50 backbone network is constructed to extract feature from multiple layers. Then, the scales of feature at different layers are unified through RoI Pooling layer to screen region proposals and be classified through the SVM classifier. Comparative experiments are conducted on UCAS_AOD dataset released by UCAS and the result shows that our model achieved relatively good performance with 0.8802 and 0.9120 f1 score in car and airplane categories respectively.

Geometry-Aware Cell Detection with Deep Learning.

Analyzing cells and tissues under a microscope is a cornerstone of biological research and clinical practice. However, the challenge faced by conventional microscopy image analysis is the fact that cell recognition through a microscope is still time-consuming and lacks both accuracy and consistency. Despite enormous progress in computer-aided microscopy cell detection, especially with recent deep-learning-based techniques, it is still difficult to translate an established method directly to a new cell target without extensive modification. The morphology of a cell is complex and highly varied, but it has long been known that cells show a nonrandom geometrical order in which a distinct and defined shape can be formed in a given type of cell. Thus, we have proposed a geometry-aware deep-learning method, geometric-feature spectrum ExtremeNet (GFS-ExtremeNet), for cell detection. GFS-ExtremeNet is built on the framework of ExtremeNet with a collection of geometric features, resulting in the accurate detection of any given cell target. We obtained promising detection results with microscopic images of publicly available mammalian cell nuclei and newly collected protozoa, whose cell shapes and sizes varied. Even more striking, our method was able to detect unicellular parasites within red blood cells without misdiagnosis of each other.IMPORTANCE Automated diagnostic microscopy powered by deep learning is useful, particularly in rural areas. However, there is no general method for object detection of different cells. In this study, we developed GFS-ExtremeNet, a geometry-aware deep-learning method which is based on the detection of four extreme key points for each object (topmost, bottommost, rightmost, and leftmost) and its center point. A postprocessing step, namely, adjacency spectrum, was employed to measure whether the distances between the key points were below a certain threshold for a particular cell candidate. Our newly proposed geometry-aware deep-learning method outperformed other conventional object detection methods and could be applied to any type of cell with a certain geometrical order. Our GFS-ExtremeNet approach opens a new window for the development of an automated cell detection system.

Detecting 6D Poses of Target Objects From Cluttered Scenes by Learning to Align the Point Cloud Patches With the CAD Models

6D target object detection is of great importance to many applications such as robotics, industrial automation, and unmanned vehicles and is increasingly influencing broad industries including manufacturing, transportation, and retail industries, to name a few. This paper focuses on detecting the 6D poses of the target objects from the point cloud of a cluttered scene. However, conventional point cloud-based 6D object detection methods rely on the robustness of key-point detection results that are not straightforward for humans to understand. The drawback makes conventional point cloud-based methods require expert knowledge to tune. In this paper, we introduced a 6D target object detection method that uses segmented object point cloud patches instead of key points to predict object 6D poses and identity. Our main contributions are an end-to-end data-driven pose correction model that is enhanced with a novel simple yet efficient basis spanning layer booster. Experiments show that although the proposed model is trained only using object CAD models, its 6D detection performance matches that of the models using view data. Thus, the proposed method is suitable for 6D detection applications that have object CAD models instead of labeled scene data.

BMF-CNN: an object detection method based on multi-scale feature fusion in VHR remote sensing images

ABSTRACTObject detection in very-high-resolution (VHR) remote sensing images is one of the important technical means in many fields. In recent years, conventional object detection methods have been completely replaced by Convolutional Neural Network (CNN)-based methods, which are more accurate and efficient. However, most current CNN-based methods applied in VHR image sets have certain defects: (1) Scale preference is common in the framework designs, and the representation ability of feature maps for large and small objects is quite different, so accuracy promotion can hardly be made comprehensively in the detection of different objects. (2) The scale difference of the objects leads to training difficulties. (3) Some high-precision methods require high hardware costs, and the overall frameworks lack practicality. To address such problems, we propose a new object detection method in this paper, namely Balanced Multi-Scale Fusion-based CNN (BMF-CNN). It is a redesigned two-stage detection framework according to the region-based object detection methods, which enabled the detection accuracy of both large and small objects to reach a high level. Through the evaluation in the open VHR remote sensing image sets, we found that BMF-CNN showed a better integrative performance than the current mainstream detection methods.

A novel hardware-oriented ultra-high-speed object detection algorithm based on convolutional neural network

This paper describes a hardware-oriented two-stage algorithm that can be deployed in a resource-limited field-programmable gate array (FPGA) for fast-object detection and recognition with out external memory. The first stage is the bounding boxes proposal with a conventional object detection method, and the second is convolutional neural network (CNN)-based classification for accuracy improvement. Frequently accessing external memories significantly affects the execution efficiency of object classification. Unfortunately, the existing CNN models with a large number of parameters are difficult to deploy in FPGAs with limited on-chip memory resources. In this study, we designed a compact CNN model and performed the hardware-oriented quantization for parameters and intermediate results. As a result, CNN-based ultra-fast-object classification was realized with all parameters and intermediate results stored on chip. Several evaluations were performed to demonstrate the performance of the proposed algorithm. The object classification module consumes only 163.67 Kbits of on-chip memories for ten regions of interest (ROIs), this is suitable for low-end FPGA devices. In the aspect of accuracy, our method provides a correctness rate of 98.01% in open-source data set MNIST and over 96.5% in other three self-built data sets, which is distinctly better than conventional ultra-high-speed object detection algorithms.

Glance and Stare: Trapping Flying Birds in Aerial Videos by Adaptive Deep Spatio-Temporal Features

Flying bird detection has recently attracted increasing attention in computer vision, which becomes an urgent task with the opening up of the low-altitude airspace. However, compared to conventional object detection tasks, it is much more challenging to trap flying birds in aerial videos due to small target sizes, complex backgrounds of great variations and disturbances of bird-like objects. In this paper, we propose a unified framework termed glance-and-stare detection (GSD) to trap flying birds in aerial videos. The GSD is inspired by the fact that human beings first glance at the whole image and then stare at the areas where the suspected object is most likely to appear until the confirmation is obtained. Specifically, we propose the zooming-in algorithm to generate region proposals for accurate localization of flying birds; to represent region proposal sequences of different lengths, we propose adaptive deep spatio-temporal features by leveraging the strength of 3D convolutional neural networks, based on which classification is conducted to achieve final detection. In contrast to conventional methods, the GSD enables localization and classification to be conducted jointly in an alternating iterative way, which mutually enhances each other to improve their performance. In order to validate the proposed GSD algorithm, we build flying bird data sets including images and videos, which provide new benchmarks for evaluation of flying bird detection systems. Experiments on the data sets demonstrate that the GSD can achieve high detection accuracy and largely outperform the state-of-the-art detection methods.

Early Detection of Vacant Parking Spaces Using Dashcam Videos

A major problem in metropolitan areas is finding parking spaces. Existing parking guidance systems often adopt fixed sensors or cameras that cannot provide information from the driver’s point of view. Motivated by the advent of dashboard cameras (dashcams), we develop neural-network-based methods for detecting vacant parking spaces in videos recorded by a dashcam. Detecting vacant parking spaces in dashcam videos enables early detection of spaces. Different from conventional object detection methods, we leverage the monotonicity of the detection confidence with respect to the distance away of the approaching target parking space and propose a new loss function, which can not only yield improved detection results but also enable early detection. To evaluate our detection method, we create a new large dataset containing 5,800 dashcam videos captured from 22 indoor and outdoor parking lots. To the best of our knowledge, this is the first and largest driver’s view video dataset that supports parking space detection and provides parking space occupancy annotations.