Deep Learning Object Detection Methods Research Articles

Fish age reading using deep learning methods for object-detection and segmentation

Abstract Determination of individual age is one essential step in the accurate assessment of fish stocks. In non-tropical environments, the manual count of ring-like growth patterns in fish otoliths (ear stones) is the standard method. It relies on visual means and individual judgment and thus is subject to bias and interpretation errors. The use of automated pattern recognition based on machine learning may help to overcome this problem. Here, we employ two deep learning methods based on Convolutional Neural Networks (CNNs). The first approach utilizes the Mask R-CNN algorithm to perform object detection on the major otolith reading axes. The second approach employs the U-Net architecture to perform semantic segmentation on the otolith image in order to segregate the regions of interest. For both methods, we applied a simple postprocessing to count the rings on the output masks returned, which corresponds to the age prediction. Multiple benchmark tests indicate the promising performance of our implemented approaches, comparable to recently published methods based on classical image processing and traditional CNN implementation. Furthermore, our algorithms showed higher robustness compared to the existing methods, while also having the capacity to extrapolate missing age groups and to adapt to a new domain or data source.

Sensor-based characterization of construction and demolition waste at high occupancy densities using synthetic training data and deep learning.

Sensor-based monitoring of construction and demolition waste (CDW) streams plays an important role in recycling (RC). Extracted knowledge about the composition of a material stream helps identifying RC paths, optimizing processing plants and form the basis for sorting. To enable economical use, it is necessary to ensure robust detection of individual objects even with high material throughput. Conventional algorithms struggle with resulting high occupancy densities and object overlap, making deep learning object detection methods more promising. In this study, different deep learning architectures for object detection (Region-based CNN/Region-based Convolutional Neural Network (Faster R-CNN), You only look once (YOLOv3), Single Shot MultiBox Detector (SSD)) are investigated with respect to their suitability for CDW characterization. A mixture of brick and sand-lime brick is considered as an exemplary waste stream. Particular attention is paid to detection performance with increasing occupancy density and particle overlap. A method for the generation of synthetic training images is presented, which avoids time-consuming manual labelling. By testing the models trained on synthetic data on real images, the success of the method is demonstrated. Requirements for synthetic training data composition, potential improvements and simplifications of different architecture approaches are discussed based on the characteristic of the detection task. In addition, the required inference time of the presented models is investigated to ensure their suitability for use under real-time conditions.

A deep learning object detection method to improve cluster analysis of two-dimensional data

A deep learning object detection method to improve cluster analysis of two-dimensional data

Vegetation and Non-Vegetation Classification Using Object Detection Techniques and Deep Learning from Low/Mixed Resolution Satellite Images

Vegetation cover classification using mixed or low-resolution scalar images is challenging. Fortunately, recently deep learning object detection methods have emerged as a replacement to the conventional machine learning methods for the detection and classification of land use and land cover. This paper presents a deep learning object detection approach for land use and land cover detection using low/mixed resolution satellite images acquired from Google Earth satellite images. Google Earth images are accessible freely using the Google Earth Pro desktop application. Our dataset consists of two (02) classes (vegetation and non-vegetation) with a total of 450 labeled images captured from different parts of Pakistan. We present a comparison of the recent anchor-free object detection model YOLOX with the anchor-based object detection model YOLOR for solving real-time problems. The end-to-end differentiability, efficient GPU utilization, and absence of hand-crafted parameters make anchor-free models a compelling choice in object detection, and yet not been explored on Land cover classification using satellite images. Our experimental study shows that YOLOX delivers an overall accuracy of 83.50% on Vegetation and 86% on Non-Vegetation classes, which outperformed YOLOR by 30% on Vegetation classes and 34% on non-Vegetation classes for our dataset. We also show how an object detection system can be used for Vegetation and Non-Vegetation classification tasks, which can then be used for change monitoring and assisting in developing geographical maps using low/mixed resolution freely available satellite images.

An improved deep learning-based optimal object detection system from images

Computer vision technology for detecting objects in a complex environment often includes other key technologies, including pattern recognition, artificial intelligence, and digital image processing. It has been shown that Fast Convolutional Neural Networks (CNNs) with You Only Look Once (YOLO) is optimal for differentiating similar objects, constant motion, and low image quality. The proposed study aims to resolve these issues by implementing three different object detection algorithms—You Only Look Once (YOLO), Single Stage Detector (SSD), and Faster Region-Based Convolutional Neural Networks (R-CNN). This paper compares three different deep-learning object detection methods to find the best possible combination of feature and accuracy. The R-CNN object detection techniques are performed better than single-stage detectors like Yolo (You Only Look Once) and Single Shot Detector (SSD) in term of accuracy, recall, precision and loss.

PLATE: A perception-latency aware estimator

Target tracking is a popular problem with many potential applications. There has been a lot of effort on improving the quality of the detection of targets using cameras through different techniques. In general, with higher computational effort applied, i.e., a longer perception-latency, a better detection accuracy is obtained. However, it is not always useful to apply the longest perception-latency allowed, particularly when the environment does not require to and when the computational resources are shared between other tasks. In this work, we propose a new Perception-LATency aware Estimator (PLATE), which uses different perception configurations in different moments of time in order to optimize a certain performance measure. This measure takes into account a perception-latency and accuracy trade-off aiming for a good compromise between quality and resource usage. Compared to other heuristic frame-skipping techniques, PLATE comes with a formal complexity and optimality analysis. The advantages of PLATE are verified by several experiments including an evaluation over a standard benchmark with real data and using state of the art deep learning object detection methods for the perception stage.

Few-Shot Object Detection in Remote Sensing Imagery via Fuse Context Dependencies and Global Features

The rapid development of Earth observation technology has promoted the continuous accumulation of images in the field of remote sensing. However, a large number of remote sensing images still lack manual annotations of objects, which makes the strongly supervised deep learning object detection method not widely used, as it lacks generalization ability for unseen object categories. Considering the above problems, this study proposes a few-shot remote sensing image object detection method that integrates context dependencies and global features. The method can be used to fine-tune the model with a small number of sample annotations based on the model trained in the base class, as a way to enhance the detection capability of new object classes. The method proposed in this study consists of three main modules, namely, the meta-feature extractor (ME), reweighting module (RM), and feature fusion module (FFM). These three modules are respectively used to enhance the context dependencies of the query set features, improve the global features of the support set that contains annotations, and finally fuse the query set features and support set features. The baseline of the meta-feature extractor of the entire framework is based on the optimized YOLOv5 framework. The reweighting module of the support set feature extraction is based on a simple convolutional neural network (CNN) framework, and the foreground feature enhancement of the support sets was made in the preprocessing stage. This study achieved beneficial results in the two benchmark datasets NWPU VHR-10 and DIOR. Compared with the comparison methods, the proposed method achieved the best performance in the object detection of the base class and the novel class.

Deep learning in detecting tumor spread through air spaces

Lung cancer is a leading cause of death on a global scale. Professor I-Fang Chung leads a team of researchers at the Institute of Biomedical Informatics, National Yang Ming Chiao Tung University, Taiwan, investigating how machine learning can help predict the risk of tumour recurrence following lung cancer surgery. The standard treatment for early-stage lung cancer patients is complete surgical resection of the tumour but disease recurrence within the first five years following surgery is common. The route of this recurrence is usually tumour spread through the air spaces (STAS) and the presence of STAS has recently been identified as a risk factor for recurrence of tumours. The researchers are exploring the importance of STAS and how targeting this phenomenon and using machine learning methods to aid in the analysis of medical imaging of diseased tissues can help in predicting the recurrence risk of post-surgery lung cancer patients. Dr Yi-Chen Yeh, a pathologist from the Taipei Veterans General Hospital, is working with Chung to employ a variety of deep-learning object detection methodologies to detect STAS in pathology images. Regions of interest (ROI) images extracted from pathology whole slide images (WSI) are marked and annotated, which provides location information for STAS. Deep learning object detection methods are used to train a model which can find STAS, then additional techniques including using pre-trained model parameters, augmenting random image data and modifying loss function are used to improve the detection rates for the model. Their model helps pathologists identify STAS and accurately predicts patient outcomes.

Localizing Post-Admixture Adaptive Variants with Object Detection on Ancestry-Painted Chromosomes.

Gene flow between previously differentiated populations during the founding of an admixed or hybrid population has the potential to introduce adaptive alleles into the new population. If the adaptive allele is common in one source population, but not the other, then as the adaptive allele rises in frequency in the admixed population, genetic ancestry from the source containing the adaptive allele will increase nearby as well. Patterns of genetic ancestry have therefore been used to identify post-admixture positive selection in humans and other animals, including examples in immunity, metabolism, and animal coloration. A common method identifies regions of the genome that have local ancestry "outliers" compared with the distribution across the rest of the genome, considering each locus independently. However, we lack theoretical models for expected distributions of ancestry under various demographic scenarios, resulting in potential false positives and false negatives. Further, ancestry patterns between distant sites are often not independent. As a result, current methods tend to infer wide genomic regions containing many genes as under selection, limiting biological interpretation. Instead, we develop a deep learning object detection method applied to images generated from local ancestry-painted genomes. This approach preserves information from the surrounding genomic context and avoids potential pitfalls of user-defined summary statistics. We find the method is robust to a variety of demographic misspecifications using simulated data. Applied to human genotype data from Cabo Verde, we localize a known adaptive locus to a single narrow region compared with multiple or long windows obtained using two other ancestry-based methods.

Anchor-free Proposal Generation Network for Efficient Object Detection

Deep learning object detection methods are usually based on anchor-free or anchor-based scheme for extracting object proposals and one-stage or two-stage structure for producing final predictions. As each scheme or structure has its own strength and weakness, combining their strength in a unified framework is an interesting research topic. However, this topic has not attracted much attention in recent years. This paper presents a two-stage object detection method that utilizes an anchor-free scheme for generating object proposals in the initial stage. For proposal generation, this paper employs an efficient anchor-free network for predicting object corners and assigns object proposals based on detected corners. For object prediction, an efficient detection network is designed to enhance both detection accuracy and speed. The detection network includes a lightweight binary classification subnetwork for removing most false positive object candidates and a light-head detection subnetwork for generating final predictions. Experimental results on the MS-COCO dataset demonstrate that the proposed method outperforms both anchor-free and two-stage object detection baselines in terms of detection performance.

Detection and classification of diseased pine trees with different levels of severity from UAV remote sensing images

The accurate detection and classification of diseased pine trees with different levels of severity is important in terms of monitoring the growth of these trees and for preventing and controlling disease within pine forests. Our method combines a DDYOLOv5 with a ResNet50 network for detecting and classifying levels of pine tree disease from remote sensing UAV images. In this approach, images are preprocessed to increase the background diversity of the training samples, and efficient channel attention (ECA) and hybrid dilated convolution (HDC) modules are introduced to DDYOLOv5 to improve the detection accuracy. The ECA modules enable the network to focus on the characteristics of diseased pine trees, and solve the problem of low detection accuracy caused by the similarities in color and texture between diseased pine trees and the complex backgrounds. The HDC modules capture the contextual information of targets at different scales; they increase the receptive field to focus on targets of different sizes, and address the difficulty of detection caused by large variations in the shapes and sizes of diseased pine trees. In addition, a low confidence threshold is adopted to reduce missed detections and a ResNet50 classification network is applied to classify the detection results into different levels of severity, in order to reduce the number of false detections and improve the classification accuracy. Our experimental results show that the proposed method improves the precision by 13.55%, the recall by 5.06% and the F1-score by 9.71% on 8 test images compared with YOLOv5. Moreover, the detection and classification results from our approach show that it outperforms classical deep learning object detection methods such as Faster R-CNN and RetinaNet.

CrossGAN-Detection: A generative adversarial network with directly controllable fusion for target detection

Most of the deep learning object detection methods based on multi-modal information fusion cannot directly control the quality of the fused images at present, because the fusion only depends on the detection results. The indirectness of control is not conducive to the target detection of the network in principle. For the sake of the problem, we propose a multimodal information cross-fusion detection method based on a generative adversarial network (CrossGAN-Detection), which is composed of GAN and a target detection network. And the target detection network acts as the second discriminator of GAN during training. Through the content loss function and dual discriminator, directly controllable guidance is provided for the generator, which is designed to learn the relationship between different modes adaptively through cross fusion. We conduct abundant experiments on the KITTI dataset, which is the prevalent dataset in the fusion-detection field. The experimental results show that the AP of the novel method for vehicle detection achieves 96.66%, 87.15%, and 78.46% in easy, moderate, and hard categories respectively, which is improved about 7% compared to the state-of-art methods.

Comparison between convolutional neural network and K-nearest neighbours object detection for autonomous drone

In autonomous drones, the drone’s ability to move depends on the drone’s capacity to know its position, either in relative or absolute position. The Pinhole model is one of the methods to calculate a drone’s relative position based on the triangle similarity concept using a single camera. This method utilizes bounding box information generated from an object detection algorithm. Thus, accuracy of the generated bounding box is crucial, and selection of object detection algorithm is necessary. This paper compares and evaluates machine learning and deep learning object detection methods to determine which method is suitable for distance measurement using a single camera for autonomous drone’s controller based on pinhole model. A novel K-nearest neighbours-based (KNN-based) object detection is constructed to represent the machine learning method while you only look once version 5 (YOLOv5) convolutional neural network (CNN) architecture is selected to represent the deep learning method. A dataset consists of two different classes, with a total of 1520 images, collected from the unmanned aerial vehicle (UAV) camera for training and evaluation purposes. Confusion matrix and intersection over union (IoU)/generalized intersection of union (GIoU) matrix are used to evaluate the performance of both methods. The result of this paper shows the performance of each system and concludes the suitable type of object detection algorithm for the autonomous UAV purpose.

Detection and Recognition of Pollen Grains in Multilabel Microscopic Images.

Analysis of pollen material obtained from the Hirst-type apparatus, which is a tedious and labor-intensive process, is usually performed by hand under a microscope by specialists in palynology. This research evaluated the automatic analysis of pollen material performed based on digital microscopic photos. A deep neural network called YOLO was used to analyze microscopic images containing the reference grains of three taxa typical of Central and Eastern Europe. YOLO networks perform recognition and detection; hence, there is no need to segment the image before classification. The obtained results were compared to other deep learning object detection methods, i.e., Faster R-CNN and RetinaNet. YOLO outperformed the other methods, as it gave the mean average precision (mAP@.5:.95) between 86.8% and 92.4% for the test sets included in the study. Among the difficulties related to the correct classification of the research material, the following should be noted: significant similarities of the grains of the analyzed taxa, the possibility of their simultaneous occurrence in one image, and mutual overlapping of objects.

Visual object detection for autonomous transport vehicles in smart factories

Autonomous transport vehicles (ATVs) are one of the most substantial components of smart factories of Industry 4.0. They are primarily considered to transfer the goods or perform some certain navigation tasks in the factory with self driving. The recent developments on computer vision studies allow the vehicles to visually perceive the environment and the objects in the environment. There are numerous applications especially for smart traffic networks in outdoor environments but there is lack of application and databases for autonomous transport vehicles in indoor industrial environments. There exist some essential safety and direction signs in smart factories and these signs have an important place in safety issues. Therefore, the detection of these signs by ATVs is crucial. In this study, a visual dataset which includes important indoor safety signs to simulate a factory environment is created. The dataset has been used to train different fast-responding popular deep learning object detection methods: faster R-CNN, YOLOv3, YOLOv4, SSD, and RetinaNet. These methods can be executed in real time to enhance the visual understanding of the ATV, which, in turn, helps the agent to navigate in a safe and reliable state in smart factories. The trained network models were compared in terms of accuracy on our created dataset, and YOLOv4 achieved the best performance among all the tested methods.

Gastric polyp detection in gastroscopic images using deep neural network.

This paper presents the research results of detecting gastric polyps with deep learning object detection method in gastroscopic images. Gastric polyps have various sizes. The difficulty of polyp detection is that small polyps are difficult to detect from the background. We propose a feature extraction and fusion module and combine it with the YOLOv3 network to form our network. This method performs better than other methods in the detection of small polyps because it can fuse the semantic information of high-level feature maps with low-level feature maps to help small polyps detection. In this work, we use a dataset of gastric polyps created by ourselves, containing 1433 training images and 508 validation images. We train and validate our network on our dataset. In comparison with other methods of polyps detection, our method has a significant improvement in precision, recall rate, F1, and F2 score. The precision, recall rate, F1 score, and F2 score of our method can achieve 91.6%, 86.2%, 88.8%, and 87.2%.

An Improved Object Detection Method for Automatic Electrical Equipment Defect Detection

Employing computer vision and machine learning on equipment defect detection has become an important trend of electric power inspection. This paper presents a new electrical equipment default detection method based on improved YOLOv3. By combining abundant geometric measures, Complete IOU(CIOU) make the bounding box regression during NMS more accurate. Focal loss function, which focus on differentiate between easy and hard examples, is employed to deal with the class imbalance problem. Experiment results show that the proposed approach obtains competitive performance compared with state-of-the-art deep learning object detection methods.

YOLOrs: Object Detection in Multimodal Remote Sensing Imagery

Deep-learning object detection methods that are designed for computer vision applications tend to underperform when applied to remote sensing data. This is because contrary to computer vision, in remote sensing, training data are harder to collect and targets can be very small, occupying only a few pixels in the entire image, and exhibit arbitrary perspective transformations. Detection performance can improve by fusing data from multiple remote sensing modalities, including red, green, blue, infrared, hyperspectral, multispectral, synthetic aperture radar, and light detection and ranging, to name a few. In this article, we propose YOLOrs: a new convolutional neural network, specifically designed for real-time object detection in multimodal remote sensing imagery. YOLOrs can detect objects at multiple scales, with smaller receptive fields to account for small targets, as well as predict target orientations. In addition, YOLOrs introduces a novel mid-level fusion architecture that renders it applicable to multimodal aerial imagery. Our experimental studies compare YOLOrs with contemporary alternatives and corroborate its merits.

CenterNet-Triplets application: surveillance camera illegal management detection

There are two types of illegal business in city management: out-of-store management and illegal mobile management, we simply refer to as outmanagement and flowmagement. The main way to deal with illegal buessiness is manual management. However, this requires a lot of costs, and it is impossible for city management personnel to conduct 24hour inspections. In order to improve management efficiency and save labor costs, this article uses city surveillance cameras, combined with deep learning object detection methods, to conduct real-time detection of illegal operations. Aiming at the situation where the amount of surveillance video data is small, a data enhancement method is proposed to improve the detection effect. CenterNet-Triplets is a object detection algorithm without anchor boxes. Its backbone network is Hourglass Network. The AP of CenterNet-Triplets on MS-COCO dataset is 47.0%, and it has a good detection speed. Experimental results show that CenterNet-Triplets is suitable as a method for detecting illegal operations in city management.

GinJinn: An object-detection pipeline for automated feature extraction from herbarium specimens.

PremiseThe generation of morphological data in evolutionary, taxonomic, and ecological studies of plants using herbarium material has traditionally been a labor‐intensive task. Recent progress in machine learning using deep artificial neural networks (deep learning) for image classification and object detection has facilitated the establishment of a pipeline for the automatic recognition and extraction of relevant structures in images of herbarium specimens.Methods and ResultsWe implemented an extendable pipeline based on state‐of‐the‐art deep‐learning object‐detection methods to collect leaf images from herbarium specimens of two species of the genus Leucanthemum. Using 183 specimens as the training data set, our pipeline extracted one or more intact leaves in 95% of the 61 test images.ConclusionsWe establish GinJinn as a deep‐learning object‐detection tool for the automatic recognition and extraction of individual leaves or other structures from herbarium specimens. Our pipeline offers greater flexibility and a lower entrance barrier than previous image‐processing approaches based on hand‐crafted features.