Learning-based Object Detection Algorithm Research Articles

YOLO-GD: A Deep Learning-Based Object Detection Algorithm for Empty-Dish Recycling Robots

Due to the workforce shortage caused by the declining birth rate and aging population, robotics is one of the solutions to replace humans and overcome this urgent problem. This paper introduces a deep learning-based object detection algorithm for empty-dish recycling robots to automatically recycle dishes in restaurants and canteens, etc. In detail, a lightweight object detection model YOLO-GD (Ghost Net and Depthwise convolution) is proposed for detecting dishes in images such as cups, chopsticks, bowls, towels, etc., and an image processing-based catch point calculation is designed for extracting the catch point coordinates of the different-type dishes. The coordinates are used to recycle the target dishes by controlling the robot arm. Jetson Nano is equipped on the robot as a computer module, and the YOLO-GD model is also quantized by TensorRT for improving the performance. The experimental results demonstrate that the YOLO-GD model is only 1/5 size of the state-of-the-art model YOLOv4, and the mAP of YOLO-GD achieves 97.38%, 3.41% higher than YOLOv4. After quantization, the YOLO-GD model decreases the inference time per image from 207.92 ms to 32.75 ms, and the mAP is 97.42%, which is slightly higher than the model without quantization. Through the proposed image processing method, the catch points of various types of dishes are effectively extracted. The functions of empty-dish recycling are realized and will lead to further development toward practical use.

YOLOv4-CSP 기반 실신 감시 시스템

When swoon occurs, it is important to detect it immediately. If immediate and quick detection is not possible, additional serious accidents may occur. Therefore, this study proposed swoon monitoring system that can detect swoon from real-time images of CCTV based on the increasing number and installation of CCTV. For the detection of swoon, deep learning based object detection algorithm applied with the Cross Stage Partial connections network model was used. Through this, swoon detection is performed within a single frame, and the amount of computation required for swoon detection is reduced and can have a fast processing speed. In addition, it is possible to have high detection performance through various BoF and BoS techniques with CSP network model. The result of swoon detection and classification performance through object detection algorithm showed 94.2% F1-Score, 92.6% mAP and 95.4% Accuracy.

Intrusion Detection of Foreign Objects in Overhead Power System for Preventive Maintenance in High-Speed Railway Catenary Inspection

Foreign object intrusion detection in overhead power systems (OPSs) is critical during preventive maintenance in catenary inspection. However, the OPS images characteristics, including appearance degradation and noisy representations, increase the difficulty of foreign object detection. In this article, a sparse cross attention (SCA) based transformer detector (SCATD) is developed for detecting foreign objects in OPSs. Specifically, a two-stage refinement architecture is proposed for extracting the features of foreign objects in OPS images. In the first stage, a spatiotemporal enhanced (STE) convolutional neural network is proposed to leverage feature-level spatiotemporal coherence across frames to emphasize the spatial responses of foreign objects. Then, a spatial memory based feature aggregation (SMFA) module is constructed to iteratively update the feature affinities during training. Moreover, the SCA network sparsely builds different weak transformer detectors to produce weak predictive results. All the predictive results are fused to achieve the better predictive performance via voting schemes in an adaptive way. Finally, our SCATD is compared with state-of-the-art deep learning based object detection algorithms. Experiments on real OPS dataset demonstrate the high effectiveness of the proposed SCATD scheme in detecting foreign instances in OPSs.

PG-YOLO: A Novel Lightweight Object Detection Method for Edge Devices in Industrial Internet of Things

With the rapid development of Industrial Internet of Things (IIoT) technology, video surveillance devices and video data in IIoT environments are massively growing and increasingly important. Deploying rapidly evolving deep learning-based object detection algorithms in IIoT can improve the efficiency of video data utilization and increase the automation and intelligence of the IIoT. Facing the transmission latency problem of massive video data, the algorithms are better deployed in edge devices. However, due to the large size and high computing power requirements of existing object detection algorithms, it is difficult to deploy them on edge devices with limited performance. Therefore, to solve this problem, we propose a smaller and less computationally intensive object detection algorithm, PG-YOLO. We replaced all the convolutional modules of YOLOV5 with low computational ghost modules and streamlined the backbone network to improve the network performance. Then, we propose a better pruning algorithm to compress the network, and finally improve the accuracy by distillation to obtain PG-YOLO. PG-YOLO is smaller and faster and can be deployed in edge devices. In addition, we conducted experimental validation on the SHWD dataset. The experimental results show that PG-YOLO can compress the volume of the model by 9 times compared with YOLOv5s, and the compressed detection accuracy reaches 0.934, with only 0.1% loss of accuracy. Also, the time required for inference is reduced by 32.7%, and the inference speed is improved by 10 FPS. Compared with other object detection algorithms, PG-YOLO also has advantages.

Multi-operator feature enhancement methods for industrial defect detection

Deep learning based object detection algorithms have been gradually applied to industrial defect detection, but the resulted accuracy does not fully meet the needs of industrial inspection. In order to enhance image features, this paper proposes a series of image preprocessing schemes based on edge detection operators, using a single-operator preprocessing scheme, a multi-operator serial preprocessing scheme and a multi-operator parallel preprocessing scheme for image preprocessing of data to enhance the edge features of images. The validation experiment of the SSD based object detection algorithm is performed on dataset used for industrial inspection, to verify the effectiveness of the processing schemes above. The result shows that the multi-operator based image preprocessing method is effective in improving the accuracy of surface defect detection in the field of industrial defect detection.

Textile Fabric Defect Detection Based on Improved Faster R-CNN

To identify and locate industrial textile defects accurately, this study proposes a textile detection model based on a convolution neural network (CNN) known as Faster R-CNN. First, a textile defect feature map was extracted by ResNet-101 deep convolution network. Faster R-CNN only extracts features from the last layer of the feature map, which leads to a loss of low-level location information. The proposed method adds the feature pyramid network (FPN) to the network architecture to make an independent prediction for each level in the feature extraction stage. The extracted feature map is input into the regional proposal network, among which the overlapping regional proposals are suppressed. The proposed improved Faster R-CNN model with Region Proposal Network (RPN), Soft Non-Maximum Suppression (NMS), and Region of Interest (ROI) Align can achieve a detection accuracy of 98% and an mean of Average Precision (mAP) of 85%, which is more competitive than the state-of-the-art deep learning-based object detection algorithms.

Polar representation-based cell nucleus segmentation in non-small cell lung cancer histopathological images

Image segmentation is a major area of interest within the field of medical image analysis and processing. In the last few decades, cell nucleus segmentation in non-small lung cancer histopathological images has spawned considerable critical attention. However, most of the previously presented studies have only been concerned with revealing the representation of contours in Cartesian coordinates and suffer from a lack of clarity in connecting the points into a whole contour. Bearing this in mind, we propose a polar representation-based nucleus segmentation model by leveraging the fully convolutional one-stage object detection. In general, center classification and length regression are simultaneously adapted to yield the contour of the nucleus in a polar coordinate. To evaluate the performance of our approach, the comparing experiments are conducted between the state-of-the-art deep learning-based object detection algorithms and the proposed method on one manually collected dataset. Experimental results demonstrate that our model outperformed the state-of-the-art both in efficiency and effectiveness.

Tactics Overview for Implementing High-Performance Computing on Embedded Platforms

Future space missions will rely on novel high-performance computing to support advanced intelligent on-board algorithms with substantial workloads that mandates firm real-time and power constraints requirements. Consequently, these advanced algorithms require significantly faster processing beyond the conventional space-grade central processing unit capabilities. Moreover, they require careful selection of the target embedded platform from a diverse set of available architectures along with several implementation tactics to map the algorithms to the target architecture to fully unlock its capabilities. In this paper, we present a study of different architectures and embedded computing platforms for the satellite on-board computers. Moreover, we present a comprehensive overview of recent implementation tactics such as source code mapping and transformations. Additionally, we highlight some optimization techniques such as partitioning and co-designing using hardware accelerators. Finally, we discuss several implementation analysis methodologies to derive optimized code implementations. The top ranked YOLO-v3, as a deep learning based object detection algorithm, is selected as a case study model to be optimized using OpenVINO toolkit. The experimental results show an improvement ratios up to 73%, 41%, and 34% in terms of frames per second, CPU utilization, and cache memory, respectively. The study presented in this paper aims to guide the researchers in the field of high performance embedded computing in terms of different hardware architectures along with several implementation tactics.

Edge computing-based person detection system for top view surveillance: Using CenterNet with transfer learning

Edge computing significantly expands the range of information technology in smart video surveillance applications in the era of intelligent and connected cities. Edge devices, including Internet of Things-based cameras and sensors, produce a large amount of data and have frequently become prominent components for various public surveillance and monitoring applications. The data generated by these smart devices are in the form of videos and images that need to be processed and analyzed in real-time with substantial computation resources. These developed techniques still require large computation resources for real-time surveillance applications. In this regard, Edge computing plays a promising role in order to provide high computation and low-latency requirements. With these motivations, in this work, a real-time top view-based person detection system is presented. We utilize a one-stage deep learning-based object detection algorithm, i.e., CenterNet, for person detection. The model detects the human as a single point, also referred to as its bounding box’s center point. The model does a key-point calculation to obtain the center point and regresses all other information regarding the target object’s features, size, location, and orientation. Training and testing of the model are performed on a top view data set. The detection results are also compared with conventional detection methods using the same data set. The overall detection accuracy of the model is 95%.

Experimental Evaluation of Computer Vision and Machine Learning-Based UAV Detection and Ranging

We consider the problem of vision-based detection and ranging of a target UAV using the video feed from a monocular camera onboard a pursuer UAV. Our previously published work in this area employed a cascade classifier algorithm to locate the target UAV, which was found to perform poorly in complex background scenes. We thus study the replacement of the cascade classifier algorithm with newer machine learning-based object detection algorithms. Five candidate algorithms are implemented and quantitatively tested in terms of their efficiency (measured as frames per second processing rate), accuracy (measured as the root mean squared error between ground truth and detected location), and consistency (measured as mean average precision) in a variety of flight patterns, backgrounds, and test conditions. Assigning relative weights of 20%, 40% and 40% to these three criteria, we find that when flying over a white background, the top three performers are YOLO v2 (76.73 out of 100), Faster RCNN v2 (63.65 out of 100), and Tiny YOLO (59.50 out of 100), while over a realistic background, the top three performers are Faster RCNN v2 (54.35 out of 100, SSD MobileNet v1 (51.68 out of 100) and SSD Inception v2 (50.72 out of 100), leading us to recommend Faster RCNN v2 as the recommended solution. We then provide a roadmap for further work in integrating the object detector into our vision-based UAV tracking system.

DESIGN OF BALL COLLECTING ROBOT

A concept of four-wheeled mobile service robot capable of collecting balls from the Tennis court is proposed in this article. The robot is capable of manoeuvring on its own in an autonomous way and also it can be operated wirelessly. The robot is built using the Arduino Uno R3 microcontroller board along with sensors like LiDAR, ultrasound sensor, optical encoders, infrared sensor, camera for vision system. Sensor fusion can be implemented for localization of the robot while detection of the tennis ball can be attained by learning based object detection algorithm using OpenCV. A* algorithm is proposed for the navigation of the robot towards the object. Wireless operation of the robot can be achieved with the Bluetooth technology using the Android application via a smartphone. The article is an overview and information.

A Deep Learning-Based FPGA Function Block Detection Method With Bitstream to Image Transformation

In the context of various application scenarios and/or for the sake of strengthening field-programmable gate array (FPGA) security, the system functions of an FPGA design need to be analyzed, which can be achieved by systematically partitioning the FPGA's bitstream into manageable functional blocks and detecting their functionalities thereafter. In this paper, we propose a novel deep learning-based FPGA function block detection method with three major steps. In specific, we first analyze the format of the bitstream to obtain the mapping relationship between the configuration bits and configurable logic blocks because of the discontinuity of the configuration bits in the bitstream for one element. In order to reap the maturity of object detection techniques based on deep learning, our next step is to convert an FPGA bitstream to an image, following the proposed transformation method that takes account of both the adjacency nature of the programmable logic and the high degree of redundancy of configuration information. Once the image is obtained, a deep learning-based object detection algorithm is applied to this transformed image, and the objects detected can be reflected back to determine the function blocks of the original FPGA design. The deep neural network used for function block detection is trained and validated with a specially crafted bitstream/image dataset. Experiments have confirmed high detection accuracy of the proposed function detection method, showing a 98.11% of mean Average Precision (IoU=0.5) for 10 function blocks within a YOLOv3 detector implemented on Xilinx Zynq-7000 SoCs and Zynq UltraScale+ MPSoCs.

Arbitrary-angle bounding box based location for object detection in remote sensing image

ABSTRACT Object location is a fundamental yet challenging problem in object detection. In the remote sensing image, different imaging projection directions make the same object have various rotation angles, and in some scenes, the object distribution is relatively dense. Most of the existing deep learning-based object detection algorithms utilize horizontal bounding box to locate objects, which causes inaccurate location of the objects with dense distribution or arbitrary direction, thus leading to the detection misses. In this paper, we propose an arbitrary-angle bounding box based object location and embed it into the Faster R-CNN, developing a new framework called Rotated Faster R-CNN (R-FRCNN) for object detection in remote sensing image. In R-FRCNN, we specially improve anchor ratios to adapt to the objects like ship with large aspect ratio and increase the weights of the horizontal bounding box regression to reduce the interference of the arbitrary-angle bounding box on the horizontal bounding box prediction. Comprehensive experiments on a public dataset and a self-assembled dataset (which we make publically available) show the superior performance of our method compared to standalone state-of-the-art object detectors.

Comparison Analysis and Case Study for Deep Learning-based Object Detection Algorithm

Comparison Analysis and Case Study for Deep Learning-based Object Detection Algorithm

Deep learning-based object detection in low-altitude UAV datasets: A survey

Deep learning-based object detection solutions emerged from computer vision has captivated full attention in recent years. The growing UAV market trends and interest in potential applications such as surveillance, visual navigation, object detection, and sensors-based obstacle avoidance planning have been holding good promises in the area of deep learning. Object detection algorithms implemented in deep learning framework have rapidly became a method for processing of moving images captured from drones. The primary objective of the paper is to provide a comprehensive review of the state of the art deep learning based object detection algorithms and analyze recent contributions of these algorithms to low altitude UAV datasets. The core focus of the studies is low-altitude UAV datasets because relatively less contribution was seen in the literature when compared with standard or remote-sensing based datasets. The paper discusses the following algorithms: Faster RCNN, Cascade RCNN, R-FCN etc. into two-stage, YOLO and its variants, SSD, RetinaNet into one-stage and CornerNet, Objects as Point etc. under advanced stages in deep learning based detectors. Further, one-two and advanced stages of detectors are studied in detail focusing on low-altitude UAV datasets. The paper provides a broad summary of low altitude datasets along with their respective literature in detection algorithms for the potential use of researchers. Various research gaps and challenges for object detection and classification in UAV datasets that need to deal with for improving the performance are also listed.

Edge Computing-Enabled Deep Learning for Real-time Video Optimization in IIoT

Real-time multimedia applications have gained immense popularity in the industrial Internet of Things (IIoT) paradigm. Due to the impact of the complex industrial environment, the transmission of video streaming is usually unstable. In the duration of a low bandwidth transmission, existing optimization methods often reduce the original resolution of some frames in a random way to avoid the video interruption. If the key frames with some important content are selected to be transmitted with a low resolution, it will greatly reduce the effect of industrial supervision. In view of this challenge, a real-time video streaming optimization method by reducing the number of video frames transmitted in the IIoT environment is proposed. Concretely, a deep learning-based object detection algorithm is recruited to effectively select the key frames in our method. The key frames with the original resolution will be transmitted along with audio data. As some nonkey frames are selectively discarded, it is helpful for smooth network transmitting with fewer bandwidth requirements. Moreover, we employ edge servers to run the object detection algorithm, and adjust video transmission flexibly. Extensive experiments are conducted to validate the effectiveness, and dependability of our method.

FCC-Net: A Full-Coverage Collaborative Network for Weakly Supervised Remote Sensing Object Detection

With an ever-increasing resolution of optical remote-sensing images, how to extract information from these images efficiently and effectively has gradually become a challenging problem. As it is prohibitively expensive to label every object in these high-resolution images manually, there is only a small number of high-resolution images with detailed object labels available, highly insufficient for common machine learning-based object detection algorithms. Another challenge is the huge range of object sizes: it is difficult to locate large objects, such as buildings and small objects, such as vehicles, simultaneously. To tackle these problems, we propose a novel neural network based remote sensing object detector called full-coverage collaborative network (FCC-Net). The detector employs various tailored designs, such as hybrid dilated convolutions and multi-level pooling, to enhance multiscale feature extraction and improve its robustness in dealing with objects of different sizes. Moreover, by utilizing asynchronous iterative training alternating between strongly supervised and weakly supervised detectors, the proposed method only requires image-level ground truth labels for training. To evaluate the approach, we compare it against a few state-of-the-art techniques on two large-scale remote-sensing image benchmark sets. The experimental results show that FCC-Net significantly outperforms other weakly supervised methods in detection accuracy. Through a comprehensive ablation study, we also demonstrate the efficacy of the proposed dilated convolutions and multi-level pooling in increasing the scale invariance of an object detector.

A Deep Learning-Based Fragment Detection Approach for the Arena Fragmentation Test

The arena fragmentation test (AFT) is one of the tests used to design an effective warhead. Conventionally, complex and expensive measuring equipment is used for testing a warhead and measuring important factors such as the size, velocity, and the spatial distribution of fragments where the fragments penetrate steel target plates. In this paper, instead of using specific sensors and equipment, we proposed the use of a deep learning-based object detection algorithm to detect fragments in the AFT. To this end, we acquired many high-speed videos and built an AFT image dataset with bounding boxes of warhead fragments. Our method fine-tuned an existing object detection network named the Faster R-convolutional neural network (CNN) on this dataset with modification of the network’s anchor boxes. We also employed a novel temporal filtering method, which was demonstrated as an effective non-fragment filtering scheme in our recent previous image processing-based fragment detection approach, to capture only the first penetrating fragments from all detected fragments. We showed that the performance of the proposed method was comparable to that of a sensor-based system under the same experimental conditions. We also demonstrated that the use of deep learning technologies in the task of AFT significantly enhanced the performance via a quantitative comparison between our proposed method and our recent previous image processing-based method. In other words, our proposed method outperformed the previous image processing-based method. The proposed method produced outstanding results in terms of finding the exact fragment positions.

Dynamic prioritization of surveillance video data in real-time automated detection systems

Automated object detection systems are a key component of modern surveillance applications. These systems rely on computationally expensive computer vision algorithms that perform object detection on visual data recorded by surveillance cameras. Due to the security and safety implications of these systems, this visual data st be processed accurately and in real-time. However, many of the frames that are created by the surveillance cameras may be of low importance, providing little or no useful information to the object detection system. Sub-sampling surveillance data by prioritizing important camera frames can greatly reduce unnecessary computation. Consequently, several works have explored dynamic visual data sub-sampling using various modalities of information (ie. spatial or temporal information) for prioritization. Few works, however, have combined and evaluated different modalities of information together for real-time prioritization of visual surveillance data. This work evaluates several individual and combined prioritization metrics derived from different modalities of information for use with a modern deep learning-based object detection algorithm. Both processing time and object detection rate are measured and used to rank the prioritization metrics. A novel approach that uses the historical detection confidences created by the object detection algorithm was demonstrated to be the best standalone prioritization metric. Additionally, a novel ensemble method that uses a KNN regressor to combine the best of the previously evaluated metrics to create a dynamic prioritization method is presented. This ensemble approach is shown to increase the object detection rate by up to 60% as compared to a static sub-sampling baseline as demonstrated using three publicly available datasets. The increased object detection rate was achieved while meeting the real-time constraints of the automated object detection system.

Pedestrian Detection Algorithm for Intelligent Vehicles in Complex Scenarios.

Pedestrian detection is an important aspect of the development of intelligent vehicles. To address problems in which traditional pedestrian detection is susceptible to environmental factors and are unable to meet the requirements of accuracy in real time, this study proposes a pedestrian detection algorithm for intelligent vehicles in complex scenarios. YOLOv3 is one of the deep learning-based object detection algorithms with good performance at present. In this article, the basic principle of YOLOv3 is elaborated and analyzed firstly to determine its limitations in pedestrian detection. Then, on the basis of the original YOLOv3 network model, many improvements are made, including modifying grid cell size, adopting improved k-means clustering algorithm, improving multi-scale bounding box prediction based on receptive field, and using Soft-NMS algorithm. Finally, based on INRIA person and PASCAL VOC 2012 datasets, pedestrian detection experiments are conducted to test the performance of the algorithm in various complex scenarios. The experimental results show that the mean Average Precision (mAP) value reaches 90.42%, and the average processing time of each frame is 9.6 ms. Compared with other detection algorithms, the proposed algorithm exhibits accuracy and real-time performance together, good robustness and anti-interference ability in complex scenarios, strong generalization ability, high network stability, and detection accuracy and detection speed have been markedly improved. Such improvements are significant in protecting the road safety of pedestrians and reducing traffic accidents, and are conducive to ensuring the steady development of the technological level of intelligent vehicle driving assistance.