Deep Learning-based Object Detection Research Articles

Lightweight object detection algorithm for robots with improved YOLOv5

Robot object detection is important for the realisation of robot intelligence. Currently, deep learning-based object detection algorithms are used for robotic object detection. However, it faces some challenges in practical applications, such as the fact that robots frequently use resource-constrained devices, resulting in detection algorithms with long computation times and undesired detection rates. In order to address these concerns, this paper proposes a lightweight object detection algorithm for robots with an improved YOLOv5. To reduce the amount of processing required for feature extraction and increase the speed of detection, the C3Ghost and GhostConv modules have been introduced into the YOLOv5 backbone. The DWConv module was used in conjunction with the C3Ghost module in the YOLOv5 neck network to further reduce the number of model parameters and maintain accuracy. The CA (Coordinated Attention) module is also introduced to improve the extraction of features from detected objects and suppress irrelevant features, thus improving the algorithm’s detection accuracy. To verify the performance of the method, we tested it with a self-built dataset (4561 robot images in total) and the PascalVOC dataset respectively. The results show that compared with the YOLOv5s on the self-built dataset, the algorithm has a 54% decrease in FLOPs and a 52.53% decrease in the number of model parameters without a decrease in mAP (0.5). The effectiveness and superiority of the algorithm is demonstrated through case studies and comparisons.

A sensor fusion system with thermal infrared camera and LiDAR for autonomous vehicles and deep learning based object detection

Vision, Radar, and LiDAR sensors are widely used for autonomous vehicle perception technology. Especially object detection and classification are primarily dependent on vision sensors. However, under poor lighting conditions, dazzling sunlight, or bad weather an object might be difficult to be identified with general vision sensors. In this paper, we propose a sensor fusion system that combines a thermal infrared camera and a LiDAR sensor that can reliably detect and identify objects even in environments with poor visibility, such as day or night. The proposed method obtains the external parameters of the two sensors by designing and manufacturing a 3D calibration target to externally calibrate the thermal infrared camera and the LiDAR sensor. To verify the performance, experiments were conducted in day and night environments. The proposed sensor system and fusion algorithm show that it can reliably detect and identify objects even in environments with poor visibility, such as day or night.

Faster region-based convolutional neural network for plant-parasitic and non-parasitic nematode detection

Nematodes represent very abundant and the largest species diversity in the world. Nematodes, which live in a soil environment, possess several functions in agricultural systems. There are two huge groups of soil nematodes, a non-parasitic nematode, which contributes positively to ecological processes, and a plant-parasitic nematode, which cause various disease and reduces yield losses in the agricultural system. Early detection and classification in the agricultural area infected with plant-parasitic nematode and interpreting the soil level condition in this area required a fast and reliable detection system. However, nematode identification is challenging and time-consuming due to their similar morphology. This study applied a pre-trained faster region-based convolutional neural network (RCNN) for plant-parasitic and non-parasitic nematodes detection. These deep learning-based object detection models gave satisfactory results as the accuracy reached 87.5%.

Toward reliable calcification detection: calibration of uncertainty in object detection from coronary optical coherence tomography images.

Optical coherence tomography (OCT) has become increasingly essential in assisting the treatment of coronary artery disease (CAD). However, unidentified calcified regions within a narrowed artery could impair the outcome of the treatment. Fast and objective identification is paramount to automatically procuring accurate readings on calcifications within the artery. We aim to rapidly identify calcification in coronary OCT images using a bounding box and reduce the prediction bias in automated prediction models. We first adopt a deep learning-based object detection model to rapidly draw the calcified region from coronary OCT images using a bounding box. We measure the uncertainty of predictions based on the expected calibration errors, thus assessing the certainty level of detection results. To calibrate confidence scores of predictions, we implement dependent logistic calibration using each detection result's confidence and center coordinates. We implemented an object detection module to draw the boundary of the calcified region at a rate of 140 frames per second. With the calibrated confidence score of each prediction, we lower the uncertainty of predictions in calcification detection and eliminate the estimation bias from various object detection methods. The calibrated confidence of prediction results in a confidence error of , suggesting that the confidence calibration on calcification detection could provide a more trustworthy result. Given the rapid detection and effective calibration of the proposed work, we expect that it can assist in clinical evaluation of treating the CAD during the imaging-guided procedure.

English

Navigation of a lander craft to accurately soft land in a predetermined spot on an extra-terrestrial body is a challenging task. Due to the absence of GPS and real-time ground communication links, the lander craft has to navigate autonomously to the targeted landing site solely with the help of its onboard sensors. One of the major challenges in position estimation of the lander craft is the measurement inaccuracies associated with the inertial navigation systems. Optical landmark detection-based techniques are employed to aid the lander navigation and guidance system. In this technique, craters and their relative positions are used as landmarks for position estimation. Typical crater detection algorithms employ handcrafted feature extractors for crater identification. Recently deep learning-based approaches have been employed for crater detection, mainly for geomorphological studies and cataloguing. This paper proposes using deep learning-based object detection techniques applied to autonomous landing missions and their implementation on a Xilinx MPSoC FPGA device. Prototype hardware has been developed, and the YOLOv4-tiny algorithm has been implemented on it with an inference time of 471 ms per image. The results are presented, and their performance is evaluated.

Tile-based microscopic image processing for malaria screening using a deep learning approach

BackgroundManual microscopic examination remains the golden standard for malaria diagnosis. But it is laborious, and pathologists with experience are needed for accurate diagnosis. The need for computer-aided diagnosis methods is driven by the enormous workload and difficulties associated with manual microscopy based examination. While the importance of computer-aided diagnosis is increasing at an enormous pace, fostered by the advancement of deep learning algorithms, there are still challenges in detecting small objects such as malaria parasites in microscopic images of blood films. The state-of-the-art (SOTA) deep learning-based object detection models are inefficient in detecting small objects accurately because they are underrepresented on benchmark datasets. The performance of these models is affected by the loss of detailed spatial information due to in-network feature map downscaling. This is due to the fact that the SOTA models cannot directly process high-resolution images due to their low-resolution network input layer.MethodsIn this study, an efficient and robust tile-based image processing method is proposed to enhance the performance of malaria parasites detection SOTA models. Three variants of YOLOV4-based object detectors are adopted considering their detection accuracy and speed. These models were trained using tiles generated from 1780 high-resolution P. falciparum-infected thick smear microscopic images. The tiling of high-resolution images improves the performance of the object detection models. The detection accuracy and the generalization capability of these models have been evaluated using three datasets acquired from different regions.ResultsThe best-performing model using the proposed tile-based approach outperforms the baseline method significantly (Recall, [95.3%] vs [57%] and Average Precision, [87.1%] vs [76%]). Furthermore, the proposed method has outperformed the existing approaches that used different machine learning techniques evaluated on similar datasets.ConclusionsThe experimental results show that the proposed method significantly improves P. falciparum detection from thick smear microscopic images while maintaining real-time detection speed. Furthermore, the proposed method has the potential to assist and reduce the workload of laboratory technicians in malaria-endemic remote areas of developing countries where there is a critical skill gap and a shortage of experts.

ODRP: a new approach for spatial street sign detection from EXIF using deep learning-based object detection, distance estimation, rotation and projection system

ODRP: a new approach for spatial street sign detection from EXIF using deep learning-based object detection, distance estimation, rotation and projection system

Small-object detection based on YOLOv5 in autonomous driving systems

With the rapid advancements in the field of autonomous driving, the need for faster and more accurate object detection frameworks has become a necessity. Many recent deep learning-based object detectors have shown compelling performance for the detection of large objects in a variety of real-time driving applications. However, the detection of small objects such as traffic signs and traffic lights is a challenging task owing to the complex nature of such objects. Additionally, the complexity present in a few images due to the existence of foreground/background imbalance and perspective distortion caused by adverse weather and low-lighting conditions further makes it difficult to detect small objects accurately. In this letter, we investigate how an existing object detector can be adjusted to address specific tasks and how these modifications can impact the detection of small objects. To achieve this, we explore and introduce architectural changes to the popular YOLOv5 model to improve its performance in the detection of small objects without sacrificing the detection accuracy of large objects, particularly in autonomous driving. We will show that our modifications barely increase the computational complexity but significantly improve the detection accuracy and speed. Compared to the conventional YOLOv5, the proposed iS-YOLOv5 model increases the mean Average Precision (mAP) by 3.35% on the BDD100K dataset. Nevertheless, our proposed model improves the detection speed by 2.57 frames per second (FPS) compared to the YOLOv5 model.

An Adaptive Adversarial Patch-Generating Algorithm for Defending against the Intelligent Low, Slow, and Small Target

The “low, slow, and small” target (LSST) poses a significant threat to the military ground unit. It is hard to defend against due to its invisibility to numerous detecting devices. With the onboard deep learning-based object detection methods, the intelligent LSST (ILSST) can find and detect the ground unit autonomously in a denied environment. This paper proposes an adversarial patch-based defending method to blind the ILSST by attacking its onboard object detection network. First, an adversarial influence score was established to indicate the influence of the adversarial noise on the objects. Then, based on this score, we used the least squares algorithm and Bisectional search methods to search the patch’s optimal coordinates and size. Using the optimal coordinates and size, an adaptive patch-generating network was constructed to automatically generate patches on ground units and hide the ground units from the deep learning-based object detection network. To evaluate the efficiency of our algorithm, a new LSST view dataset was collected, and extensive attacking experiments are carried out on this dataset. The results demonstrate that our algorithm can effectively attack the object detection networks, is better than state-of-the-art adversarial patch-generating algorithms in hiding the ground units from the object detection networks, and has high transferability among the object detection networks.

A Deep Learning approach for fire object detection in Autonomous vehicles

In recent years, object detection has emerged as a crucial component of many popular consumer applications, including video surveillance and security systems, mobile text recognition, etc. The potential for autonomous vehicles (AVs) to increase driver satisfaction and decrease fatalities and injuries in traffic accidents has attracted a lot of interest in recent years. Object detection is critical to autonomous driving infrastructure. Autonomous automobiles need precise environmental interpretation to drive safely. Locating and identifying these things in real time is a significant challenge, but deep learning-based object detectors play a crucial role in this endeavour. In this paper, a prototype of the autonomous vehicle controlled by a microcontroller for fire object detection is proposed. Localization of the fire object in a picture using the deep learning model is performed using the live video feed from the camera installed on the prototype remote-operated car. The gadget may also pick up on specifics and alert the user. Experimental findings showed that the suggested prototype with deep learning architecture recognised and alerted devastating fires with high speed and accuracy in diverse weather conditions—sunny or overcast, day or night.

Multiple face mask wearer detection based on YOLOv3 approach

<div align="left"><a name="_Hlk108683337"></a><span lang="EN-US">The coronavirus disease 2019 (COVID-19) is a highly infectious disease caused by the SARS-CoV-2 coronavirus. In breaking the transmission chain of SARS-CoV-2, the government has made it compulsory for the people to wear a mask in public places to prevent COVID-19 transmission. Hence, an automated face mask detection is crucial to facilitate the monitoring process in ensuring people to wear a face mask in public. This project aims to develop an automated face and face mask detection for multiple people by applying deep learning-based object detection algorithm you only look once version 3 (YOLOv3). YOLOv3 object detection algorithm was concatenated with different backbones including ResNet-50 and Darknet-53 to develop the face and face mask detection model. Datasets were collected from online resources including Kaggle and Github and the images were filtered and labelled accordingly. The models were trained on 4393 images and evaluated based on precision, recall, mean average precision and the detection time. In conclusion, DarkNet53_YOLOv3 was chosen as the better model compared to ResNet50_YOLOv3 model with its good performance on accuracy with a mAP of 95.94% and a fast detection speed with a detection time of 50 seconds on 776 images. </span></div>

Sea-Surface Object Detection Based on Electro-Optical Sensors: A Review

Sea-surface object detection is critical for navigation safety of autonomous ships. Electro-optical (EO) sensors, such as video cameras, complement radar on board in detecting small obstacle sea-surface objects. Traditionally, researchers have used horizon detection, background subtraction, and foreground segmentation techniques to detect sea-surface objects. Recently, deep learning-based object detection technologies have been gradually applied to sea-surface object detection. This article demonstrates a comprehensive overview of sea-surface object-detection approaches where the advantages and drawbacks of each technique are compared, covering four essential aspects: EO sensors and image types, traditional object-detection methods, deep learning methods, and maritime datasets collection. In particular, sea-surface object detections based on deep learning methods are thoroughly analyzed and compared with highly influential public datasets introduced as benchmarks to verify the effectiveness of these approaches. The article also proposes the direction of future research for sea-surface object detection based on EO sensors.

A lightweight object detection network in low-light conditions based on depthwise separable pyramid network and attention mechanism on embedded platforms

Deep learning-based object detection algorithms have been widely used and are effective in autonomous driving. But their performance degrades dramatically under low-light conditions. The existing mitigation approach for low-light object detection uses image enhancement preprocessing to improve detection performance with limited success. In addition, this preprocessing approach incurs extra computation costs, consumes more resources, and makes it more challenging to implement the algorithm on embedded platforms. This paper proposes a novel object detection network in low-light conditions for embedded platforms. The new system consists of two innovative modules, including a lightweight low-light enhancement network DS-PyLENet and an anchor-free lightweight object detector CFEDet. DS-PyLENet is a three-level pyramid network configured with depthwise separable convolution residual blocks (DSCRBs) and multiscale DSCRBs. The CFEDet is configured with an improved EfficientNet backbone network with Coordinate Attention (CA) and vanilla MBconv, i.e., CA-fused EfficientNet, an improved Path Aggregation Network (PAN) fusion module, and an anchor-free Global Focal Loss (GFL) detection head. Two evaluation metrics, relative recall ratios, are proposed to depict the effectiveness of image enhancement on object detection more intuitively. The new model is demonstrated to be superior to the assembled model of EnlightenGAN and YOLOv5n in terms of detection accuracy and speed on GPU and embedded platforms. The testing results of the new model reveal that 83.5% mAP is achieved on the Exdark dataset and 67.4% mAP on the DarkFace dataset. The running rates of GTX 1080Ti and NVIDIA Jetson Xavier NX are 22 FPS and 2.6 FPS, respectively. The recall of CFEDet with DS-PyLENet increased to 22.1% and significantly improved over 18% with Zero-DCE and 17.2% with EnlightenGAN.

Plant growth information measurement based on object detection and image fusion using a smart farm robot

Traditionally, vegetable and fruit production has relied on empirical and ambiguous decisions made by human farmers. To overcome this uncertainty in agriculture, smart farm robots have been widely studied in recent years. However, measuring growth information with robots remains a challenge because of the similarity in the appearance of the target plant and those around it. In this study, we propose a smart farm robot that accurately measures the growth information of a target plant based on object detection, image fusion, and data augmentation with fused images. The proposed smart farm robot uses an end-to-end real-time deep learning-based object detector that shows state-of-the-art performances. To distinguish the target plant from other plants with a higher accuracy and improved robustness than those of existing methods, we exploited image fusion using both RGB and depth images. In particular, the data augmentation, based on the fused RGB, and depth information, contributes to the precise measurement of growth information from smart farms, regardless of the high density of vegetables and fruits in these farms. We propose and evaluate a real-time measurement system to obtain precise target-plant growth information in precision agriculture. The code and models are publicly available on Github: https://github.com/kistvision/Plant_growth_measurement.

Adversarial pan-sharpening attacks for object detection in remote sensing

Pan-sharpening, as one of the most commonly used techniques in remote sensing systems, aims to fuse texture-rich PAN images and multi-spectral MS images to obtain texture-rich MS images. With the development of deep learning, CNN based Pan-sharpening methods have received more and more attention in recent years. Since Pan-sharpening technique can integrate the complementary information of Pan and MS images, researchers usually apply object detectors on these pan-sharpened images to achieve reliable detection results. However, recent studies have shown that Deep Learning-based object detection methods are vulnerable to adversarial examples, i.e., adding imperceptible noise to clean images can fool well-trained deep neural networks. It is interesting to combine the pan-sharpening technique with adversarial examples to attack object detectors in remote sensing. In this paper, we propose a framework to generate adversarial pan-sharpened images. Specifically, we propose a two-stream network to generate the pan-sharpened images, and then utilize the shape loss and label loss to perform the attack task. To guarantee the quality of pan-sharpened images, a perceptual loss is utilized to balance spectral preservation and attacking performance. Experimental results demonstrate that the proposed method can generate effective adversarial pan-sharpened images that maintain a high success rate for white-box attacks and achieve transferability for black-box attacks.

Surface defect detection method for electronic panels based on double branching and decoupling head structure.

During the production of electronic panels, surface defects will inevitably appear. How to quickly and accurately detect these defects is very important to improve product quality. However, some problems such as high cost and low accuracy are still prominent when existing manual detection and traditional techniques are used to solve such problems. Therefore, more and more computer vision techniques are proposed to solve such problems, but the current application of deep learning-based object detection networks for surface defect detection of electronic panels is in a gap. The analysis found that there are two main reasons for this phenomenon. On the one hand, the surface defects of electronic panels have their unique characteristics such as multi-scale and irregular shape, and the current object detection networks cannot effectively solve these problems. On the other hand, the regression and classification tasks coupled in the current computational mechanism of each network are commonly found to cause the problem of conflict between them, which makes it more difficult to adapt these network models to the detection tasks in this scenario. Based on this, we design a supervised object detection network for electronic panel surface defect detection scenario for the first time. The computational mechanism of this network includes a prediction box generation strategy based on the double branch structure and a detection head design strategy that decouples the regression task from the classification task. In addition, we validated the designed network and the proposed method on our own collected dataset of surface defects in electronic panels. The final results of the comparative and ablation experiments show that our proposed method achieves an average accuracy of 78.897% for 64 surface defect categories, proving that its application to electronic panel surface defect detection scenarios can achieve better results.

Deep learning based object detection for resource constrained devices: Systematic review, future trends and challenges ahead

Deep learning models are widely being employed for object detection due to their high performance. However, the majority of applications that require object detection are functioning on resource-constrained edge devices. In the present era, there is a need for deep learning-based object detectors that are lightweight and perform well on these constrained edge devices.Objective: The research aims to identify current trends in resource-constrained applications for deep learning-based object detectors in terms of the technique used to create the model, the type of input image involved, the type of device used, and the type of application addressed by the model.Method: To achieve the objective of our research, a systematic literature review was carried out that yielded 167 studies. The models or techniques employed in the studies were grouped to better understand the research problem at hand. This review carefully reports every decision and provides many visualizations of the final studies in order to draw clear conclusions.Conclusion: The conclusion discussed the gaps, possibilities, and future perspectives discovered throughout the research process, implying that this field of study has grown profoundly in the last decade.

Detection of sperm cells by single-stage and two-stage deep object detectors

Today, infertility is a common health concern that affects approximately 15%–20% of the world population. The evaluation of male patients with infertility includes diverse examination and laboratory tests in comparison to female patients. In the evaluation of male infertility, sperm specimens are examined in terms of morphometry, concentration, and motility. Detection of sperm is the critical step to determine the concentration and motility parameters. In this study, a fusion approach of deep learning-based object detection techniques is utilized in the sperm detection problem for obtaining more accurate and consistent concentration and motility characteristics of sperm specimens. First, 12 sperm specimen videos acquired from different infertile patients were recorded. Then, regions in the video frames were labeled as sperm and non-sperm by the experts. Two different scenarios have been tested over the labeled dataset. Differently arranged train and test data ratios were also utilized for each scenario. In the first scenario, deep learning-based object detection algorithms were individually performed for the sperm detection of patient-oriented videos in terms of different train/test split ratios. Videos with the lowest mAP (mean Average Precision) values resulted in the first scenario were selected for the second scenario as target videos. The rest of the videos were used for model training without any patient-oriented manner. In the second scenario, the detection performance for more challenging videos is aimed to increase by using different videos instead of using the small part of the same video in the model training. Additionally, a fusion idea of the utilized deep learning-based techniques was proposed and performed over these low mAP resulted videos to increase the detection performance. The results are compared regarding the general mAP, class-wise APs, and training time. In the first scenario, YOLOv5 achieved the best results, while the proposed fusion approach achieved the best mAP scores in the second scenario.

Deep Learning-Based Real-Time Traffic Sign Recognition System for Urban Environments

A traffic sign recognition system is crucial for safely operating an autonomous driving car and efficiently managing road facilities. Recent studies on traffic sign recognition tasks show significant advances in terms of accuracy on several benchmarks. However, they lack performance evaluation in driving cars in diverse road environments. In this study, we develop a traffic sign recognition framework for a vehicle to evaluate and compare deep learning-based object detection and tracking models for practical validation. We collect a large-scale highway image set using a camera-installed vehicle for training models, and evaluate the model inference during a test drive in terms of accuracy and processing time. In addition, we propose a novel categorization method for urban road scenes with possible scenarios. The experimental results show that the YOLOv5 detector and strongSORT tracking model result in better performance than other models in terms of accuracy and processing time. Furthermore, we provide an extensive discussion on possible obstacles in traffic sign recognition tasks to facilitate future research through numerous experiments for each road condition.

Big Data Management Algorithms, Deep Learning-Based Object Detection Technologies, and Geospatial Simulation and Sensor Fusion Tools in the Internet of Robotic Things

The objective of this systematic review was to analyze the recently published literature on the Internet of Robotic Things (IoRT) and integrate the insights it articulates on big data management algorithms, deep learning-based object detection technologies, and geospatial simulation and sensor fusion tools. The research problems were whether computer vision techniques, geospatial data mining, simulation-based digital twins, and real-time monitoring technology optimize remote sensing robots. Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guidelines were leveraged by a Shiny app to obtain the flow diagram comprising evidence-based collected and managed data (the search results and screening procedures). Throughout January and July 2022, a quantitative literature review of ProQuest, Scopus, and the Web of Science databases was performed, with search terms comprising “Internet of Robotic Things” + “big data management algorithms”, “deep learning-based object detection technologies”, and “geospatial simulation and sensor fusion tools”. As the analyzed research was published between 2017 and 2022, only 379 sources fulfilled the eligibility standards. A total of 105, chiefly empirical, sources have been selected after removing full-text papers that were out of scope, did not have sufficient details, or had limited rigor For screening and quality evaluation so as to attain sound outcomes and correlations, we deployed AMSTAR (Assessing the Methodological Quality of Systematic Reviews), AXIS (Appraisal tool for Cross-Sectional Studies), MMAT (Mixed Methods Appraisal Tool), and ROBIS (to assess bias risk in systematic reviews). Dimensions was leveraged as regards initial bibliometric mapping (data visualization) and VOSviewer was harnessed in terms of layout algorithms.