Deep Learning Object Detection Research Articles

An Efficient Channel Attention-Enhanced Lightweight Neural Network Model for Metal Surface Defect Detection

There are problems of low model detection accuracy, low detection speed and difficulty in deploying online inspection in industrial surface defect detection relying on deep learning object detection algorithms. In order to effectively solve this problem, an efficient channel attention-enhanced lightweight neural network model named as EMV2-YOLOX is proposed in this paper. The algorithm incorporates the ECA module into the lightweight backbone extraction network MobileNetV2 to achieve adaptive adjustment of channel information weights, which can improve the extraction capability of the algorithm. The YOLOX model is also introduced to enhance the model’s identification and localization of tiny defects. The improved algorithm can guarantee the model’s accuracy and improve the model detection performance, as well as the carrying capacity of hardware devices. The experimental results show that the highest accuracy is achieved on the GCT10 and NEU public defect datasets with mean Average Precision values of 0.86 and 0.68, respectively, which is higher than the accuracy of the EMV2yoloV4 model. The parametric model number is only 10.24[Formula: see text]M in size, and the detection rate is 54.25[Formula: see text]f/s, which is the highest performance in embedded devices. EMV2-YOLOX, combined with the attention mechanism, can efficiently extract the location and semantic information of hard-to-detect defects and plays a vital role in the intelligent detection methods.

Deep Transfer Learning Approach for Robust Hand Detection

Human hand detection in uncontrolled environments is a challenging visual recognition task due to numerous variations of hand poses and background image clutter. To achieve highly accurate results as well as provide real-time execution, we proposed a deep transfer learning approach over the state-of-the-art deep learning object detector. Our method, denoted as YOLOHANDS, is built on top of the You Only Look Once (YOLO) deep learning architecture, which is modified to adapt to the single class hand detection task. The model transfer is performed by modifying the higher convolutional layers including the last fully connected layer, while initializing lower non-modified layers with the generic pretrained weights. To address robustness issues, we introduced a comprehensive augmentation procedure over the training image dataset, specifically adapted for the hand detection problem. Experimental evaluation of the proposed method, which is performed on a challenging public dataset, has demonstrated highly accurate results, comparable to the state-of-the-art methods.

Defect inspection of indoor components in buildings using deep learning object detection and augmented reality

Defect inspection of indoor components in buildings using deep learning object detection and augmented reality

Investigation of Combining Deep Learning Object Recognition with Drones for Forest Fire Detection and Monitoring

Forest fires are a global environmental problem that can cause significant damage to natural resources and human lives. The increasing frequency and severity of forest fires have resulted in substantial losses of natural resources. To mitigate this, an effective fire detection and monitoring system is crucial. This work aims to explore and review the current advancement in the field of forest fire detection and monitoring using both drones or unmanned aerial vehicles (UAVs), and deep learning techniques. The utilization of drones fully equipped with specific sensors and cameras provides a cost-effective and efficient solution for real-time monitoring and early fire detection. In this paper, we conduct a comprehensive analysis of the latest developments in deep learning object detection, such as YOLO (You Only Look Once), R-CNN (Region-based Convolutional Neural Network), and their variants, with a focus on their potential application in the field of forest fire monitoring. The performed experiments show promising results in multiple metrics, making it a valuable tool for fire detection and monitoring.

Deep learning-based multiple moving vehicle detection and tracking using a nonlinear fourth-order reaction-diffusion based multi-scale video object analysis

<p style='text-indent:20px;'>A novel automatic vehicle detection and tracking framework is proposed in this research work. First, the moving vehicles are detected in the frames of the video sequence by combining some deep learning and Gaussian Mixture Model-based object detection techniques. Then, the correspondence between the video objects detected in successive frames is determined using a multi-scale analysis of those objects. A scale-space representation is created by applying the numerical approximation algorithm that solves a nonlinear fourth-order reaction-diffusion based model whose mathematical validity is rigorously investigated here. A color image feature extraction is then performed at each scale using SURF and HOG-based features and the feature vectors determined at multiple scales are next concatenated into a final descriptor. A novel instance matching-based vehicle tracking technique using the distances between these feature vectors is then proposed. The results of the performed detection and tracking simulations are finally discussed.</p>

OBBStacking: An Ensemble Method for Remote Sensing Object Detection

Ensemble methods are a reliable way to combine several models to achieve superior performance. However, research on the application of ensemble methods in the remote sensing object detection scenario is mostly overlooked. Two problems arise. First, one unique characteristic of remote sensing object detection is the Oriented Bounding Boxes (OBB) of the objects and the fusion of multiple OBBs requires further research attention. Second, the widely used deep learning object detectors provide a score for each detected object as an indicator of confidence, but how to use these indicators effectively in an ensemble method remains a problem. Trying to address these problems, this paper proposes OBBStacking, an ensemble method that is compatible with OBBs and combines the detection results in a learned fashion. This ensemble method helps take 1st place in the Challenge Track <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Fine-grained Object Recognition in High-Resolution Optical Images</i> , which was featured in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2021 Gaofen Challenge on Automated High-Resolution Earth Observation Image Interpretation</i> . The experiments on DOTA dataset and FAIR1M dataset demonstrate the improved performance of OBBStacking and the features of OBBStacking are analyzed. Code will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Haoning724/obbstacking</uri>

Comparative Study of Some Deep Learning Object Detection Algorithms: R-CNN, FAST R-CNN, FASTER R-CNN, SSD, and YOLO

Due to its numerous applications and new technological advancements, object detection has gained more attention in the last few years. This study examined various uses of some deep learning object detection algorithms. These algorithms are divided into two-stage detectors like Region Based Convolutional Neural Network (R-CNN), Fast Region Based Convolutional Neural Network (Faster R-CNN), and Faster Region Based Convolutional Neural Network (Faster R-CNN), and one-stage detectors like Single Shot MultiBox Detector (SSD) and You Only Look Once (YOLO) algorithms that are used in text and face detection, image retrieval, security, surveillance, traffic control, traffic sign/light detection, pedestrian detection and in medical areas among others. This research primarily focuses on three applications: drone surveillance, applications relating to traffic, and medical fields. From the analysis performed, it was found that of the various deep learning models used in the various application areas, YOLO, a one-stage detector, is the most popular algorithm for drone surveillance. SSD, another one-stage detector, was primarily used in applications related to traffic, and Faster R-CNN, a two-stage detector, is the most popular algorithm for applications in the medical field

A Temperature Monitoring Method For Power Electronic Converter Based on Infrared Image and Object Detection Algorithm

Power electronic converters are more and more widely used, and abnormal temperature of converter components is the most important factor in converter failure. To improve the reliability of the converter design, it is necessary to monitor the temperature of key components in the converter during the prototype test stage. The temperature measurement method of infrared thermal images has rich temperature information, wide detection range, and does not affect the original circuit design. However, in the current automatic temperature measurement methods, it is necessary to manually establish a standard matching template for the infrared thermal image of the circuit to be tested, which indicates a large workload and poor versatility. This paper proposes a method for fully automatic temperature monitoring of converter components. This method is based on a deep learning object detection algorithm, which can automatically identify the type of converter components, obtain partial infrared thermal images of components through heterogeneous image registration, achieve accurate component temperature monitoring and facilitate the converter state monitoring and fault detection. The advantages of this method are: 1) there is no need to manually establish a standard template for each converter; 2) it can monitor the converter temperature without manual intervention; 3) combining the temperature information and circuit prior knowledge, the state monitoring and fault diagnosis can further be realized. The experimental results also verify the feasibility and accuracy of this method.

Intelligent Deployment Solution for Tabling Adapting Deep Learning

The deep learning object detection model can well extract the coordinate information of the separating point of the concentrate ore belt boundary line. However, with the drawbacks of acquired zonal image features, it is difficult to obtain the perfect prediction of the concentrate grade and recovery rate. In this study, by adapting deep learning semantic segmentation technique with DeepLab V3+, which can effectively extract multi-dimensional image features. The mapping relationship between image features and attributes of the Tabling equipment is achieved by constructing a multi-output support vector regression model optimized of a sparrow search algorithm (SSA-MSVR). The beneficiation indicators and operating parameters of the Tabling can be continually detected and optimized by an intelligent system that mainly includes image recognition softwares, automatic control units, data processing and communication workstations, and matching intelligent equipment, which accomplished the intelligent deployment solution of the Tabling beneficiation process.

Anchor-free Proposal Generation Network for Efficient Object Detection

Deep learning object detection methods are usually based on anchor-free or anchor-based scheme for extracting object proposals and one-stage or two-stage structure for producing final predictions. As each scheme or structure has its own strength and weakness, combining their strength in a unified framework is an interesting research topic. However, this topic has not attracted much attention in recent years. This paper presents a two-stage object detection method that utilizes an anchor-free scheme for generating object proposals in the initial stage. For proposal generation, this paper employs an efficient anchor-free network for predicting object corners and assigns object proposals based on detected corners. For object prediction, an efficient detection network is designed to enhance both detection accuracy and speed. The detection network includes a lightweight binary classification subnetwork for removing most false positive object candidates and a light-head detection subnetwork for generating final predictions. Experimental results on the MS-COCO dataset demonstrate that the proposed method outperforms both anchor-free and two-stage object detection baselines in terms of detection performance.

Dental Caries Detection Using Faster R-CNN and YOLO V3

Deep learning techniques are gradually being utilized in many fields. Healthcare is a field in which deep learning can thrive. The study conducted focuses on using deep learning object detection models to detect dental cavities in an individual’s mouth. These images taken from a camera will be fed live to the object detection model to discover the precise coordinates of dental caries if it happens to exist. Previous studies depict that X-rays were often used for detecting dental caries. This study wants to put emphasis on avoiding the use of X-rays since they have a chance of harming human tissue, as well as, and they cannot detect hidden caries. Thus, it is necessary to detect dental caries in an accurate manner, with the proper tools. Studies have also conducted dental caries prediction using the frontal view of the images only. Some have made use of different angles for the images in the dataset, however, there still lies the problem of capturing the posterior teeth. Roughly 300 images get used, as the dataset, for the training and testing of the object detection model. 80% is used for training whereas 20% is used for testing. Two deep learning frameworks have been proposed to evaluate dental cavities, the You Only Once (YOLO) V3 object detection model and the Faster Region-Convolutional Neural Network object detection model. Our results show that the YOLO V3 model consists of an accuracy of 75%, while Faster R-CNN had an accuracy of 80%. The sensitivity values of YOLO V3 and Faster R-CNN were 76% and 73% respectively. The model with better performance would be used for future development of the product, along with the hardware components. Our hardware components aim to take images from outside the mouth, for the frontal teeth, and take images from inside the mouth, for the posterior teeth.

Challenges for Identification of Cow Dung by Using Drone and Object Detection Deep Learning Model in Grazing Field

Challenges for Identification of Cow Dung by Using Drone and Object Detection Deep Learning Model in Grazing Field

Study on Pig Body Condition Scoring Based on Deep Learning Model EfficientNet-B0

Pig body condition scoring is an important tool for pig farmers to ensure the health and nutritional status of pigs. An efficient and accurate body condition scoring method is very important to ensure the health of pigs. This thesis proposes an automatic and objective pig body condition scoring method based on deep learning and EfficientNet-B0. Traditional methods of visual examination and manual palpation are subjective, time-consuming, and can vary from observer to observer, leading to inconsistent and unreliable results. Deep learning model based on artificial neural network shows great potential in automating pig state scoring. The proposed method was trained and evaluated on a large pig image dataset, and compared with traditional manual methods and object detection deep learning algorithms. The results show that this method can improve the accuracy and efficiency of pig body condition grading, with a higher average accuracy of 85.66% for body condition classification, which has certain practical production significance and provides a foundation for further related research.

An innovative structural health assessment tool for existing precast concrete buildings using deep learning methods and thermal infrared satellite imagery

Currently, there is a limited number of tools that can be used to assess progressive damage of buildings in large-scale study areas. The effectiveness of such tools is also constrained by a lack of sufficient and reliable data from the buildings and the area itself. This research article presents an innovative framework for damage detection and classification of precast concrete (PC) buildings based on satellite infrared (IR) imagery. The framework uses heat leakage changes over time to assess the progressive damage of buildings. Multispectral satellite images are used for a spatial scanning and large-scale assessment of a study area. A deep learning object detection algorithm coupled with two pixel intensities classification approaches are utilized in the framework. The proposed framework is demonstrated on two case study areas (parts of Karaganda and Almaty cities) in Kazakhstan using a set of multitemporal satellite images. Overall, the proposed framework, in combination with a YOLOv3 algorithm, successfully detects 85% of the PC buildings in the study areas. The use of a peak heat leakage classification approach (in comparison to mean heat leakage classification) over the 4 years showed a good agreement with the proposed framework. On-site visual inspections confirmed that PC buildings that were classified as having “High damage probability” have indeed evident signs of deterioration, as well as a more heat leakage than the rest of the buildings in the study areas. Whilst the framework has some limitations such as its applicability to extreme continental climate and its low sensitivity to detect minor damage, the proposed innovative framework showed very promising results at detecting progressive damage in PC buildings. This article contributes towards developing more efficient long-term damage assessment tools for existing buildings in large urban areas.

Defect Detection of MEMS Based on Data Augmentation, WGAN-DIV-DC, and a YOLOv5 Model

Surface defect detection of micro-electromechanical system (MEMS) acoustic thin film plays a crucial role in MEMS device inspection and quality control. The performances of deep learning object detection models are significantly affected by the number of samples in the training dataset. However, it is difficult to collect enough defect samples during production. In this paper, an improved YOLOv5 model was used to detect MEMS defects in real time. Mosaic and one more prediction head were added into the YOLOv5 baseline model to improve the feature extraction capability. Moreover, Wasserstein divergence for generative adversarial networks with deep convolutional structure (WGAN-DIV-DC) was proposed to expand the number of defect samples and to make the training samples more diverse, which improved the detection accuracy of the YOLOv5 model. The optimal detection model achieved 0.901 mAP, 0.856 F1 score, and a real-time speed of 75.1 FPS. As compared with the baseline model trained using a non-augmented dataset, the mAP and F1 score of the optimal detection model increased by 8.16% and 6.73%, respectively. This defect detection model would provide significant convenience during MEMS production.

CNN-based object detection and growth estimation of plum fruit (Prunus mume) using RGB and depth imaging techniques

Modern people who value healthy eating habits have shown increasing interest in plum (Prunus mume) fruits, primarily owing to their nutritiousness and proven efficacy. As consumption increases, it becomes important to monitor work to prevent Prunus mume fruits from falling out. Moreover, determining the growth status of Prunus mume is also crucial and is attracting increasing attention. In this study, convolutional neural network (CNN)-based deep learning object detection was developed using RGBD images collected from Prunus mume farms. These RGBD images consider various environments, including the depth information of objects in the outdoor field. A faster region-based convolutional neural network (R-CNN), EfficientDet, Retinanet, and Single Shot Multibox Detector (SSD) were applied for detection, and the performance of all models was estimated by comparing their respective computing speeds and average precisions (APs). The test results show that the EfficientDet model is the most accurate, and SSD MobileNet is the fastest among the four models. In addition, the algorithm was developed to acquire the growth status of P. mume fruits by applying the coordinates and score values of bounding boxes to the depth map. Compared to the diameters of the artificial Prunus mume fruits used as the experimental group, the calculated diameters were very similar to those of the artificial objects. Collectively, the results demonstrate that the CNN-based deep learning Prunus mume detection and growth estimation method can be applied to real farmlands.

Survey on Image and Point-Cloud Fusion-Based Object Detection in Autonomous Vehicles

With the improvements in sensor performance (cameras, Lidars) and the application of deep learning in object detection, autonomous vehicles (AVs) are gradually becoming more notable. After 2019, AV has produced a wave of enthusiasm, and many papers on object detection were published, boasting both practicality and innovation. Due to hardware limitations, it is difficult to accomplish accurate and reliable environment perception using a single sensor. However, multi-sensor fusion technology provides an acceptable solution. Considering the AV cost and object detection accuracy, both the traditional and existing literature on object detection using image and point-cloud was reviewed in this paper. Additionally, for the fusion-based structure, the object detection method was categorized in this paper based on the image and point-cloud fusion types: early fusion, deep fusion, and late fusion. Moreover, a clear explanation of these categories was provided including both the advantages and limitations. Finally, the opportunities and challenges the environment perception may face in the future were assessed.

Traffic Lights Detection and Recognition with New Benchmark Datasets Using Deep Learning and TensorFlow Object Detection API

Today, traffic lights are widely used in places with high vehicle traffic. Especially in autonomous vehicles, fast and high accuracy detection and recognition of traffic lights are critical. Machine learning methods are generally used to do this. Deep learning models give more successful results than machine learning methods in detecting the exact location of traffic lights in different climatic conditions. In this study, Faster R-CNN Inception v2 deep learning model was trained and tested on two different datasets that we prepared and published publicly under variable traffic and climatic conditions in Turkey. Successful results were obtained with fewer data by using the Transfer Learning method with the help of TensorFlow Object Detection API in the training of the model. It has been shown that the datasets we have prepared can be developed considering the conditions in other countries and successful results will be obtained.

An improved algae-YOLO model based on deep learning for object detection of ocean microalgae considering aquacultural lightweight deployment

Algae are widely distributed and have a considerable impact on water quality. Harmful algae can degrade water quality and be detrimental to aquaculture, while beneficial algae are widely used. The accuracy and speed of existing intelligent algae detection methods are available, but the size of parameters of models is large, the equipment requirements are high, the deployment costs are high, and there is still little research on lightweight detection methods in the area of algae detection. In this paper, we propose an improved Algae-YOLO object detection approach, which is based on ShuffleNetV2 as the YOLO backbone network to reduce the parameter space, adapting the ECA attention mechanism to improve detection accuracy, and redesigning the neck structure replacing the neck structure with ghost convolution module for reducing the size of parameters, finally the method achieved the comparable accuracy. Experiments showed that the Algal-YOLO approach in this paper reduces the size of parameters by 82.3%, and the computation (FLOPs) is decreased from 16G to 2.9G with less loss of accuracy, and mAP by only 0.007 when compared to the original YOLOv5s. With high accuracy, the smaller model size are achieved, which reduces the equipment cost during actual deployment and helps to promote the practical application of algae detection.

Fruits Classification and Detection Application Using Deep Learning

Computer vision and image processing techniques are considered efficient tools for classifying various types of fruits and vegetables. In this paper, automated fruit classification and detection systems have been developed using deep learning algorithms. In this work, we used two datasets of colored fruit images. The first FIDS-30 dataset of 971 images with 30 distinct classes of fruits is publicly available. A major contribution of this work is to present a private dataset containing 761 images with eight categories of fruits, which have been collected and annotated by ourselves. In our work, the YOLOv3 deep learning object detection algorithm have been used for individual fruit detection across multiple classes, and ResNet50 and VGG16 techniques have been utilized for the final classification for the recognition of a single category of fruit in images. Next, we implemented the automatic fruit classification models with flask for the web framework. We got 86% and 85% accuracies from the public dataset with ResNet50 and VGG16, respectively. We achieved 99% accuracy with ResNet50 and 98% accuracy with the VGG16 model on the custom dataset. The domain adaptation approach is used in this work so that the proposed deep learning-based prediction model can cope with real-world problems of diverse domains. Finally, an Android smartphone application has been developed to classify and detect fruits with the camera in real-time. All the images uploaded from the Android device are automatically sent and consequently analyzed on the web, and finally, the processed data and results are returned to the smartphone. The custom dataset and implementation codes will be available after the manuscript has been accepted. The custom dataset can be found at: https://github.com/SumonAhmed334/dataset_fruit.