Deep Learning-based Object Detection Research Articles

State-of-the-Art Deep Learning Methods for Objects Detection in Remote Sensing Satellite Images.

Introduction: Object detection in remotely sensed satellite images is critical to socio-economic, bio-physical, and environmental monitoring, necessary for the prevention of natural disasters such as flooding and fires, socio-economic service delivery, and general urban and rural planning and management. Whereas deep learning approaches have recently gained popularity in remotely sensed image analysis, they have been unable to efficiently detect image objects due to complex landscape heterogeneity, high inter-class similarity and intra-class diversity, and difficulty in acquiring suitable training data that represents the complexities, among others. Methods: To address these challenges, this study employed multi-object detection deep learning algorithms with a transfer learning approach on remotely sensed satellite imagery captured on a heterogeneous landscape. In the study, a new dataset of diverse features with five object classes collected from Google Earth Engine in various locations in southern KwaZulu-Natal province in South Africa was used to evaluate the models. The dataset images were characterized with objects that have varying sizes and resolutions. Five (5) object detection methods based on R-CNN and YOLO architectures were investigated via experiments on our newly created dataset. Conclusions: This paper provides a comprehensive performance evaluation and analysis of the recent deep learning-based object detection methods for detecting objects in high-resolution remote sensing satellite images. The models were also evaluated on two publicly available datasets: Visdron and PASCAL VOC2007. Results showed that the highest detection accuracy of the vegetation and swimming pool instances was more than 90%, and the fastest detection speed 0.2 ms was observed in YOLOv8.

An early warning model for starfish disaster based on multi-sensor fusion

Starfish have a wide range of feeding habits, including starfish, sea urchins, sea cucumbers, corals, abalones, scallops, and many other marine organisms with economic or ecological value. The starfish outbreak in coastal areas will lead to severe economic losses in aquaculture and damage the ecological environment. However, the current monitoring methods are still artificial, time-consuming, and laborious. This study used an underwater observation platform with multiple sensors to observe the starfish outbreak in Weihai, Shandong Province. The platform could collect the temperature, salinity, depth, dissolved oxygen, conductivity, other water quality data, and underwater video data. Based on these data, the paper proposed an early warning model for starfish prevalence (EWSP) based on multi-sensor fusion. A deep learning-based object detection method extracts time-series information on the number of starfish from underwater video data. For the extracted starfish quantity information, the model uses the k-means clustering algorithm to divide the starfish prevalence level into four levels: no prevalence, mild prevalence, medium prevalence, and high prevalence. Correlation analysis concluded that the water quality factors most closely related to the starfish prevalence level are temperature and salinity. Therefore, the selected water quality factor and the number of historical starfish are inputted. The future starfish prevalence level of the starfish outbreak is used as an output to train the BP (back propagation) neural network to build EWSP based on multi-sensor fusion. Experiments show that the accuracy rate of this model is 97.26%, whose precision meets the needs of early warning for starfish outbreaks and has specific application feasibility.

LKC-Net: large kernel convolution object detection network

Deep learning-based object detection methods have achieved great performance improvement. However, since small kernel convolution has been widely used, the semantic feature is difficult to obtain due to the small receptive fields, and the key information cannot be highlighted, resulting in a series of problems such as wrong detection, missing detection, and repeated detection. To overcome these problems, we propose a large kernel convolution object detection network based on feature capture enhancement and vast receptive field attention, called LKC-Net. Firstly, a feature capture enhancement block based on large kernel convolution is proposed to improve the semantic feature capturing ability, and depth convolution is used to reduce the number of parameters. Then, the vast receptive filed attention mechanism is constructed to enhance channel direction information extraction ability, and it is more compatible with the proposed backbone than other existing attention mechanisms. Finally, the loss function is improved by introducing the SIoU, which can overcome the angle mismatch problem between the ground truth and prediction box. Experiments are conducted on Pascal VOC and MS COCO datasets for demonstrating the performance of LKC-Net.

Multi-Scale Feature Selective Matching Network for Object Detection

Numerous deep learning-based object detection methods have achieved excellent performance. However, the performance on small-size object detection and positive and negative sample imbalance problems is not satisfactory. We propose a multi-scale feature selective matching network (MFSMNet) to improve the performance of small-size object detection and alleviate the positive and negative sample imbalance problems. First, we construct a multi-scale semantic enhancement module (MSEM) to compensate for the information loss of small-sized targets during down-sampling by obtaining richer semantic information from features at multiple scales. Then, we design the anchor selective matching (ASM) strategy to alleviate the training dominated by negative samples caused by the imbalance of positive and negative samples, which converts the offset values of the localization branch output in the detection head into localization scores and reduces negative samples by discarding low-quality anchors. Finally, a series of quantitative and qualitative experiments on the Microsoft COCO 2017 and PASCAL VOC 2007 + 2012 datasets show that our method is competitive compared to nine other representative methods. MFSMNet runs on a GeForce RTX 3090.

Deep Learning-Based Image Classification for Major Mosquito Species Inhabiting Korea.

Mosquitoes are one of the deadliest insects, causing harm to humans worldwide. Preemptive prevention and forecasting are important to prevent mosquito-borne diseases. However, current mosquito identification is mostly conducted manually, which consumes time, wastes labor, and causes human error. In this study, we developed an automatic image analysis method to identify mosquito species using a deep learning-based object detection technique. Color and fluorescence images of live mosquitoes were acquired using a mosquito capture device and were used to develop a deep learning-based object detection model. Among the deep learning-based object identification models, the combination of a swine transformer and a faster region-convolutional neural network model demonstrated the best performance, with a 91.7% F1-score. This indicates that the proposed automatic identification method can be rapidly applied for efficient analysis of species and populations of vector-borne mosquitoes with reduced labor in the field.

Deep learning based object detection and surrounding environment description for visually impaired people

Object detection, one of the most significant contributions of computer vision and machine learning, plays an immense role in identifying and locating objects in an image or a video. We recognize distinct objects and precisely get their information through object detection, such as their size, shape, and location. This paper developed a low-cost assistive system of obstacle detection and the surrounding environment depiction to help blind people using deep learning techniques. TensorFlow object detection API and SSDLite MobileNetV2 have been used to create the proposed object detection model. The pre-trained SSDLite MobileNetV2 model is trained on the COCO dataset, with almost 328,000 images of 90 different objects. The gradient particle swarm optimization (PSO) technique has been used in this work to optimize the final layers and their corresponding hyperparameters of the MobileNetV2 model. Next, we used the Google text-to-speech module, PyAudio, playsound, and speech recognition to generate the audio feedback of the detected objects. A Raspberry Pi camera captures real-time video where real-time object detection is done frame by frame with Raspberry Pi 4B microcontroller. The proposed device is integrated into a head cap, which will help visually impaired people to detect obstacles in their path, as it is more efficient than a traditional white cane. Apart from this detection model, we trained a secondary computer vision model and named it the “ambiance mode.” In this mode, the last three convolutional layers of SSDLite MobileNetV2 are trained through transfer learning on a weather dataset. The dataset comprises around 500 images from four classes: cloudy, rainy, foggy, and sunrise. In this mode, the proposed system will narrate the surrounding scene elaborately, almost like a human describing a landscape or a beautiful sunset to a visually impaired person. The performance of the object detection and ambiance description modes are tested and evaluated in a desktop computer and Raspberry Pi embedded system. Detection accuracy and mean average precision, frame rate, confusion matrix, and ROC curve measure the model's accuracy on both setups. This low-cost proposed system is believed to help visually impaired people in their day-to-day life.

Deep learning based object detection in nailfold capillary images

Microcirculation in a subject can be examined and pathological changes can be assessed by utilizing capillaroscopy, which is a very safe, convenient and non-invasive approach. Using a microscope, doctors view the capillaries by looking through nailfold epidermis. Nailfold anatomy is ideal to evaluate the microcirculation and detect various diseases caused by vascular damages. Rheumatologists evaluate systemic diseases which involve damage in vasculature, by analyzing the red blood cells within the capillaries. Sometimes, capillary morphology may be useful as an early indicator while, severity of damage in capillary architecture may indicate internal organ involvement. Thus, in a capillaroscopic assessment, the doctor examines modifications in morphological and functional aspects of capillaries. These comprise of capillary diameter, visibility, distribution, length, microhemorrhages, blood flow and density. In this paper, a novel object detection algorithm is proposed based on deep learning architectures for detecting and locating various capillary loops in the nailfold region. Various characteristic features are extracted from the capillaries through image processing algorithms and in turn an attempt is made to differentiate between images of diseased subjects and healthy controls.

Eggsplorer: a rapid plant–insect resistance determination tool using an automated whitefly egg quantification algorithm

BackgroundA well-known method for evaluating plant resistance to insects is by measuring insect reproduction or oviposition. Whiteflies are vectors of economically important viral diseases and are, therefore, widely studied. In a common experiment, whiteflies are placed on plants using clip-on-cages, where they can lay hundreds of eggs on susceptible plants in a few days. When quantifying whitefly eggs, most researchers perform manual eye measurements using a stereomicroscope. Compared to other insect eggs, whitefly eggs are many and very tiny, usually 0.2 mm in length and 0.08 mm in width; therefore, this process takes a lot of time and effort with and without prior expert knowledge. Plant insect resistance experiments require multiple replicates from different plant accessions; therefore, an automated and rapid method for quantifying insect eggs can save time and human resources.ResultsIn this work, a novel automated tool for fast quantification of whitefly eggs is presented to accelerate the determination of plant insect resistance and susceptibility. Leaf images with whitefly eggs were collected from a commercial microscope and a custom-built imaging system. A deep learning-based object detection model was trained using the collected images. The model was incorporated into an automated whitefly egg quantification algorithm, deployed in a web-based application called Eggsplorer. Upon evaluation on a testing dataset, the algorithm was able to achieve a counting accuracy as high as 0.94, r2 of 0.99, and a counting error of ± 3 eggs relative to the actual number of eggs counted by eye. The automatically collected counting results were used to determine the resistance and susceptibility of several plant accessions and were found to yield significantly comparable results as when using the manually collected counts for analysis.ConclusionThis is the first work that presents a comprehensive step-by-step method for fast determination of plant insect resistance and susceptibility with the assistance of an automated quantification tool.

Quantifying the Simulation–Reality Gap for Deep Learning-Based Drone Detection

The detection of drones or unmanned aerial vehicles is a crucial component in protecting safety-critical infrastructures and maintaining privacy for individuals and organizations. The widespread use of optical sensors for perimeter surveillance has made optical sensors a popular choice for data collection in the context of drone detection. However, efficiently processing the obtained sensor data poses a significant challenge. Even though deep learning-based object detection models have shown promising results, their effectiveness depends on large amounts of annotated training data, which is time consuming and resource intensive to acquire. Therefore, this work investigates the applicability of synthetically generated data obtained through physically realistic simulations based on three-dimensional environments for deep learning-based drone detection. Specifically, we introduce a novel three-dimensional simulation approach built on Unreal Engine and Microsoft AirSim for generating synthetic drone data. Furthermore, we quantify the respective simulation–reality gap and evaluate established techniques for mitigating this gap by systematically exploring different compositions of real and synthetic data. Additionally, we analyze the adaptation of the simulation setup as part of a feedback loop-based training strategy and highlight the benefits of a simulation-based training setup for image-based drone detection, compared to a training strategy relying exclusively on real-world data.

Application access for chromatic color detection

The process of identifying specific examples of an item inside an image or video is referred to as “object detection,” and it is one of the computer vision techniques. When individuals look at pictures or movies, they are able to recognize and find things that are of interest to them. The primary goal of the field of object recognition is to achieve this level of intelligence via the use of a computer. Image segmentation and blob analysis, which uses simple object properties such as size, shape, or color, and Feature-based object detection, which uses feature extraction, matching, and RANSAC to estimate the location of an object, are two of the many common techniques that are used in conjunction with deep learning and machine learning-based object detection. There are also many other common techniques that are used in conjunction with deep learning and machine learning-based object detection. The most challenging aspects of object detection are the processes of categorizing things and pinpointing their locations. When seen from a different perspective, the same object might seem to be completely different. Detection algorithms have a tough time distinguishing between various things at different scales and perspectives since the sizes and proportions of items almost always alter. The camera input would be used to determine an object’s shape, according to this project’s proposal. Circle, square, and triangle are the three distinct shapes that we are determining an item to be right now. The finished result is shown in MATLAB’s graphical user interface (GUI). The productive identification process is carried out with the assistance of MATLAB. Using the SFTA Algorithm, the application extracts features from the specific image of a video. The neural network algorithm is used to classify the image of video. Image segmentation and image detection are two examples of related computer vision methods that are inextricably intertwined with object recognition. These techniques allow us to better comprehend and analyze the situations shown in movies and still photographs. It has been determined whether or not the suggested system is suitable. Because of this, the testing has shown that the suggested system generates generally accurate indications of real performance at a rate of 99.4%.

Object Detection with Voice Guidance to Assist Visually Impaired Using Yolov7

Abstract: Object recognition technology has revolutionized various domains such as autonomous vehicles, industrial facilities, and many more. However, the visually impaired individuals, who are most in need of this technology, have not been able to utilize it effectively. Therefore, this paper presents a deep learning-based object detection system that is specifically designed for the blind community. The system incorporates the YOLOV7 (You Only Look Once) algorithm for object recognition and uses text-to-speech (TTS) technology to provide a voice-guidance technique that conveys information about the objects around them. The main objective of the proposed system is to empower visually challenged individuals to independently identify objects in a particular space without relying on external assistance. The system's efficiency and performance are thoroughly scrutinized through experiments to validate its accuracy and effectiveness. The proposed system employs technical terms such as deep learning, object recognition, YOLOV7 algorithm, and also image classification techniques for feature identification and extraction of the video frames and categorize them into the respective classes. It uses COCO dataset, which consist of around 123,287 hand-labeled images classified into 80 categories, has been utilized to train the proposed system. In conclusion, the suggested system for object detection uses cutting-edge deep learning techniques along with voice guidance methods to assist individuals who are visually impaired in identifying objects independently. The performance of the system is evaluated through experiments, demonstrating its effectiveness and potential for real-world applications

Application of Deep Learning-Based Object Detection Techniques in Fish Aquaculture: A Review

Automated monitoring and analysis of fish’s growth status and behaviors can help scientific aquaculture management and reduce severe losses due to diseases or overfeeding. With developments in machine vision and deep learning (DL) techniques, DL-based object detection techniques have been extensively applied in aquaculture with the advantage of simultaneously classifying and localizing fish of interest in images. This study reviews the relevant research status of DL-based object detection techniques in fish counting, body length measurement, and individual behavior analysis in aquaculture. The research status is summarized from two aspects: image and video analysis. Moreover, the relevant technical details of DL-based object detection techniques applied to aquaculture are also summarized, including the dataset, image preprocessing methods, typical DL-based object detection algorithms, and evaluation metrics. Finally, the challenges and potential trends of DL-based object detection techniques in aquaculture are concluded and discussed. The review shows that generic DL-based object detection architectures have played important roles in aquaculture.

Improved metaheuristics with deep learning based object detector for intelligent control in autonomous vehicles

In recent times, the autonomous systems have gained considerable interest due to their improved performance and lesser requirements for manual support. The autonomous systems have been extensively implemented in various domains such as logistics, industries, health care, finance, and so on. Intelligence control is the incorporation of autonomous and Artificial Intelligence (AI) systems that assist in critical decision-making steps. Autonomous driving is a new field in intelligent transportation systems that requires automated classification, detection, and the ranging of on-road difficulties. Therefore, the current study develops an Improved Metaheuristics technique with Deep Learning-based Object Detectors for Intelligent Control in Autonomous Vehicles (IMDLOD-ICAV). The presented technique mainly detects the objects to assist, in driving the autonomous vehicles. In the current research work, the RetinaNet model is applied as an object detector whereas the hyperparameter tuning process is executed with the help of the Nadam optimizer. Besides, the Elman Neural Network (ENN) model is also exploited to recognize the objects with a high accuracy. The parameter tuning process is performed with the help of the Improved Dragonfly Algorithm (IDFA). The authors conducted a comprehensive set of experiments to establish the superior performance of the proposed IMDLOD-ICAV technique. The outcomes confirmed the enhanced performance of the IMDLOD-ICAV technique with a maximum accuracy of 99.38%.

Lightweight object detection algorithm for robots with improved YOLOv5

Robot object detection is important for the realisation of robot intelligence. Currently, deep learning-based object detection algorithms are used for robotic object detection. However, it faces some challenges in practical applications, such as the fact that robots frequently use resource-constrained devices, resulting in detection algorithms with long computation times and undesired detection rates. In order to address these concerns, this paper proposes a lightweight object detection algorithm for robots with an improved YOLOv5. To reduce the amount of processing required for feature extraction and increase the speed of detection, the C3Ghost and GhostConv modules have been introduced into the YOLOv5 backbone. The DWConv module was used in conjunction with the C3Ghost module in the YOLOv5 neck network to further reduce the number of model parameters and maintain accuracy. The CA (Coordinated Attention) module is also introduced to improve the extraction of features from detected objects and suppress irrelevant features, thus improving the algorithm’s detection accuracy. To verify the performance of the method, we tested it with a self-built dataset (4561 robot images in total) and the PascalVOC dataset respectively. The results show that compared with the YOLOv5s on the self-built dataset, the algorithm has a 54% decrease in FLOPs and a 52.53% decrease in the number of model parameters without a decrease in mAP (0.5). The effectiveness and superiority of the algorithm is demonstrated through case studies and comparisons.

A sensor fusion system with thermal infrared camera and LiDAR for autonomous vehicles and deep learning based object detection

Vision, Radar, and LiDAR sensors are widely used for autonomous vehicle perception technology. Especially object detection and classification are primarily dependent on vision sensors. However, under poor lighting conditions, dazzling sunlight, or bad weather an object might be difficult to be identified with general vision sensors. In this paper, we propose a sensor fusion system that combines a thermal infrared camera and a LiDAR sensor that can reliably detect and identify objects even in environments with poor visibility, such as day or night. The proposed method obtains the external parameters of the two sensors by designing and manufacturing a 3D calibration target to externally calibrate the thermal infrared camera and the LiDAR sensor. To verify the performance, experiments were conducted in day and night environments. The proposed sensor system and fusion algorithm show that it can reliably detect and identify objects even in environments with poor visibility, such as day or night.

Faster region-based convolutional neural network for plant-parasitic and non-parasitic nematode detection

Nematodes represent very abundant and the largest species diversity in the world. Nematodes, which live in a soil environment, possess several functions in agricultural systems. There are two huge groups of soil nematodes, a non-parasitic nematode, which contributes positively to ecological processes, and a plant-parasitic nematode, which cause various disease and reduces yield losses in the agricultural system. Early detection and classification in the agricultural area infected with plant-parasitic nematode and interpreting the soil level condition in this area required a fast and reliable detection system. However, nematode identification is challenging and time-consuming due to their similar morphology. This study applied a pre-trained faster region-based convolutional neural network (RCNN) for plant-parasitic and non-parasitic nematodes detection. These deep learning-based object detection models gave satisfactory results as the accuracy reached 87.5%.

Toward reliable calcification detection: calibration of uncertainty in object detection from coronary optical coherence tomography images.

Optical coherence tomography (OCT) has become increasingly essential in assisting the treatment of coronary artery disease (CAD). However, unidentified calcified regions within a narrowed artery could impair the outcome of the treatment. Fast and objective identification is paramount to automatically procuring accurate readings on calcifications within the artery. We aim to rapidly identify calcification in coronary OCT images using a bounding box and reduce the prediction bias in automated prediction models. We first adopt a deep learning-based object detection model to rapidly draw the calcified region from coronary OCT images using a bounding box. We measure the uncertainty of predictions based on the expected calibration errors, thus assessing the certainty level of detection results. To calibrate confidence scores of predictions, we implement dependent logistic calibration using each detection result's confidence and center coordinates. We implemented an object detection module to draw the boundary of the calcified region at a rate of 140 frames per second. With the calibrated confidence score of each prediction, we lower the uncertainty of predictions in calcification detection and eliminate the estimation bias from various object detection methods. The calibrated confidence of prediction results in a confidence error of , suggesting that the confidence calibration on calcification detection could provide a more trustworthy result. Given the rapid detection and effective calibration of the proposed work, we expect that it can assist in clinical evaluation of treating the CAD during the imaging-guided procedure.

English

Navigation of a lander craft to accurately soft land in a predetermined spot on an extra-terrestrial body is a challenging task. Due to the absence of GPS and real-time ground communication links, the lander craft has to navigate autonomously to the targeted landing site solely with the help of its onboard sensors. One of the major challenges in position estimation of the lander craft is the measurement inaccuracies associated with the inertial navigation systems. Optical landmark detection-based techniques are employed to aid the lander navigation and guidance system. In this technique, craters and their relative positions are used as landmarks for position estimation. Typical crater detection algorithms employ handcrafted feature extractors for crater identification. Recently deep learning-based approaches have been employed for crater detection, mainly for geomorphological studies and cataloguing. This paper proposes using deep learning-based object detection techniques applied to autonomous landing missions and their implementation on a Xilinx MPSoC FPGA device. Prototype hardware has been developed, and the YOLOv4-tiny algorithm has been implemented on it with an inference time of 471 ms per image. The results are presented, and their performance is evaluated.

Tile-based microscopic image processing for malaria screening using a deep learning approach

BackgroundManual microscopic examination remains the golden standard for malaria diagnosis. But it is laborious, and pathologists with experience are needed for accurate diagnosis. The need for computer-aided diagnosis methods is driven by the enormous workload and difficulties associated with manual microscopy based examination. While the importance of computer-aided diagnosis is increasing at an enormous pace, fostered by the advancement of deep learning algorithms, there are still challenges in detecting small objects such as malaria parasites in microscopic images of blood films. The state-of-the-art (SOTA) deep learning-based object detection models are inefficient in detecting small objects accurately because they are underrepresented on benchmark datasets. The performance of these models is affected by the loss of detailed spatial information due to in-network feature map downscaling. This is due to the fact that the SOTA models cannot directly process high-resolution images due to their low-resolution network input layer.MethodsIn this study, an efficient and robust tile-based image processing method is proposed to enhance the performance of malaria parasites detection SOTA models. Three variants of YOLOV4-based object detectors are adopted considering their detection accuracy and speed. These models were trained using tiles generated from 1780 high-resolution P. falciparum-infected thick smear microscopic images. The tiling of high-resolution images improves the performance of the object detection models. The detection accuracy and the generalization capability of these models have been evaluated using three datasets acquired from different regions.ResultsThe best-performing model using the proposed tile-based approach outperforms the baseline method significantly (Recall, [95.3%] vs [57%] and Average Precision, [87.1%] vs [76%]). Furthermore, the proposed method has outperformed the existing approaches that used different machine learning techniques evaluated on similar datasets.ConclusionsThe experimental results show that the proposed method significantly improves P. falciparum detection from thick smear microscopic images while maintaining real-time detection speed. Furthermore, the proposed method has the potential to assist and reduce the workload of laboratory technicians in malaria-endemic remote areas of developing countries where there is a critical skill gap and a shortage of experts.

ODRP: a new approach for spatial street sign detection from EXIF using deep learning-based object detection, distance estimation, rotation and projection system

ODRP: a new approach for spatial street sign detection from EXIF using deep learning-based object detection, distance estimation, rotation and projection system