Deep Learning-based Object Detection Research Articles

Unveiling Urbanization Effects on Trees Outside Forests Along the Urban-Rural Gradient in Megacity Bengaluru

Rapid urbanization has caused significant changes along the urban-rural gradient, leading to a variety of landscapes that are mainly shaped by human activities. This dynamic interplay also influences the distribution and characteristics of trees outside forests (TOF). Understanding the pattern of these trees will support informed decision-making in urban planning, in conservation strategies, and altogether in sustainable land management practices in the urban context. In this study, we employed a deep learning-based object detection model and high resolution satellite imagery to identify 1.3 million trees with bounding boxes within a 250 km2 research transect spanning the urban-rural gradient of Bengaluru, a megacity in Southern India. Additionally, we developed an allometric equation to estimate diameter at breast height (DBH) from the tree crown diameter (CD) derived from the detected bounding boxes. Our study focused on analyzing variations in tree density and tree size along this gradient. The findings revealed distinct patterns: urban domains displayed larger tree crown diameters (mean: 8.87 m) and DBH (mean: 43.78 cm) but having relatively low tree density (32 trees per hectare). Furthermore, with increasing distance from the city center, tree density increased, while the mean tree crown diameter and mean tree basal area decreased, showing clear differences of tree density and size between urban and rural domains in Bengaluru. This study offers an efficient methodology that helps generating instructive insights into the dynamics of TOF along the urban-rural gradient. This may inform urban planning and management strategies for enhancing green infrastructure and biodiversity conservation in rapidly urbanizing cities like Bengaluru.

YOLOv8-E: An Improved YOLOv8 Algorithm for Eggplant Disease Detection

During the developmental stages, eggplants are susceptible to diseases, which can impact crop yields and farmers’ economic returns. Therefore, timely and effective detection of eggplant diseases is crucial. Deep learning-based object detection algorithms can automatically extract features from images of eggplants affected by diseases. However, eggplant disease images captured in complex farmland environments present challenges such as varying disease sizes, occlusion, overlap, and small target detection, making it difficult for existing deep-learning models to achieve satisfactory detection performance. To address this challenge, this study proposed an optimized eggplant disease detection algorithm, YOLOv8-E, based on You Only Look Once version 8 nano (YOLOv8n). Firstly, we integrate switchable atrous convolution (SAConv) into the C2f module to design the C2f_SAConv module, replacing some of the C2f modules in the backbone network of YOLOv8n, enabling our proposed algorithm to better extract eggplant disease features. Secondly, to facilitate the deployment of the detection model on mobile devices, we reconstruct the Neck network of YOLOv8n using the SlimNeck module, making the model lighter. Additionally, to tackle the issue of missing small targets, we embed the large separable kernel attention (LSKA) module within SlimNeck, enhancing the model’s attention to fine-grained information. Lastly, we combined intersection over union with auxiliary bounding box (Inner-IoU) and minimum point distance intersection over union (MPDIoU), introducing the Inner-MPDIoU loss to speed up convergence of the model and raise detection precision of overlapped and occluded targets. Ablation studies demonstrated that, compared to YOLOv8n, the mean average precision (mAP) and F1 score of YOLOv8-E reached 79.4% and 75.7%, respectively, which obtained a 5.5% increment and a 4.5% increase, while also reducing the model size and computational complexity. Furthermore, YOLOv8-E achieved higher detection performance than other mainstream algorithms. YOLOv8-E exhibits significant potential for practical application in eggplant disease detection.

Thyroid-DETR: Thyroid nodule detection model with transformer in ultrasound images

Deep learning-based object detection methods using ultrasound images to assist clinicians in rapid and accurate diagnosis of thyroid nodules has been an attractive area of research. Despite the great progress made by object detectors, these may still produce unsatisfactory performance due to certain factors, such as the presence of abundant background information and significant nodule size variations in ultrasound images of thyroid nodules. In this study, we draw on the idea of a Transformer and design a powerful thyroid nodule ultrasound imaging detection model, namely Thyroid-DETR. The proposed Thyroid-DETR has the following features: (i) a deformable convolution v2 is introduced flexibly capture small nodules of irregular shapes and small sizes, (ii) a multi-head self-attention structure is designed to acquire and learn broader correlated information in the shallow layer, (iii) a dual-stream training structure with BatchFormerV2 is developed to fully capture the effective information of medical images; (iv) fine-tuning methods to optimize the model are used to improve the diagnostic accuracy for detecting thyroid nodules. To evaluate the efficacy and effectiveness of the proposed approach. Extensive experiments and comprehensive comparisons are conducted on real thyroid nodule ultrasound images from the hospital. The results show that our method significantly outperforms the state-of-the-art methods.

Intelligent cotter pins defect detection for electrified railway based on improved faster R-CNN and dilated convolution

The cotter pin (CP) is a vital fastener for the catenary support components (CSCs) of high-speed electrified railways. Due to the vibration and excitation caused by the passing of railway vehicles, some CPs may be broken or fallen off over time, which poses a significant safety hazard to the railway systems. Currently, the CP defect detection is primarily conducted by humans, which is inefficient and inconsistent. Therefore, there is an urgent need for automatic CP defect detection to ensure railway safety. However, this task is very challenging as it requires covering hundreds or thousands of miles in limited times when the railway stops running. To this end, we first design a traffic track intelligent imaging device to capture catenary images at various angles at high speed. Then, inspired by the success of deep learning-based object detection, we develop a CP detection model based on an improved Faster R-CNN with a multi-scale region proposal network (MS-RPN) and propose the positive sample adaptive loss function (PSALF) to enhance detection accuracy. Finally, we propose a module to recognize the CP defect based on dilated convolution. The experimental results show that our method can effectively detect the CP defect in the catenary image, achieving 99.05 % precision and 98.40 % recall rate on CP defect detection. Furthermore, CP detection method and CP defect detection are significantly faster than baseline method, with FPS improvements of 2.76 and 24.67, respectively, thus making it more suitable for real-time applications in railway systems.

An unmanned aerial system benchmark object detection dataset for deep learning in outfall surveys

ABSTRACT Deep-learning-based object detection in UAS imagery is crucial for outfall surveys and basin environmental protection. Comprehensive UAS image datasets serve as a foundation for creating deep-learning-based outfall-detection models and evaluating outfall-detection algorithms. However, existing remote sensing deep learning datasets lack specific outfall data. This study introduces a benchmark UAS image dataset of outfalls, categorized into three main and seven subcategories. Over 10,000 labeled images were collected from UAS images with a 10 cm resolution for the Yangtze River, Yellow River, and other basins from 2019 to 2022, covering nearly all types of outfalls in China. Each sample was matched with digital surface model (DSM) data generated through photographometry or the DSM transfer method. Evaluation with seven widely used deep learning-based object detection algorithms demonstrated the dataset's viability, achieving an average precision ( A P 50 ) of 64.6, surpassing performance on Microsoft Common Objects in Context. Further experiments indicated that DSMs attached to this dataset could benefit from geo-deep-learning-based object detection algorithms for outfalls, achieving an A P 50 exceeding 70. This study presents a UAS benchmark image dataset for outfall surveys, potentially advancing the application of deep learning in environmental protection.

Aerial Map-Based Navigation by Ground Object Pattern Matching

This paper proposes a novel approach to map-based navigation for unmanned aircraft. The proposed approach employs pattern matching of ground objects, not feature-to-feature or image-to-image matching, between an aerial image and a map database. Deep learning-based object detection converts the ground objects into labeled points, and the objects’ configuration is used to find the corresponding location in the map database. Using the deep learning technique as a tool for extracting high-level features reduces the image-based localization problem to a pattern-matching problem. The pattern-matching algorithm proposed in this paper does not require altitude information or a camera model to estimate the horizontal geographical coordinates of the vehicle. Moreover, it requires significantly less storage because the map database is represented as a set of tuples, each consisting of a label, latitude, and longitude. Probabilistic data fusion with the inertial measurements by the Kalman filter is incorporated to deliver a comprehensive navigational solution. Flight experiments demonstrate the effectiveness of the proposed system in real-world environments. The map-based navigation system successfully provides the position estimates with RMSEs within 3.5 m at heights over 90 m without the aid of the GNSS.

CubeSats for space debris removal from LEO: Prototype design of a robotic arm-based deorbiter CubeSat

Space debris has become a growing problem, and various studies indicate that the population of space objects has reached a critical density in low Earth orbits (LEO). Therefore, removing debris from the LEO is key to stabilizing the growth of space objects and making some of the orbits reusable for future launches. Deorbiting larger debris objects using miniaturized satellites, CubeSats is both economically viable and requires relatively less time for design & deployment. In this paper, we present a system-level study of orbital debris removal from LEO using CubeSats, detailing various existing debris tracking, rendezvous, capture, and deorbit mechanisms. A deep learning-based object detection algorithm has been utilized to improve the accuracy of detecting and localizing debris. We also present a discussion on the CubeSat-compatible capture & deorbit mechanisms identified from our study and an upper bound on the debris mass that can be deorbited from a given LEO orbit using CubeSats. We aim to demonstrate a case study for the orbital decay analysis of a CubeSat deorbiting the XSat, a microsatellite launched by Singapore. We detail the design of a 27U CubeSat deorbiter prototype equipped with a pair of robotic arms to capture the XSat. This analysis considers active deorbiting using Hall-effect thrusters, for which the required delta-V & propellant mass have also been computed.

A robust method for multi object tracking in autonomous ship navigation systems

A novel Maritime Multi-Object Tracking method is proposed, combining a deep learning-based object detector with target association algorithms to achieve robust sea-surface multi-object tracking. Specifically, the Multi-Object Tracking employs You Only Look Once version 7 detector for object detection. In the data association part, a module for onboard camera motion compensation is developed, a maritime dynamic spatial information-based intersection-over-union is presented as a similarity metric, and a progressive refinement cascade matching strategy is designed to enhance the tracker's sea-surface multi-object tracking capabilities. The Jari Maritime Tracking Dataset is utilised to validate the effectiveness performance of the proposed method. Experimental results demonstrate that compared to the earlier process, the proposed method exhibits a significant enhancement in multiple object tracking accuracy, with an increase of 27.8% and an achieved score of 81.3. In particular, it reduces the number of identifications switching and missed targets, achieving holistically preferable performance. Meanwhile, the speed of the proposed Multi-Object Tracking fulfils the engineering application requirements for an autonomous ship navigation system.

Optimal deep learning based object detection for pedestrian and anomaly recognition model

Optimal deep learning based object detection for pedestrian and anomaly recognition model

Robotic Grasping of Unknown Objects Based on Deep Learning-Based Feature Detection.

In recent years, the integration of deep learning into robotic grasping algorithms has led to significant advancements in this field. However, one of the challenges faced by many existing deep learning-based grasping algorithms is their reliance on extensive training data, which makes them less effective when encountering unknown objects not present in the training dataset. This paper presents a simple and effective grasping algorithm that addresses this challenge through the utilization of a deep learning-based object detector, focusing on oriented detection of key features shared among most objects, namely straight edges and corners. By integrating these features with information obtained through image segmentation, the proposed algorithm can logically deduce a grasping pose without being limited by the size of the training dataset. Experimental results on actual robotic grasping of unknown objects over 400 trials show that the proposed method can achieve a higher grasp success rate of 98.25% compared to existing methods.

Detection of Threats to Farm Animals Using Deep Learning Models: A Comparative Study

The increasing global population and environmental changes pose significant challenges to food security and sustainable agricultural practices. To overcome these challenges, protecting farm animals and effectively detecting potential environmental threats is critical for economic and ecological sustainability. In this context, the current study examined the animal detection capabilities and efficiency of advanced deep learning models, such as YOLOv8, Yolo-NAS, and Fast-RNN, across a dataset of 2462 images encompassing various animal species that could pose a risk to farm animals. After converting the images into a standardized format, they were divided into three sets for training, validation, and testing, and each model was evaluated on this dataset during the analysis process. The findings indicated that the YOLOv8 model demonstrated superior performance, with 93% precision, 85.2% recall, and 93.1% mAP50 values, while Yolo-NAS was particularly noteworthy for its high recall value, indicating a remarkable detection ability. The Fast-RNN model also offered significant efficiency with balanced performance. The results reveal the considerable potential of deep learning-based object detection technologies in protecting farm animals and enhancing farm security. Additionally, this study provides valuable insights for future model optimization and customization research.

A Neuroinspired Contrast Mechanism enables Few-Shot Object Detection

Deep learning-based object detectors often demand abundant annotated data for training. However, in practice, only limited training data are available, making Few-Shot Object Detection (FSOD) an attractive research topic. Existing two-stage proposal-based Faster R-CNN detectors for FSOD struggle to match the performance of models trained on large datasets. We argue that detectors trained with limited samples cannot establish robust comparison-based relationships. Additionally, FSOD methods only explore these relationships during the training phase. To address these issues, we draw inspiration from neuroscience studies and propose Residual Contrast Faster R-CNN (RcFRCN). RcFRCN incorporates two novel customized contrast blocks: a Residual Spatial Contrast Block and a Residual Proposal Contrast Block. These blocks capture cross-spatial and cross-proposal contrast information, enhancing both training and testing phases. We conduct comprehensive experiments on two FSOD benchmarks: PASCAL VOC and MS-COCO. Our RcFRCN achieves a mAP of 21.9 under a 30-shot setting on MS-COCO. It also achieves AP scores of 69.1, 55.8, and 64.0 under 10-shot settings of different splits on PASCAL VOC, respectively. Moreover, we apply RcFRCN on remote sensing and use our contrast blocks for open-vocabulary detection. Experiment results on these tasks also demonstrate the robustness and generalization ability of our methods.

Object Detection Algorithms Based on Deep Learning: A Review

With the continuous development of deep learning, object detection algorithms based on deep learning have made significant progress in the field of computer vision, widely applied in areas such as autonomous driving, industrial inspection, agriculture, transportation, and medicine. Traditional object detection algorithms face issues such as low detection efficiency and poor robustness. However, deep learning-based object detection algorithms significantly enhance detection accuracy and generalization by learning low-level and high-level image features. This article first introduces traditional object detection algorithms and their existing problems, then elaborates on the main processes, innovations, advantages, disadvantages, and experimental results on datasets of deep learning-based object detection algorithms. It focuses on the development of Two-Stage and One-Stage object detection algorithms, and provides an outlook on the future development of object detection algorithms, discussing challenges such as the coordination of detection speed and accuracy, difficulties in detecting small objects, real-time detection tasks, and multi-modal fusion applications, and proposes possible future directions.

Design and implementation of deep learning-based object detection and tracking system

Many human tracking methods by deep learning rely on powerful computing resources. For embedded platforms with limited resources, efficient use of resources is a priority. In this paper, we design an object detection and tracking system based on deep learning methods. We propose an efficient system with software and hardware design. We apply the framework of Vitis AI and its Deep Learning Processing Unit using a hardware/software co-design approach. This approach capitalizes on a higher-level acceleration design framework, where the convolutional models can be updated more flexibly and rapidly. This design approach not only provides a fast design flow but also has good performance in terms of throughput. We facilitate the design and accelerate the object detection model YOLO v3 to achieve higher throughput and energy efficiency. Our tracking method achieves a 1.27x improvement in processing speed with the addition of a single-object tracker. Our proposed human tracking methods can achieve better performance than the others in precision with the same test sequences.

Morphological Rule-Constrained Object Detection of Key Structures in Infant Fundus Image.

The detection of optic disc and macula is an essential step for ROP (Retinopathy of prematurity) zone segmentation and disease diagnosis. This paper aims to enhance deep learning-based object detection with domain-specific morphological rules. Based on the fundus morphology, we define five morphological rules, i.e., number restriction (maximum number of optic disc and macula is one), size restriction (e.g., optic disc width: 1.05 +/- 0.13 mm), distance restriction (distance between the optic disc and macula/fovea: 4.4 +/- 0.4 mm), angle/slope restriction (optic disc and macula should roughly be positioned in the same horizontal line), position restriction (In OD, the macula is on the left side of the optic disc; vice versa for OS). A case study on 2953 infant fundus images (with 2935 optic disc instances and 2892 macula instances) proves the effectiveness of the proposed method. Without the morphological rules, naïve object detection accuracies of optic disc and macula are 0.955 and 0.719, respectively. With the proposed method, false-positive ROIs (region of interest) are further ruled out, and the accuracy of the macula is raised to 0.811. The IoU (intersection over union) and RCE (relative center error) metrics are also improved .

YOLO-Based Personal Protective Equipment Monitoring System for Workplace Safety

Occupational health and safety are of paramount importance in industrial environments and various work fields. In this context, tracking personal protective equipment (PPE) is highly necessary. This article investigates the performance and application of deep learning-based object detection models to enhance the accuracy and speed of tracking personal protective equipment for ensuring occupational health and safety. These models detect personal protective equipment in images, enabling monitoring of their correct usage and intervention when necessary. The study aims to minimize damage resulting from accidents through the use of protective equipment and to prevent possible accidents. In our study, a dataset consisting of 2581 images, encompassing different workplace environments and workers, was prepared. This dataset was evaluated for performance using deep learning models. Popular deep-learning models such as YOLO-NAS, YOLOv8, and YOLOv9 were utilized in the comparisons. During the training of the models, the number of epochs was kept consistent for fair comparison. Upon examining the results, it is observed that the YOLO-NAS and YOLOv9 models generally exhibit similar and high performance.

A dataset of drilling site object detection in underground coal mines

Drilling in underground coal mine is an important measure for dealing with gas, water and hidden geological disasters, which can significantly enhance the effectiveness of disaster prevention and control in coal mining operations. In order to monitor the drilling process in real time and improve drilling efficiency, it is necessary to carry out object detection to identify and locate key targets at the drilling site. Compared with traditional object detection method, deep learning-based object detection method can improve accuracy, timeliness and stability of object detection, but it requires high-quality datasets to perform well. At present, research on object detection in underground coal mine drilling sites mainly relies on small-scale private datasets, which are insufficient for providing necessary or reliable data for deep neural network model training. In this study, we constructed a dataset of drilling site object detection using photos taken by intrinsic safety law enforcement recorders. This dataset is developed through several steps, including data cleaning, data labeling, and expert sampling verification. The mainstream YOLO series object detection model is used for data quality assessment. This dataset comprises 70,948 images from drilling sites under different environmental conditions, covering five categories of objects: gripper, chuck, coal miner, mine safety helmet, and drill pipe. It provides annotated files in PASCAL VOC format. This dataset can provide strong data support for object detection research in underground coal mine drilling sites, and plays an important role in promoting intelligent underground coal mine monitoring and early warning.

Utilizing improved YOLOv8 based on SPD-BRSA-AFPN for ultrasonic phased array non-destructive testing

Non-destructive testing (NDT) is a technique for inspecting materials and their defects without causing damage to the tested components. Phased array ultrasonic testing (PAUT) has emerged as a hot topic in industrial NDT applications. Currently, the collection of ultrasound data is mostly automated, while the analysis of the data is still predominantly carried out manually. Manual analysis of scan image defects is inefficient and prone to instability, prompting the need for computer-based solutions. Deep learning-based object detection methods have shown promise in addressing such challenges recently. This approach typically demands a substantial amount of high-resolution, well-annotated training data, which is challenging to obtain in NDT. Consequently, it becomes difficult to detect low-resolution images and defects with varying positional sizes. This work proposes improvements based on the state-of-the-art YOLOv8 algorithm to enhance the accuracy and efficiency of defect detection in phased-array ultrasonic testing. The space-to-depth convolution (SPD-Conv) is imported to replace strided convolution, mitigating information loss during convolution operations and improving detection performance on low-resolution images. Additionally, this paper constructs and incorporates the bi-level routing and spatial attention module (BRSA) into the backbone, generating multiscale feature maps with richer details. In the neck section, the original structure is replaced by the asymptotic feature pyramid network (AFPN) to reduce model parameters and computational complexity. After testing on public datasets, in comparison to YOLOv8 (the baseline), this algorithm achieves high-quality detection of flat bottom holes (FBH) and aluminium blocks on the simulated dataset. More importantly, for the challenging-to-detect defect side-drilled holes (SDH), it achieves F1 scores (weighted average of precision and recall) of 82.50% and intersection over union (IOU) of 65.96%, representing an improvement of 17.56% and 0.43%. On the experimental dataset, the F1 score and IOU for FBH reach 75.68% (an increase of 9.01%) and 83.79%, respectively. Simultaneously, the proposed algorithm demonstrates robust performance in the presence of external noise, while maintaining exceptionally high computational efficiency and inference speed. These experimental results validate the high detection performance of the proposed intelligent defect detection algorithm for ultrasonic images, which contributes to the advancement of the smart industry.

Enhancing fall risk assessment: instrumenting vision with deep learning during walks

BackgroundFalls are common in a range of clinical cohorts, where routine risk assessment often comprises subjective visual observation only. Typically, observational assessment involves evaluation of an individual’s gait during scripted walking protocols within a lab to identify deficits that potentially increase fall risk, but subtle deficits may not be (readily) observable. Therefore, objective approaches (e.g., inertial measurement units, IMUs) are useful for quantifying high resolution gait characteristics, enabling more informed fall risk assessment by capturing subtle deficits. However, IMU-based gait instrumentation alone is limited, failing to consider participant behaviour and details within the environment (e.g., obstacles). Video-based eye-tracking glasses may provide additional insight to fall risk, clarifying how people traverse environments based on head and eye movements. Recording head and eye movements can provide insights into how the allocation of visual attention to environmental stimuli influences successful navigation around obstacles. Yet, manual review of video data to evaluate head and eye movements is time-consuming and subjective. An automated approach is needed but none currently exists. This paper proposes a deep learning-based object detection algorithm (VARFA) to instrument vision and video data during walks, complementing instrumented gait.MethodThe approach automatically labels video data captured in a gait lab to assess visual attention and details of the environment. The proposed algorithm uses a YoloV8 model trained on with a novel lab-based dataset.ResultsVARFA achieved excellent evaluation metrics (0.93 mAP50), identifying, and localizing static objects (e.g., obstacles in the walking path) with an average accuracy of 93%. Similarly, a U-NET based track/path segmentation model achieved good metrics (IoU 0.82), suggesting that the predicted tracks (i.e., walking paths) align closely with the actual track, with an overlap of 82%. Notably, both models achieved these metrics while processing at real-time speeds, demonstrating efficiency and effectiveness for pragmatic applications.ConclusionThe instrumented approach improves the efficiency and accuracy of fall risk assessment by evaluating the visual allocation of attention (i.e., information about when and where a person is attending) during navigation, improving the breadth of instrumentation in this area. Use of VARFA to instrument vision could be used to better inform fall risk assessment by providing behaviour and context data to complement instrumented e.g., IMU data during gait tasks. That may have notable (e.g., personalized) rehabilitation implications across a wide range of clinical cohorts where poor gait and increased fall risk are common.

A Data Matrix Code Recognition Method Based on L-Shaped Dashed Edge Localization Using Central Prior.

The recognition of data matrix (DM) codes plays a crucial role in industrial production. Significant progress has been made with existing methods. However, for low-quality images with protrusions and interruptions on the L-shaped solid edge (finder pattern) and the dashed edge (timing pattern) of DM codes in industrial production environments, the recognition accuracy rate of existing methods sharply declines due to a lack of consideration for these interference issues. Therefore, ensuring recognition accuracy in the presence of these interference issues is a highly challenging task. To address such interference issues, unlike most existing methods focused on locating the L-shaped solid edge for DM code recognition, we in this paper propose a novel DM code recognition method based on locating the L-shaped dashed edge by incorporating the prior information of the center of the DM code. Specifically, we first use a deep learning-based object detection method to obtain the center of the DM code. Next, to enhance the accuracy of L-shaped dashed edge localization, we design a two-level screening strategy that combines the general constraints and central constraints. The central constraints fully exploit the prior information of the center of the DM code. Finally, we employ libdmtx to decode the content from the precise position image of the DM code. The image is generated by using the L-shaped dashed edge. Experimental results on various types of DM code datasets demonstrate that the proposed method outperforms the compared methods in terms of recognition accuracy rate and time consumption, thus holding significant practical value in an industrial production environment.