Cas-OVD: Cascaded Open-Vocabulary Detection of Small Objects Using Multi-Refined Region Proposal Network in Autonomous Driving

The accurate detection of small objects remains a critical challenge in autonomous driving systems, where improving detection performance typically comes at the cost of increased model complexity, conflicting with the lightweight requirements of edge deployment. To address this dilemma, this paper proposes LEAD-YOLO (Lightweight Efficient Autonomous Driving YOLO), an enhanced network architecture based on YOLOv11n that achieves superior small object detection while maintaining computational efficiency. The proposed framework incorporates three innovative components: First, the Backbone integrates a lightweight Convolutional Gated Transformer (CGF) module, which employs normalized gating mechanisms with residual connections, and a Dilated Feature Fusion (DFF) structure that enables progressive multi-scale context modeling through dilated convolutions. These components synergistically enhance small object perception and environmental context understanding without compromising network efficiency. Second, the neck features a hierarchical feature fusion module (HFFM) that establishes guided feature aggregation paths through hierarchical structuring, facilitating collaborative modeling between local structural information and global semantics for robust multi-scale object detection in complex traffic scenarios. Third, the head implements a shared feature detection head (SFDH) structure, incorporating shared convolution modules for efficient cross-scale feature sharing and detail enhancement branches for improved texture and edge modeling. Extensive experiments validate the effectiveness of LEAD-YOLO: on the nuImages dataset, the method achieves 3.8% and 5.4% improvements in mAP@0.5 and mAP@[0.5:0.95], respectively, while reducing parameters by 24.1%. On the VisDrone2019 dataset, performance gains reach 7.9% and 6.4% for corresponding metrics. These findings demonstrate that LEAD-YOLO achieves an excellent balance between detection accuracy and model efficiency, thereby showcasing substantial potential for applications in autonomous driving.

Arguably the most representative application of artificial intelligence, autonomous driving systems usually rely on computer vision techniques to detect the situations of the external environment. Object detection underpins the ability of scene understanding in such systems. However, existing object detection algorithms often behave as a black box, so when a model fails, no information is available on When, Where and How the failure happened. In this paper, we propose a visual analytics approach to help model developers interpret the model failures. The system includes the micro- and macro-interpreting modules to address the interpretability problem of object detection in autonomous driving. The micro-interpreting module extracts and visualizes the features of a convolutional neural network (CNN) algorithm with density maps, while the macro-interpreting module provides spatial-temporal information of an autonomous driving vehicle and its environment. With the situation awareness of the spatial, temporal and neural network information, our system facilitates the understanding of the results of object detection algorithms, and helps the model developers better understand, tune and develop the models. We use real-world autonomous driving data to perform case studies by involving domain experts in computer vision and autonomous driving to evaluate our system. The results from our interviews with them show the effectiveness of our approach.

Many successful deep learning networks for object detection have been proposed in recent years, but their direct comparison is difficult due to different resolutions, training and validation data and frameworks used to train these networks. Moreover, suitability of these object detectors in context of autonomous driving is not compared or studied extensively. Autonomous driving has a wide operational domain containing diverse open world scenarios like different weather, daylight and road conditions etc. Most of the object detection datasets are from specific operational domains, e.g. KITTI dataset is mostly urban scenarios. In this work, we aim to first club many open source datasets (simulated and real) available for object detection in autonomous driving scenario. Clubbing the datasets makes them diverse by including different weather, lighting, time of the day and road conditions. Secondly, by clubbing most of the publicly available object detection datasets, we show the generalization ability of object detection networks in diverse scenarios. We also provide comparative study of accuracy vs resolution and the results of DNN based object detection on real images by training only on simulated data. Additionally, range of detection is really important for autonomous driving basic functionality like highway driving, and market trend is to move towards higher resolution images, yet most of the well known DNN based object detectors operate at very small image resolution which restricts range of detection. We demonstrate that the use of ROI instead of entire image can help in increasing detection range. Finally, in the context of autonomous driving these DNNs needs to run on low powered Edge devices. We measure the deployability of these DNNs by measuring their run time performance on Nvidia Drive PX2.

<div class="section abstract"><div class="htmlview paragraph">Object detection (OD) is one of the most important aspects in Autonomous Driving (AD) application. This depends on the strategic sensor’s selection and placement of sensors around the vehicle. The sensors should be selected based on various constraints such as range, use-case, and cost limitation. This paper introduces a systematic approach for identifying the optimal practices for selecting sensors in AD object detection, offering guidance for those looking to expand their expertise in this field and select the most suitable sensors accordingly. In general, object detection typically involves utilizing RADAR, LiDAR, and cameras. RADAR excels in accurately measuring longitudinal distances over both long and short ranges, but its accuracy in lateral distances is limited. LiDAR is known for its ability to provide accurate range data, but it struggles to identify objects in various weather conditions. On the other hand, camera-based systems offer superior recognition capabilities but lack the precision in range resolution. Fusion of all the three sensors could improve object detection results, however at a higher cost and may be redundant in some cases. In autonomous driving, different functions like dynamic fusion, static fusion, and road model are used to detect a variety of objects like vehicles, motorcycles, guardrails, and road lanes. The paper presents an in-depth analysis of the mechanisms of each sensor, the nature of the data it generates, its level of accuracy, and the limitations it encounters in detecting various objects. For each object, the paper outlines important steps and recommendations that can be implemented to achieve optimal results. This paper elucidates a framework for multi-sensor fusion in object detection, demonstrating superior performance through a practical use case. Model output is rigorously validated against ground truth data using proven devices. The proposed methodology yields demonstrably improved and refined obstacle and environment classification.</div></div>

With autonomous driving developing in a booming stage, accurate object detection in complex scenarios attract wide attention to ensure the safety of autonomous driving. Millimeter wave (mmWave) radar and vision fusion is a mainstream solution for accurate obstacle detection. This article presents a detailed survey on mmWave radar and vision fusion based obstacle detection methods. First, we introduce the tasks, evaluation criteria, and datasets of object detection for autonomous driving. The process of mmWave radar and vision fusion is then divided into three parts: sensor deployment, sensor calibration, and sensor fusion, which are reviewed comprehensively. Specifically, we classify the fusion methods into data level, decision level, and feature level fusion methods. In addition, we introduce three-dimensional(3D) object detection, the fusion of lidar and vision in autonomous driving and multimodal information fusion, which are promising for the future. Finally, we summarize this article.

With the rapid development and popularization of autonomous driving, object detection technology has become increasingly crucial in practical applications and is now a research hotspot in the field. As a core component of the autonomous driving perception system, object detection is responsible for real-time identification of critical environmental elements such as vehicles, pedestrians, and obstacles, providing accurate environmental perception input for subsequent path planning and decision-making control modules. Based on years of research results from scholars at home and abroad, this review systematically discusses object detection technology in autonomous driving. It first elaborates on the research background and significance of object detection in autonomous driving. Secondly, it deeply analyzes the principles of various common algorithms from the two core directions of 2D and 3D detection, comparing their advantages and disadvantages. Then, it summarizes representative datasets. Finally, it addresses current challenges in object detection technology, forecasts future research directions in autonomous driving, and proposes practical solutions.

Power inspection in low-illuminance environments is of great significance for ensuring the all-weather stable operation of the power system. However, low visibility at night seriously interferes with the detection performance of small-sized power devices. In response to the issue, we propose a small object real-time detection method for power line inspection in low-illuminance environments. We design an adaptive transformer-ISP (ATISP) module, in which the optimal parameter regression module generates hyperparameters by sensing input image features to guide the image signal processors (ISPs) to perform image enhancement. With the advantage of ISPs, the ATISP has the advantages of fast inference speed and less training cost. Furthermore, the optimal parameter regression module extracts local features and long-distance dependencies through CNN and Transformer to be able to more fully perceive the input image, so that the generated hyperparameters better enhance image defects. In addition, we use lightweight neural network MobileNetv3 to improve YOLOv7, so that the algorithm maintains excellent small object detection performance while significantly increasing the detection speed. Moreover, the integrated model optimisation uses only the object detection loss functions, which allows ATISP to perform image enhancement just according to the object detection needs, improving small object detection effect and shortening the inference time of ATISP. In extensive experiments, compared with 9 state-of-the-art object detection algorithms, our algorithm has the best small-scale insulator faults detection precision (mAP:75.38 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> ) in our DIFE, best small object detection precision (mAP:56.31 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> ) in public dataset Exdark, and faster detection speed (FPS:98.81 and 97.53), which prove our method can achieve fast and accurate low-illuminance insulators detection.

Recently outstanding object detection results are achieved by the faster region-based convolutional network (faster R-CNN). Particularly, high-quality detection proposals are obtained by the region proposal network (RPN). Nevertheless, part of the parameters in RPN is assigned by prior knowledge. Therefore the underfitting problem is likely to appear on the training model of RPN. In other words, the generalization ability of RPN is not enough. Increasing parameters is an effective solution to this problem. Thereupon a strengthened RPN (SRPN) is designed to expand the exploring space of RPN. Acquiring the optimal parameter values of SRPN is a non-deterministic polynomial-time hard problem, which can be solved by swarm intelligence algorithms. Thereafter a particle swarm optimization (PSO) and bacterial foraging optimization (BFO)-based learning strategy (PBLS) is introduced to optimize the classifier and loss function of SRPN. In SRPN, a novel multi-level extracting network is created to improve the feature sampling ability. Moreover, the mathematical model of the smooth L1 loss function is improved to boost the fitting ability. Additionally, support vector machine (SVM) method is applied to enhance the classifier learning capability. PBLS is applied to SRPN (PBLS_SRPN). The parameters of SVM and the improved loss function are optimized by the BFO and PSO methods, respectively. Then, the performance of SRPN is further promoted. The excellent results are obtained by our proposed methods on PASCAL VOC 2007, 2012, MS COCO, and KITTI data sets. Consequently, PBLS_SRPN is effective for object detection in autonomous driving.

Object detection technology is a core component of autonomous driving systems, primarily tasked with the real-time identification and localization of objects in the surrounding environment, such as pedestrians, vehicles, and traffic signs. With the advancement of deep learning, various efficient object detection algorithms have emerged in recent years, such as the YOLO series, RetinaNet, and CenterNet. These algorithms have shown significant improvements in both processing speed and accuracy. However, the complexity and variability of autonomous driving environments still pose severe challenges to object detection. This review summarizes the current research status of object detection algorithms in autonomous driving, focusing specifically on the basic processes and characteristics of YOLO, RetinaNet, and CenterNet. It also discusses their improvements and optimization effects in practical applications. Additionally, this paper reveals the advantages and disadvantages of these three algorithms through a comparative analysis of experimental results, providing direction for future research. Finally, the paper discusses the main challenges faced by current object detection methods in the autonomous driving field and potential solutions, aiming to promote further development and application in this area.

An accurate and robust perception system is key to understanding the driving environment of autonomous driving and robots. Autonomous driving needs 3-D information about objects, including the object’s location and pose, to understand the driving environment clearly. A camera sensor is widely used in autonomous driving because of its richness in color and texture, and low price. The major problem with the camera is the lack of 3-D information, which is necessary to understand the 3-D driving environment. In addition, the object’s scale change and occlusion make 3-D object detection more challenging. Many deep learning-based methods, such as depth estimation, have been developed to solve the lack of 3-D information. This survey presents the image 3-D object detection 3-D bounding box encoding techniques and evaluation metrics. The image-based methods are categorized based on the technique used to estimate an image’s depth information, and insights are added to each method. Then, state-of-the-art (SOTA) monocular and stereo camera-based methods are summarized. We also compare the performance of the selected 3-D object detection models and present challenges and future directions in 3-D object detection.

Self-driving cars are the latest innovation in which the car runs by itself. The self-driving cars can be called as autonomous cars. This involves many technologies like artificial intelligence, machine learning and deep learning. When coming to the self-driving cars, the main aspect which it needed to be taken care of is the obstacle detection and the object detection. The object detection by a car in more simpler words object recognition process done by a machine which involves the concepts of machine learning and deep learning. Deep learning helps in achieving the object and the obstacle detection. There are various algorithms which help in the object detection like artificial neural network, convolutional neural network, AlexNet, VGG Net, GoogleNet, etc. GoogleNet is the CNN architecture which makes the image recognition an easier task. For the self-driving cars, obstacle and object detection GoogLeNet are not much addressed in the recent works. So, it can be considered as a latest technology. In this paper, the recent works about the self-driving cars and object detection and obstacle detection and the future scope of it are discussed.

ABSTRACTObject detection is a critical aspect of computer vision (CV) applications, especially within autonomous driving systems (AVs), where it is fundamental to ensuring safety and reducing traffic accidents. Recent advancements in computational resources have enabled the widespread adoption of Deep Learning (DL) techniques, significantly enhancing the efficiency and accuracy of object detection tasks. However, the technology for autonomous driving has yet to reach a level of maturity that guarantees consistent performance, reliability, and safety, with several challenges remaining unresolved. This study specifically focuses on 2D image‐based object detection methods, which offer several advantages over other modalities, such as cost‐effectiveness and the ability to capture visual features like colour and texture that are not detectable by LiDAR. We provide a comprehensive survey of DL‐based strategies for detecting vehicles and pedestrians using 2D images, analysing both one‐stage and two‐stage detection frameworks. Additionally, we review the most commonly used publicly available datasets in autonomous driving research and highlight their relevance to 2D detection tasks. The paper concludes by discussing the current challenges in this domain and proposing potential future directions, aiming to bridge the gap between the capabilities of 2D image‐based models and the requirements of real‐world autonomous driving applications. Comparative tables are included to facilitate a clear understanding of the different approaches and datasets.

Capturing uncertainty in object detection is indispensable for safe autonomous driving. In recent years, deep learning has become the de-facto approach for object detection, and many probabilistic object detectors have been proposed. However, there is no summary on uncertainty estimation in deep object detection, and existing methods are either built with different network architectures and uncertainty estimation methods, or evaluated on different datasets with a wide range of evaluation metrics. As a result, a comparison among methods remains challenging, as does the selection of a model that best suits a particular application. This paper aims to alleviate this problem by providing a review and comparative study on existing probabilistic object detection methods for autonomous driving applications. First, we provide an overview of practical uncertainty estimation methods in deep learning, and then systematically survey existing methods and evaluation metrics for probabilistic object detection. Next, we present a strict comparative study for probabilistic object detection based on an image detector and three public autonomous driving datasets. Finally, we present a discussion of the remaining challenges and future works. Code has been made available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/asharakeh/pod_compare.git</uri> .

Object detection, a more advanced application of computer vision than image classification, utilizes deep neural networks to predict objects in an input image and determine their locations through bounding boxes. The field of artificial intelligence has increasingly focused on the demands of autonomous driving, which require both high accuracy and fast inference speeds. This research paper aims to address this demand by introducing an efficient lightweight network for object detection specifically designed for self-driving vehicles. The proposed network, named MobDet3, incorporates a modified MobileNetV3 as its backbone, leveraging its lightweight convolutional neural network algorithm to extract and aggregate image features. Furthermore, the network integrates altered techniques in computer vision and adjusts to the most recent iteration of the PyTorch framework. The MobDet3 network enhances not only object positioning ability but also the reusability of feature maps across different scales. Extensive evaluations were conducted to assess the effectiveness of the proposed network, utilizing an autonomous driving dataset, as well as large-scale everyday human and object datasets. These evaluations were performed on NXP BlueBox 2.0, an advanced edge development platform designed for autonomous vehicles. The results demonstrate that the proposed lightweight object detection network achieves a mean precision of up to 58.30% on the BDD100K dataset and a high inference speed of up to 88.92 frames per second on NXP BlueBox 2.0, making it well-suited for real-time object detection in autonomous driving applications.

Recently, autonomous driving technology has been in the spotlight. However, autonomous driving is still in its infancy in the railway industry. In the case of railways, there are fewer control elements than autonomous driving of cars due to the characteristics of running on railways, but there is a disadvantage in that evasive maneuvers cannot be made in the event of a dangerous situation. In addition, when braking, it cannot be decelerated quickly for the weight of the body and the safety of the passengers. In the case of a tram, one of the railway systems, research has already been conducted on how to generate a profile that plans braking and acceleration as a base technology for autonomous driving, and to find the location coordinates of surrounding objects through object recognition. In pilot research about the tram's automated driving, YOLOv3 was used for object detection to find object coordinates. YOLOv3 is an artificial intelligence model that finds coordinates, sizes, and classes of objects in an image. YOLOv3 is the third upgrade of YOLO, which is one of the most famous object detection technologies based on CNN. YOLO's object detection performance is characterized by ordinary accuracy and fast speed. For this paper, we conducted a study to find out whether the object detection performance required for autonomous trams can be sufficiently implemented with the already developed object detection model. For this experiment, we used the YOLOv4 which is the fourth upgrade of YOLO.