Drivable Area Research Articles

See the Unseen: Grid-Wise Drivable Area Detection Dataset and Network Using LiDAR

Drivable Area (DA) detection is crucial for autonomous driving. Camera-based methods rely heavily on illumination conditions and often fail to capture accurate 3D information, while LiDAR-based methods offer accurate 3D data and are less susceptible to illumination conditions. However, existing LiDAR-based methods focus on point-wise detection, so are prone to occlusion and limited by point cloud sparsity, which leads to decreased performance in motion planning and localization. We propose Argoverse-grid, a grid-wise DA detection dataset derived from Argoverse 1, comprising over 20K frames with fine-grained BEV DA labels across various scenarios. We also introduce Grid-DATrNet, a first grid-wise DA detection model utilizing global attention through transformers. Our experiments demonstrate the superiority of Grid-DATrNet over various methods, including both LiDAR and camera-based approaches, in detecting grid-wise DA on the proposed Argoverse-grid dataset. Grid-DATrNet achieves state-of-the-art results with an accuracy of 93.28% and an F1-score of 0.8328. We show that Grid-DATrNet can detect grids even in occluded and unmeasured areas by leveraging contextual and semantic information through global attention, unlike CNN-based DA detection methods. The preprocessing code for Argoverse-grid, experiment code, Grid-DATrNet implementation, and result visualization code are available at AVE Laboratory official git hub.

SwinLabNet: Jujube Orchard Drivable Area Segmentation Based on Lightweight CNN-Transformer Architecture

Identifying drivable areas between orchard rows is crucial for intelligent agricultural equipment. However, challenges remain in this field’s accuracy, real-time performance, and generalization of deep learning models. This study proposed the SwinLabNet model in the context of jujube orchards, an innovative network model that utilized a lightweight CNN-transformer hybrid architecture. This approach optimized feature extraction and contextual information capture, effectively addressing long-range dependencies, global information acquisition, and detailed boundary processing. After training on the jujube orchard dataset, the SwinLabNet model demonstrated significant performance advantages: training accuracy reached 97.24%, the mean Intersection over Union (IoU) was 95.73%, and the recall rate was as high as 98.36%. Furthermore, the model performed exceptionally well on vegetable datasets, highlighting its generalization capability across different crop environments. This study successfully applied the SwinLabNet model in orchard environments, providing essential support for developing intelligent agricultural equipment, advancing the identification of drivable areas between rows, and laying a solid foundation for promoting and applying intelligent agrarian technologies.

When-to-Loop: Enhanced Loop Closure for LiDAR SLAM in Urban Environments Based on SCAN CONTEXT

Global Navigation Satellite Systems (GNSSs) frequently encounter challenges in providing reliable navigation and positioning within urban canyons due to signal obstruction. Micro-Electro-Mechanical System (MEMS) Inertial Measurement Units (IMUs) offers an alternative for autonomous navigation, but they are susceptible to accumulating errors. To mitigate these influences, a LiDAR-based Simultaneous Localization and Mapping (SLAM) system is often employed. However, these systems face challenges in drift and error accumulation over time. This paper presents a novel approach to loop closure detection within LiDAR-based SLAM, focusing on the identification of previously visited locations to correct time-accumulated errors. Specifically, the proposed method leverages the vehicular drivable area and IMU trajectory to identify significant environmental changes in keyframe selection. This approach differs from conventional methods that only rely on distance or time intervals. Furthermore, the proposed method extends the SCAN CONTEXT algorithm. This technique incorporates the overall distribution of point clouds within a region rather than solely relying on maximum height to establish more robust loop closure constraints. Finally, the effectiveness of the proposed method is validated through experiments conducted on the KITTI dataset with an enhanced accuracy of 6%, and the local scenarios exhibit a remarkable improvement in accuracy of 17%, demonstrating improved robustness in loop closure detection for LiDAR-based SLAM.

A Method for All-Weather Unstructured Road Drivable Area Detection Based on Improved Lite-Mobilenetv2

This paper presents an all-weather drivable area detection method based on deep learning, addressing the challenges of recognizing unstructured roads and achieving clear environmental perception under adverse weather conditions in current autonomous driving systems. The method enhances the Lite-Mobilenetv2 feature extraction module and integrates a pyramid pooling module with an attention mechanism. Moreover, it introduces a defogging preprocessing module suitable for real-time detection, which transforms foggy images into clear ones for accurate drivable area detection. The experiments adopt a transfer learning-based training approach, training an all-road-condition semantic segmentation model on four datasets that include both structured and unstructured roads, with and without fog. This strategy reduces computational load and enhances detection accuracy. Experimental results demonstrate a 3.84% efficiency improvement compared to existing algorithms.

Multi-Task Environmental Perception Methods for Autonomous Driving.

In autonomous driving, environmental perception technology often encounters challenges such as false positives, missed detections, and low accuracy, particularly in detecting small objects and complex scenarios. Existing algorithms frequently suffer from issues like feature redundancy, insufficient contextual interaction, and inadequate information fusion, making it difficult to perform multi-task detection and segmentation efficiently. To address these challenges, this paper proposes an end-to-end multi-task environmental perception model named YOLO-Mg, designed to simultaneously perform traffic object detection, lane line detection, and drivable area segmentation. First, a multi-stage gated aggregation network (MogaNet) is employed during the feature extraction process to enhance contextual interaction by improving diversity in the channel dimension, thereby compensating for the limitations of feed-forward neural networks in contextual understanding. Second, to further improve the model's accuracy in detecting objects of various scales, a restructured weighted bidirectional feature pyramid network (BiFPN) is introduced, optimizing cross-level information fusion and enabling the model to handle object detection at different scales more accurately. Finally, the model is equipped with one detection head and two segmentation heads to achieve efficient multi-task environmental perception, ensuring the simultaneous execution of multiple tasks. The experimental results on the BDD100K dataset demonstrate that the model achieves a mean average precision (mAP50) of 81.4% in object detection, an Intersection over Union (IoU) of 28.9% in lane detection, and a mean Intersection over Union (mIoU) of 92.6% in drivable area segmentation. The tests conducted in real-world scenarios show that the model performs effectively, significantly enhancing environmental perception in autonomous driving and laying a solid foundation for safer and more reliable autonomous driving systems.

Road Anomaly Detection with Unknown Scenes Using DifferNet-Based Automatic Labeling Segmentation

Obstacle avoidance is essential for the effective operation of autonomous mobile robots, enabling them to detect and navigate around obstacles in their environment. While deep learning provides significant benefits for autonomous navigation, it typically requires large, accurately labeled datasets, making the data’s preparation and processing time-consuming and labor-intensive. To address this challenge, this study introduces a transfer learning (TL)-based automatic labeling segmentation (ALS) framework. This framework utilizes a pretrained attention-based network, DifferNet, to efficiently perform semantic segmentation tasks on new, unlabeled datasets. DifferNet leverages prior knowledge from the Cityscapes dataset to identify high-entropy areas as road obstacles by analyzing differences between the input and resynthesized images. The resulting road anomaly map was refined using depth information to produce a robust drivable area and map of road anomalies. Several off-the-shelf RGB-D semantic segmentation neural networks were trained using pseudo-labels generated by the ALS framework, with validation conducted on the GMRPD dataset. Experimental results demonstrated that the proposed ALS framework achieved mean precision, mean recall, and mean intersection over union (IoU) rates of 80.31%, 84.42%, and 71.99%, respectively. The ALS framework, through the use of transfer learning and the DifferNet network, offers an efficient solution for semantic segmentation of new, unlabeled datasets, underscoring its potential for improving obstacle avoidance in autonomous mobile robots.

A panoramic driving perception fusion algorithm based on multi-task learning.

With the rapid development of intelligent connected vehicles, there is an increasing demand for hardware facilities and onboard systems of driver assistance systems. Currently, most vehicles are constrained by the hardware resources of onboard systems, which mainly process single-task and single-sensor data. This poses a significant challenge in achieving complex panoramic driving perception technology. While the panoramic driving perception algorithm YOLOP has achieved outstanding performance in multi-task processing, it suffers from poor adaptability of feature map pooling operations and loss of details during downsampling. To address these issues, this paper proposes a panoramic driving perception fusion algorithm based on multi-task learning. The model training involves the introduction of different loss functions and a series of processing steps for lidar point cloud data. Subsequently, the perception information from lidar and vision sensors is fused to achieve synchronized processing of multi-task and multi-sensor data, thereby effectively improving the performance and reliability of the panoramic driving perception system. To evaluate the performance of the proposed algorithm in multi-task processing, the BDD100K dataset is used. The results demonstrate that, compared to the YOLOP model, the multi-task learning network performs better in lane detection, drivable area detection, and vehicle detection tasks. Specifically, the lane detection accuracy improves by 11.6%, the mean Intersection over Union (mIoU) for drivable area detection increases by 2.1%, and the mean Average Precision at 50% IoU (mAP50) for vehicle detection improves by 3.7%.

Research on LiDAR point cloud data transformation method based on weighted altitude difference map

Road surface detection plays a pivotal role in the realm of autonomous vehicle navigation. Contemporary methodologies primarily leverage LiDAR for acquiring three-dimensional data and utilize imagery for chromatic information. However, these approaches encounter significant integration challenges, particularly due to the inherently unstructured nature of 3D point clouds. Addressing this, our novel algorithm, specifically tailored for predicting drivable areas, synergistically combines LiDAR point clouds with bidimensional imagery. Initially, it constructs an altitude discrepancy map via LiDAR, capitalizing on the height uniformity characteristic of planar road surfaces. Subsequently, we introduce an innovative and more efficacious attention mechanism, streamlined for image feature extraction. This mechanism employs adaptive weighting coefficients for the amalgamation of the altitude disparity imagery and two-dimensional image features, thereby facilitating road area delineation within a semantic segmentation framework. Empirical evaluations conducted using the KITTI dataset underscore our methodology’s superior road surface discernment and extraction precision, substantiating the efficacy of our proposed network architecture and data processing paradigms. This research endeavor seeks to propel the advancement of three-dimensional perception technology in the autonomous driving domain.

A Novel Network Framework on Simultaneous Road Segmentation and Vehicle Detection for UAV Aerial Traffic Images.

Unmanned Aerial Vehicle (UAV) aerial sensors are an important means of collecting ground image data. Through the road segmentation and vehicle detection of drivable areas in UAV aerial images, they can be applied to monitoring roads, traffic flow detection, traffic management, etc. As well, they can be integrated with intelligent transportation systems to support the related work of transportation departments. Existing algorithms only realize a single task, while intelligent transportation requires the simultaneous processing of multiple tasks, which cannot meet complex practical needs. However, UAV aerial images have the characteristics of variable road scenes, a large number of small targets, and dense vehicles, which make it difficult to complete the tasks. In response to these issues, we propose to implement road segmentation and on-road vehicle detection tasks in the same framework for UAV aerial images, and we conduct experiments on a self-constructed dataset based on the DroneVehicle dataset. For road segmentation, we propose a new algorithm C-DeepLabV3+. The new algorithm introduces the coordinate attention (CA) module, which can obtain more accurate segmentation target location information and make the segmentation target edges more continuous. Also, the improved algorithm introduces the cascade feature fusion module to prevent the loss of detail information in road segmentation and to obtain better segmentation performance. For vehicle detection, we propose an improved algorithm S-YOLOv5 by adding a parameter-free lightweight attention module SimAM. Finally, the proposed road segmentation-vehicle detection framework is utilized to unite the C-DeepLabV3+ and S-YOLOv5 algorithms for the implementation of the serial tasks. The experimental results show that on the constructed ViDroneVehicle dataset, the C-DeepLabV3+ algorithm has an mPA value of 98.75% and an mIoU value of 97.53%, which can better segment the road area and solve the problem of occlusion. The mAP value of the S-YOLOv5 algorithm has an mAP value of 97.40%, which is more than YOLOv5's 96.95%, which effectively reduces the vehicle omission and false detection rates. By comparison, the results of both algorithms are superior to multiple state-of-the-art methods. The overall framework proposed in this paper has superior performance and is capable of realizing high-quality and high-precision road segmentation and vehicle detection from UAV aerial images.

Multi-Task Visual Perception for Object Detection and Semantic Segmentation in Intelligent Driving

With the rapid development of intelligent driving vehicles, multi-task visual perception based on deep learning emerges as a key technological pathway toward safe vehicle navigation in real traffic scenarios. However, due to the high-precision and high-efficiency requirements of intelligent driving vehicles in practical driving environments, multi-task visual perception remains a challenging task. Existing methods typically adopt effective multi-task learning networks to concurrently handle multiple tasks. Despite the fact that they obtain remarkable achievements, better performance can be achieved through tackling existing problems like underutilized high-resolution features and underexploited non-local contextual dependencies. In this work, we propose YOLOPv3, an efficient anchor-based multi-task visual perception network capable of handling traffic object detection, drivable area segmentation, and lane detection simultaneously. Compared to prior works, we make essential improvements. On the one hand, we propose architecture enhancements that can utilize multi-scale high-resolution features and non-local contextual dependencies for improving network performance. On the other hand, we propose optimization improvements aiming at enhancing network training, enabling our YOLOPv3 to achieve optimal performance via straightforward end-to-end training. The experimental results on the BDD100K dataset demonstrate that YOLOPv3 sets a new state of the art (SOTA): 96.9% recall and 84.3% mAP50 in traffic object detection, 93.2% mIoU in drivable area segmentation, and 88.3% accuracy and 28.0% IoU in lane detection. In addition, YOLOPv3 maintains competitive inference speed against the lightweight YOLOP. Thus, YOLOPv3 stands as a robust solution for handling multi-task visual perception problems. The code and trained models have been released on GitHub.

Computer Vision-Based Lane Detection and Detection of Vehicle, Traffic Sign, Pedestrian Using YOLOv5

There has been a global increase in the number of vehicles in use, resulting in a higher occurrence of traffic accidents. Advancements in computer vision and deep learning enable vehicles to independently perceive and navigate their environment, making decisions that enhance road safety and reduce traffic accidents. Worldwide accidents can be prevented in both driver-operated and autonomous vehicles by detecting living and inanimate objects such as vehicles, pedestrians, animals, and traffic signs in the environment, as well as identifying lanes and obstacles. In our proposed system, road images are captured using a camera positioned behind the front windshield of the vehicle. Computer vision techniques are employed to detect straight or curved lanes in the captured images. The right and left lanes within the driving area of the vehicle are identified, and the drivable area of the vehicle is highlighted with a different color. To detect traffic signs, pedestrians, cars, and bicycles around the vehicle, we utilize the YOLOv5 model, which is based on Convolutional Neural Networks. We use a combination of study-specific images and the GRAZ dataset in our research. In the object detection study, which involves 10 different objects, we evaluate the performance of five different versions of the YOLOv5 model. Our evaluation metrics include precision, recall, precision-recall curves, F1 score, and mean average precision. The experimental results clearly demonstrate the effectiveness of our proposed lane detection and object detection method.

Segment all roads: Domain generalized freespace detection by robust surface normal information embedding and edge-aware learning

Freespace detection (FD), also known as drivable area detection, plays a crucial role in autonomous driving. However, existing FD methods, which are based on supervised learning, struggle to perform well when testing on out-of-distribution data. Domain generalization is a prospective solution to address this issue, aiming to improve the generalization ability on out-of-distribution data (unseen domains). In this paper, we explore a new task, domain generalized freespace detection, and design a novel approach, which can generalize to both structured urban on-road scenes and unstructured off-road scenes. First, considering the robustness of surface normal (SN) information in diverse domains, we present a robust information embedding module to embed SN information into RGB features. Then, a cross-scale feature robustness enhancement module is introduced to aggregate RGB-SN features across different scales to obtain more domain-invariant features. Moreover, an RGB-SN edge-aware learning strategy is devised to further improve the generalization ability. Extensive experiments demonstrate that the proposed approach achieves superior performance compared to current approaches.

Lidar-based classification and detection system for drivable area on roads

Lidar-based classification and detection system for drivable area on roads

Minimum decision of collision injury based on drivable area in unavoidable scene

The existing technology for assisted driving collision avoidance has improved the active safety of vehicles, but it is primarily designed for scenarios where collisions can be avoided. It is not suitable for situations where pedestrians suddenly intrude. Therefore, there is a need to develop a control strategy that can minimize collision injuries in unavoidable situations. This study focuses on typical collision scenarios where pedestrians suddenly intrude. It proposes a real-time online double-layer optimal pre-collision control strategy based on drivable area, using the Head Injury Criteria ( HIC) as a quantifiable measure of collision injury. The upper-layer controller designs a cost function based on a three Degrees of Freedom (DOF) collision solving model to minimize collision injuries. It takes the constraints of vehicle dynamics, maximum front wheel steering angle, pedestrian avoidance into account, and the optimal pre-impact state point with the minimum HIC value is calculated in real-time. Meanwhile, the lower-layer controller is designed to track the optimal pre-impact state point using Model Predict Control (MPC) algorithm. The effectiveness of the proposed control strategy is verified through Simulink simulation, achieving the dual objectives of avoiding pedestrians suddenly intruding and minimizing passenger collision injuries.

Drivable Area Detection: a Comparative Study of Algorithms Based on Deep Learning

Early techniques, frequently based on computer vision, classified drivable zones into four types: sensor-based, rule-based, map-based, and traditional machine learning (ML). Due to the limitations of these methods, they were later replaced by deep learning-based methods. The advancements in methods for segmenting drivable areas, encompassing both conventional computer vision-based approaches and deep learning-based algorithms are discussed in this article. This paper aims to conduct comparative research on the principles underlying Fast-SCNN, EDANet, and D3NET algorithms along with their performances on the CityScapes dataset. The accuracy and efficiency of the drivable area segmentation task deserve significant attention. Furthermore, a comparative study is conducted separately on PC and embedded platforms to facilitate readers in selecting appropriate datasets and validation platforms. Lastly, the existing challenges as well as future trends pertaining to these algorithms are discussed. This study's comparisons offer valuable insights into the strengths and limitations of the reviewed methods, thereby aiding in the construction of the algorithm model.

Research on multi-task perception network of traffic scene based on feature fusion1

Aiming at the problems of low precision and low real-time performance when deploying to embedded platforms in existing multi-task networks, this paper proposes a traffic scene multi-task perception network model (ETS_YOLOP) based on feature fusion. Firstly, an Efficient Attention Control Aggregation Network Module (EACAN) is constructed to improve the real-time perception of the model, and the Space Pyramid Pool Fast Convolutional Module (SPPFCSPC) is used at the end of the backbone network to increase the receptive field. Finally, a Multiscale Convolution Transformer Fusion Module (CTFM) is designed in the task branch to better capture global information and rich context information. The experimental results show that compared with the YOLOP model, the ETS_YOLOP model has a significant improvement in perception accuracy, 156% in real-time performance, 0.4% increase in mAP on the object detection task, 0.5% increase in mIoU on the drivable area segmentation task, and an 11.4% increase in accuracy on the lane detection task. In order to verify the real-time perception of the model on the embedded platform, the ETS_YOLOP model is deployed on the Huawei MDC300F computing platform. Under the condition of the image input size of 640×640, the average frame rate can reach 55FPS, which can realize real-time perception on the embedded platform.

IDS-MODEL: An Efficient Multitask Model of Road Scene Instance and Drivable Area Segmentation for Autonomous Driving

IDS-MODEL: An Efficient Multitask Model of Road Scene Instance and Drivable Area Segmentation for Autonomous Driving

A Multi-Task Network Based on Dual-Neck Structure for Autonomous Driving Perception.

A vision-based autonomous driving perception system necessitates the accomplishment of a suite of tasks, including vehicle detection, drivable area segmentation, and lane line segmentation. In light of the limited computational resources available, multi-task learning has emerged as the preeminent methodology for crafting such systems. In this article, we introduce a highly efficient end-to-end multi-task learning model that showcases promising performance on all fronts. Our approach entails the development of a reliable feature extraction network by introducing a feature extraction module called C2SPD. Moreover, to account for the disparities among various tasks, we propose a dual-neck architecture. Finally, we present an optimized design for the decoders of each task. Our model evinces strong performance on the demanding BDD100K dataset, attaining remarkable accuracy (Acc) in vehicle detection and superior precision in drivable area segmentation (mIoU). In addition, this is the first work that can process these three visual perception tasks simultaneously in real time on an embedded device Atlas 200I A2 and maintain excellent accuracy.

Research on Lane-Changing Decision Making and Planning of Autonomous Vehicles Based on GCN and Multi-Segment Polynomial Curve Optimization.

This paper considers the interactive effects between the ego vehicle and other vehicles in a dynamic driving environment and proposes an autonomous vehicle lane-changing behavior decision-making and trajectory planning method based on graph convolutional networks (GCNs) and multi-segment polynomial curve optimization. Firstly, hierarchical modeling is applied to the dynamic driving environment, aggregating the dynamic interaction information of driving scenes in the form of graph-structured data. Graph convolutional neural networks are employed to process interaction information and generate ego vehicle's driving behavior decision commands. Subsequently, collision-free drivable areas are constructed based on the dynamic driving scene information. An optimization-based multi-segment polynomial curve trajectory planning method is employed to solve the optimization model, obtaining collision-free motion trajectories satisfying dynamic constraints and efficiently completing the lane-changing behavior of the vehicle. Finally, simulation and on-road vehicle experiments are conducted for the proposed method. The experimental results demonstrate that the proposed method outperforms traditional decision-making and planning methods, exhibiting good robustness, real-time performance, and strong scenario generalization capabilities.

Semantic Segmentation of 3D Point Clouds in Outdoor Environments Based on Local Dual-Enhancement

Semantic segmentation of 3D point clouds in drivable areas is very important for unmanned vehicles. Due to the imbalance between the size of various outdoor scene objects and the sample size, the object boundaries are not clear, and small sample features cannot be extracted. As a result, the semantic segmentation accuracy of 3D point clouds in outdoor environment is not high. To solve these problems, we propose a local dual-enhancement network (LDE-Net) for semantic segmentation of 3D point clouds in outdoor environments for unmanned vehicles. The network is composed of local-global feature extraction modules, and a local feature aggregation classifier. The local-global feature extraction module captures both local and global features, which can improve the accuracy and robustness of semantic segmentation. The local feature aggregation classifier considers the feature information of neighboring points to ensure clarity of object boundaries and the high overall accuracy of semantic segmentation. Experimental results show that provides clearer boundaries between various objects, and has higher identification accuracy for small sample objects. The LDE-Net has good performance for semantic segmentation of 3D point clouds in outdoor environments.