Drivable Region Research Articles

Drivable Region Completion via a 3D LiDAR

Drivable Region Completion via a 3D LiDAR

UpBEV: Fast and Accurate LiDAR-Based Drivable Region Detection Utilizing Uniform Polar BEV

UpBEV: Fast and Accurate LiDAR-Based Drivable Region Detection Utilizing Uniform Polar BEV

Snow coverage estimation using camera data for automated driving applications

In the U.S., over 38,000 fatalities occur every year due to automotive accidents where 24% of these accidents are attributable to inclement weather. Automated driving systems have shown to decrease up to 21% of potential collisions, however, these systems do not operate in inclement weather. The camera’s reliance on clear lane line detections cease the functionality of the safety systems when occlusions occur due to precipitation. For these systems to become operational during conditions such as snow coverage, therefore leading to a greater impact on safety, new research and development is needed to focus on inclement weather scenarios. This study addresses this need by first collecting a new dataset consisting of raw camera images along arterial roads in Kalamazoo, MI and additionally collecting snow precipitation data from the National Center for Environmental Information. With this data, snow coverage estimation models were developed to automatically determine categories of snow coverage. The models were developed by investigating various machine learning algorithm types, image predictors, and the presence of snow precipitation data. The final model resulted in 95.63% accuracy for categorizing the instance as either none, standard, or heavy snow coverage. These categories are important for future development of purpose-build algorithms that identify drivable regions in various levels of snow coverage for future automated driving systems. The results demonstrate that snow estimation is a near-term achievable task and that the presence weather data improves accuracy. With the addition of snow-coverage estimation, automated driving systems can be developed to react to these different conditions respectively and further reduce the nearly 6,000 annual fatalities caused driving in adverse weather.

Drivable Dirt Road Region Identification Using Image and Point Cloud Semantic Segmentation Fusion

Driving scene understanding is an essential technique for realizing autonomous driving. Although many large-scale datasets for autonomous driving have enabled studies in terms of driving scene understanding, unpaved dirt roads have not been considered as their target driving environment. Drivable region identification is important in dirt roads to prevent damage to the vehicle and passengers. Semantic segmentation of camera images and lidar point clouds have been used to recognize the environment around an autonomous vehicle. The road and objects are recognized by classifying image pixels and point clouds into semantic classes. In this study, we introduce a perception method for drivable region identification on dirt roads through fusion of image and point cloud semantic segmentation. Our approach includes an image semantic segmentation algorithm and a point cloud semantic segmentation algorithm, which we combine into a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">bird’s-eye-view grid map</i> format. To transform the point-wise drivable region identification results into the area-wise information, we adopt the alphashape algorithm. Speed improvements are made by using small proportion of drivable points, and the accuracy degradation is compensated by accumulating the perception results along the time.

역진자 주행로봇의 피치 모션에 강인한 경계추종 자율주행 기법

In a vision-based autonomous driving of a mobile robot, heavy pitch motions can significantly deteriorate the boundary, or lane-following, performance. Such motions can result from the high maneuvers of an inverted pendulum balancing robot. If the vision camera frequently tilts forward or backward according to the mobile platform’s pitch motion, it would fail to extract drivable regions from the image data. It would also fail to detect successive lanes or boundaries to determine local paths, which is mainly due to the vision camera’s extrinsic parameter variation. In this paper, we investigate a local path planning method that accounts for the heavy pitch motion of a mobile robot by considering the camera posture in real-time image processing. Fundamentally, this method is based on the vanishing point tracking and the pinhole camera model. The highlight of this paper lies in employing the vertical filter to exclude vertical outliers from the boundary data and also the image horizon filter to determine the valid vanishing point candidates. Indoor and outdoor experiments demonstrate the effectiveness of the proposed method.

TA-Unet: Integrating Triplet Attention Module for Drivable Road Region Segmentation.

Road segmentation has been one of the leading research areas in the realm of autonomous driving cars due to the possible benefits autonomous vehicles can offer. Significant reduction of crashes, greater independence for the people with disabilities, and reduced traffic congestion on the roads are some of the vivid examples of them. Considering the importance of self-driving cars, it is vital to develop models that can accurately segment drivable regions of roads. The recent advances in the area of deep learning have presented effective methods and techniques to tackle road segmentation tasks effectively. However, the results of most of them are not satisfactory for implementing them into practice. To tackle this issue, in this paper, we propose a novel model, dubbed as TA-Unet, that is able to produce quality drivable road region segmentation maps. The proposed model incorporates a triplet attention module into the encoding stage of the U-Net network to compute attention weights through the triplet branch structure. Additionally, to overcome the class-imbalance problem, we experiment on different loss functions, and confirm that using a mixed loss function leads to a boost in performance. To validate the performance and efficiency of the proposed method, we adopt the publicly available UAS dataset, and compare its results to the framework of the dataset and also to four state-of-the-art segmentation models. Extensive experiments demonstrate that the proposed TA-Unet outperforms baseline methods both in terms of pixel accuracy and mIoU, with 98.74% and 97.41%, respectively. Finally, the proposed method yields clearer segmentation maps on different sample sets compared to other baseline methods.

Tire Track Identification: A Method for Drivable Region Detection in Conditions of Snow-Occluded Lane Lines

&lt;div class="section abstract"&gt;&lt;div class="htmlview paragraph"&gt;Today’s Advanced Driver Assistance Systems (ADAS) predominantly utilize cameras to increase driver and passenger safety. Computer vision, as the enabler of this technology, extracts two key environmental features: the drivable region and surrounding objects (e.g., vehicles, pedestrians, bicycles). Lane lines are the most common characteristic extracted for drivable region detection, which is the core perception task enabling ADAS features such as lane departure warnings, lane-keeping assistance, and lane-centering. However, when subject to adverse weather conditions (e.g., occluded lane lines) the lane line detection algorithms are no longer operational. This prevents the ADAS feature from providing the benefit of increased safety to the driver. The performance of one of the leading computer vision system providers was tested in conditions of variable snow coverage and lane line occlusion during the 2020-2021 winter in Kalamazoo, Michigan. The results show that this computer vision system was only able to provide high confidence detections in less than 1% of all frames recorded. This is an alarming result, as 21% of all crashes in the U.S. are weather-related. To increase the capabilities of ADAS when snow-occlusions are present, a tire track identification system was developed by comparing various supervised machine learning models. A custom dataset was collected using the Energy Efficient and Autonomous Vehicles lab’s research platform from Western Michigan University. A data preparation pipeline was implemented to label tire tracks and train the machine learning models. The best model achieved high confidence detections of tire tracks in 83% of all frames of which tire tracks were present, an 82% increase in detections than the leading computer vision system provider.&lt;/div&gt;&lt;/div&gt;

One-Shot Summary Prototypical Network Toward Accurate Unpaved Road Semantic Segmentation

Recent studies of driving scene understanding based on image semantic segmentation have achieved dramatic advances in speed and accuracy. Large-scale public datasets for semantic segmentation of paved road driving scenes have led the advances, but there is no large-scale public dataset for unpaved road environments. Building a large-scale image semantic segmentation dataset for unpaved roads is very expensive, and domain gaps between geographically distributed locations and those of seasonal changes hinder building a training dataset that is adequate to train a convolutional neural network model. In this paper, to resolve the data insufficiency problem, we use an one-shot learning setting in unpaved road driving scene understanding. Our One-shot Summary Prototypical Network (OSPNet) is trained with paved road driving scenes, and it identifies drivable regions in unpaved roads given only a single support image and unpaved road mask data. The OSPNet improves previous two branch few-shot segmentation approaches by introducing the summary branch which enables channel-wise weighting for important features in the feature map of support and query branches. Our experiments show that our model quantitatively and qualitatively outperforms recent supervised and few-shot segmentation models.

Detection of highway lane lines and drivable regions based on dynamic image enhancement algorithm under unfavorable vision

The detection of lane lines and drivable regions is the basis for the development of advanced driving assistance systems. Aiming at the problem of the poor robustness of highway detection under unfavorable visual conditions (UVCs), a new road detection method based on the dynamic image enhancement algorithm is proposed. The classification of images under different UVCs is obtained using gray feature and definition feature, and the classification result is employed to select an appropriate enhancement algorithm. The definition parameters of the images, which are used to dynamically adjust the parameters of the image enhancement algorithm, are acquired based on the definition evaluation model. On this basis, the improved probabilistic Hough transform algorithm and the adaptive region growth algorithm based on Gaussian model are applied to detect lane lines and drivable regions, respectively. The experimental results demonstrate the robust adaptation and real-time effectiveness of the approach under UVCs.

Autonomous Crop Row Guidance Using Adaptive Multi-ROI in Strawberry Fields

Automated robotic platforms are an important part of precision agriculture solutions for sustainable food production. Agri-robots require robust and accurate guidance systems in order to navigate between crops and to and from their base station. Onboard sensors such as machine vision cameras offer a flexible guidance alternative to more expensive solutions for structured environments such as scanning lidar or RTK-GNSS. The main challenges for visual crop row guidance are the dramatic differences in appearance of crops between farms and throughout the season and the variations in crop spacing and contours of the crop rows. Here we present a visual guidance pipeline for an agri-robot operating in strawberry fields in Norway that is based on semantic segmentation with a convolution neural network (CNN) to segment input RGB images into crop and not-crop (i.e., drivable terrain) regions. To handle the uneven contours of crop rows in Norway’s hilly agricultural regions, we develop a new adaptive multi-ROI method for fitting trajectories to the drivable regions. We test our approach in open-loop trials with a real agri-robot operating in the field and show that our approach compares favourably to other traditional guidance approaches.

Real-time Drivable Region Planning Based on 3D LiDAR

Stable, real-time, drivable area planning in a dynamic environment is an essential feature of autonomous vehicles. This paper presents an efficient 3D laser radar-based drivable area planning algorithm. In order to extract the drivable area, the original point cloud data is first downsampled to obtain a relatively sparse point cloud to reduce the complexity. Then, based on the geometric features of the pavement point and dividing the road plane, the obstacle point after the road plane is divided. The cloud is transformed into a 2D aerial view, and a series of expansion, convolution and other regional operations are performed on the bird's-eye view, and the road edge points are extracted, and the curve fitting is performed based on the least squares method to plan the drivable region. The algorithm proposed in this paper was tested on the KITTI dataset and obtained robust, high-efficiency experimental results.

Lane Line Detection by LiDAR Intensity Value Interpolation

&lt;div class="section abstract"&gt;&lt;div class="htmlview paragraph"&gt;Lane marks are an important aspect for autonomous driving. Autonomous vehicles rely on lane mark information to determine a safe and legal path to drive. In this paper an approach to estimate lane lines on straight or slightly curved roads using a LiDAR unit for autonomous vehicles is presented. By comparing the difference in elevation of LiDAR channels, a drivable region is defined. The presented approach used in this paper differs from previous LiDAR lane line detection methods by reducing the drivable region from three to two dimensions exploring only the x-y trace. In addition, potential lane markings are extracted by filtering a range of intensity values as opposed to the traditional approach of comparing neighboring intensity values. Further, by calculating the standard deviation of the potential lane markings in the y-axis, the data can be further refined to specific points of interest. By applying a statistical approximation, to these points of interest, the results given show a linear approximation of the lane lines.&lt;/div&gt;&lt;/div&gt;

ChipNet: Real-Time LiDAR Processing for Drivable Region Segmentation on an FPGA

This paper presents a field-programmable gate array (FPGA) design of a segmentation algorithm based on convolutional neural network (CNN) that can process light detection and ranging (LiDAR) data in real-time. For autonomous vehicles, drivable region segmentation is an essential step that sets up the static constraints for planning tasks. Traditional drivable region segmentation algorithms are mostly developed on camera data, so their performance is susceptible to the light conditions and the qualities of road markings. LiDAR sensors can obtain the 3D geometry information of the vehicle surroundings with high precision. However, it is a computational challenge to process a large amount of LiDAR data in real-time. In this paper, a convolutional neural network model is proposed and trained to perform semantic segmentation using data from the LiDAR sensor. An efficient hardware architecture is proposed and implemented on an FPGA that can process each LiDAR scan in 17.59 ms, which is much faster than the previous works. Evaluated using Ford and KITTI road detection benchmarks, the proposed solution achieves both high accuracy in performance and real-time processing in speed.

RoadNet: An 80-mW Hardware Accelerator for Road Detection

As a fundamental feature of intelligent vehicles, vision-based road detection must be executed on a real-time embedded platform with high accuracy. Road detection is often applied in conjunction with lane detection to determine the drivable regions. Although some existing research based on large deep learning models achieved high accuracy using the road detection dataset, they often did not consider the low power requirement of a typical embedded system. In this letter, an ultralow-power hardware accelerator for road detection is proposed. By adopting a top-down convolutional neural network (CNN) structure, a small CNN, namely RoadNet, is trained that can achieve near state-of-the-art detection accuracy. Furthermore, each CNN layer is trimmed to be computationally identical and every processing element in the architecture is fully utilized. When implemented using 32-nm process technology, the proposed hardware accelerator requires the chip area of 0.45 mm2 and the power consumption of only 80 mW, which results in an equivalent power efficiency of about 300 GOP/s/W. The RoadNet chip is capable of processing 241 frames/s at 1080P image resolution. It stands out as an ultralow-power hardware accelerator in an embedded system for road detection.

DriveSpace: Towards context-aware drivable area detection

Free space is an essential component of any autonomous driving system. It describes the region, which is typically the road surface, around the vehicle which is free from obstacles. However, in practice, free space should not solely describe the area where a vehicle can plan a trajectory. For instance, in a single lane road with two way traffic the opposite lane should not be included as an area where the vehicle can plan a driving path although it will be detected as free space. In this paper, we introduce a new conceptual representation called DriveSpace which corresponds to semantic understanding and context of the scene. We formulate it based on combination of dense 3d reconstruction and semantic segmentation. We use a graphical model approach to fuse and learn the drivable area. As the drivable region is highly dependent on the situation and dynamics of other objects, it remains a bit subjective. We analyze various scenarios of DriveSpace and propose a general method to detect all scenarios. As it is a new concept, there are no datasets available for development and test, however, we are working on creating the same to show quantitative results of the proposed method.

Road Segmentation and Tracking on Indian Road Scenes

Vision-based road detection is a challenging task due to surrounding scenes and types of roads. This paper describes an efficient and effective algorithm for general road segmentation and tracking. The major contributions of this paper are three aspects: 1) an optimised voter selection strategy-based modified voting process for vanishing point detection; 2) use of Kalman filter to avoid false detection of vanishing point and reduction in computational complexity of final road segmentation; 3) the algorithm is evaluated against different road types with varying surrounding scenes. The method has been implemented and tested with 10,000 video frames of Indian road scenes. Experimental results demonstrate that the algorithm achieves better efficiency compared to some of the texture-based vanishing point detection algorithms and successfully segments drivable road regions from varying Indian road scenes.

Intersection detection based on recognition of drivable region using reflection light intensity

Intersection detection based on recognition of drivable region using reflection light intensity

Integration Framework of Monocular Vision-Based Drivable Region Detection and Contour-Based Vehicle Localization for Autonomous Driving Systems

Integration Framework of Monocular Vision-Based Drivable Region Detection and Contour-Based Vehicle Localization for Autonomous Driving Systems

Real-Time Road Lane Detection in Urban Areas Using LiDAR Data

The generation of digital maps with lane-level resolution is rapidly becoming a necessity, as semi- or fully-autonomous driving vehicles are now commercially available. In this paper, we present a practical real-time working prototype for road lane detection using LiDAR data, which can be further extended to automatic lane-level map generation. Conventional lane detection methods are limited to simple road conditions and are not suitable for complex urban roads with various road signs on the ground. Given a 3D point cloud scanned by a 3D LiDAR sensor, we categorized the points of the drivable region and distinguished the points of the road signs on the ground. Then, we developed an expectation-maximization method to detect parallel lines and update the 3D line parameters in real time, as the probe vehicle equipped with the LiDAR sensor moved forward. The detected and recorded line parameters were integrated to build a lane-level digital map with the help of a GPS/INS sensor. The proposed system was tested to generate accurate lane-level maps of two complex urban routes. The experimental results showed that the proposed system was fast and practical in terms of effectively detecting road lines and generating lane-level maps.

Robust and efficient vanishing point detection in unstructured road scenes for assistive navigation

PurposeThis paper aims to propose a robust and efficient method for vanishing point detection in unstructured road scenes.Design/methodology/approachThe proposed method includes two main stages: drivable region estimation and vanishing point detection. In drivable region estimation stage, the road image is segmented into a set of patches; then the drivable region is estimated by the patch-wise manifold ranking. In vanishing point detection stage, the LSD method is used to extract the straight lines; then a series of principles are proposed to remove the noise lines. Finally, the vanishing point is detected by a novel voting strategy.FindingsThe proposed method is validated on various unstructured road images collected from the real world. It is more robust and more efficient than the state-of-the-art method and the other three recent methods. Experimental results demonstrate that the detected vanishing point is practical for vision-sensor-based navigation in complex unstructured road scenes.Originality/valueThis paper proposes a patch-wise manifold ranking method to estimate the drivable region that contains most of the informative clues for vanishing point detection. Based on the removal of the noise lines through a series of principles, a novel voting strategy is proposed to detect the vanishing point.