Lane Line Research Articles

Study on Relative Accuracy and Verification Method of High-Definition Maps for Autonomous Driving

High-definition maps (HDM) for autonomous driving (AD) are an important component of AD systems. HDMs accurately provide a priori information, including lane lines, and road signs, for AD systems. It is an important task to make a reasonable accuracy assessment of the HDM. The current methods for relative accuracy evaluation of general maps in the field of mapping are not fully applicable to HDMs. In this study, a method based on point set alignment and resampling is used to evaluate the relative accuracy of lane lines, and experiments are conducted based on relevant real HDM data. The results show that the relative accuracy of the lane lines is more detailed and relevant than the traditional method. This has implications for the quality control of HDM production.

Real-Time Lane Line Tracking Algorithm to Mini Vehicles

Abstract Autonomous navigation is important not only in autonomous cars but also in other transportation systems. In many applications, an autonomous vehicle has to follow the curvature of a real or artificial road or in other words lane lines. In those application, the key is the lane detection. In this paper, we present a real-time lane line tracking algorithm mainly designed to mini vehicles with relatively low computation capacity and single camera sensor. The proposed algorithm exploits computer vision techniques in combination with digital filtering. To demonstrate the performance of the method, experiments are conducted in an indoor, self-made test track where the effect of several external influencing factors can be observed. Experimental results show that the proposed algorithm works well independently of shadows, bends, reflection and lighting changes.

Image Adaptive Contrast Enhancement for Low-illumination Lane Lines Based on Improved Retinex and Guided Filter

ABSTRACT In a low-illumination environment, the contrast between lane lines and the ground is relatively low. Traditional image enhancement algorithms, such as gamma correction, Histogram Equalization, and multiple-scale Retinex, may result in over enhancement and detail loss, which decreases the detection accuracy of driver assistance systems. In this work, we introduce a low-illumination image enhancement algorithm based on improved Retinex theory and apply it to lane-line detection. A luminance channel optimization method based on a bimodal energy function is adopted to select the weight of the linear combination. A guided filter with edge-preservation is applied to obtain the illumination component of the scene. Furthermore, the reflection component is estimated in a hyperbolic tangent space, and the luminance and contrast are adaptively adjusted to obtain the enhanced image. Experimental results show that proposed method can effectively extract the lane-line edges and suppress the noise in the dark area with low lane-line illumination. Moreover, it can improve the detection success rate of an assisted driving system.

Steering angle prediction YOLOv5-based end-to-end adaptive neural network control for autonomous vehicles

The combination of steering angle prediction and control of autonomous vehicles (AVs) is a challenging task. To improve the real-time steering angle prediction accuracy and the effectiveness of steering control, a novel steering angle prediction YOLOv5-based end-to-end adaptive neural network control for AVs is proposed. Firstly, since most of the lane line datasets are simulated images and lack of diversity, a novel lane dataset derived from the real roads are made manually to train the You Only Look Once version 5 (YOLOv5) network model. To improve the detection accuracy of the network model, the Generalized Intersection over Union (GIoU) of the bounding box regression loss function is updated to a Complete Intersection over Union (CIoU) with a better convergence effect. Furthermore, the neural network-based controller and disturbance observer are proposed to effectively control the steering angle predicted by YOLOv5 and estimate the lumped uncertainty. Meanwhile, a composite adaptive updating law is constructed by utilizing the tracking error and modeling error to improve steering performance. Finally, the system stability is proved by Lyapunov theory and the effectiveness of the proposed method is verified with experiments.

Lane Lines Detection under Complex Environment by Fusion of Detection and Prediction Models

The lane lines’ length, width, and direction are very regular, serialized, and structurally associated, which are not easily affected by the environment. To enhance lane detection in a complicated environment, an approach combines visual information with the spatial distribution. Firstly, the grid density of the target detection algorithm YOLOv3 (you only look once V3) is improved from S×S to S×2S, aiming at the particular points in the bird’s-eye view where the lane lines had different densities in the horizontal and vertical directions. The obtained YOLOv3 (S×2S) is more suitable for detecting objects with small and large aspect ratios. It also identifies image features along with balances the detection speed and accuracy. Secondly, based on a bi-directional gated recurrent unit (BGRU), a new lane line prediction model BGRU-Lane (BGRU-L) based on the distribution of lane lines is proposed using the characteristic of lane line serialization and structural correlation. Finally, Dempster-Shafer (D-S) algorithm based on confidence was used to integrate the results of YOLOv3 (S×2S) and BGRU-L to improve the lane line detection ability under complex environments. The experiment was carried out on the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) dataset, while Euro Truck Simulator 2 (ETS2) is used as a supplement dataset. After fusing YOLOv3 (S×2S) and BGRU-L models in the D-S model, the detection results have high accuracy in a complex environment by 90.28 mAP. The detection speed is 40.20fps, which enables real-time detection.

Real-time lane detection model based on non bottleneck skip residual connections and attention pyramids.

The security of car driving is of interest due to the growing number of motor vehicles and frequent occurrence of road traffic accidents, and the combination of advanced driving assistance system (ADAS) and vehicle-road cooperation can prevent more than 90% of traffic accidents. Lane detection, as a vital part of ADAS, has poor real-time performance and accuracy in multiple scenarios, such as road damage, light changes, and traffic jams. Moreover, the sparse pixels of lane lines on the road pose a tremendous challenge to the task of lane line detection. In this study, we propose a model that fuses non bottleneck skip residual connections and an improved attention pyramid (IAP) to effectively obtain contextual information about real-time scenes and improve the robustness and real-time performance of current lane detection models. The proposed model modifies the efficient residual factorized pyramid scene parsing network (ERF-PSPNet) and utilizes skip residual connections in non bottleneck-1D modules. A decoder with an IAP provides high-level feature maps with pixel-level attention. We add an auxiliary segmenter and a lane predictor side-by-side after the encoder, the former for lane prediction and the latter to assist with semantic segmentation for classification purposes, as well as to solve the gradient disappearance problem. On the CULane dataset, the F1 metric reaches 92.20% in the normal scenario, and the F1 metric of the model is higher than the F1 metrics of other existing models, such as ERFNet-HESA, ENet_LGAD, and DSB+LDCDI, in normal, crowded, night, dazzling light and no line scenarios; in addition, the mean F1 of the nine scenarios reached 74.10%, the runtime (time taken to test 100 images) of the model was 5.88 ms, and the number of parameters was 2.31M, which means that the model achieves a good trade-off between real-time performance and accuracy compared to the current best results (i.e., a running time of 13.4 ms and 0.98M parameters).

Fusing GNSS/INS/Vision With A Priori Feature Map for High-Precision and Continuous Navigation

Nowadays, the autonomous vehicle has undergone an impressive development, in which one of the most challenging tasks is to pursue high-precision and continuous navigation in complex scenarios. However, GNSS signals are very likely to be blocked and the inertial system suffers from error drift over time. To achieve centimeter-level positioning accuracy, a tightly-coupled model is built via fusing visual, GNSS, and INS measurements, based on a priori visual feature map. The visual map is first generated by a visual mobile surveying system, and the quality of the map is improved by outlier elimination and road lane tracking. This visual feature map consists of two layers that involve 3D coordinates of robust visual features and road lane markings, then two GNSS/INS/Vision fusion models are developed. The first one fuses visual 2D-3D feature matching, GNSS, and INS measurement. The proposed system is effective even when satellites are all out of lock. The second one fuses lane line constraints with GNSS, INS, and distance measurement indicator (DMI). In the experiment part, we show how a visual feature map is generated and refined, then, the performance of the positioning algorithm is evaluated. For the first fusion model, we achieve 10 cm RMS accuracy when only one feature point was matched and all GNSS signals are blocked for 20 minutes. For the other fusion model, the system maintained 10 cm RMS accuracy or even better when a few GNSS satellites are out of lock for 300 seconds.

DALaneNet: A Dual Attention Instance Segmentation Network for Real-Time Lane Detection

Lane detection is a challenging task due to problems like the diversity of lanes, occlusion, dazzle light, and so on. We believe that two factors are helpful to solve the above problems and therefore improve detection performance, including global context dependency and effective feature representation focusing on important feature channels. In this work, we propose an instance segmentation approach and develop a novel dual attention network DALane- Net for real-time lane detection. The network leverages the spatial attention and channel attention mechanism to achieve better feature representational power. The spatial attention learns position relation between pixels while channel attention learns the importance of feature channel characteristics. The usage of the dual attention mechanism strengthens the lane line feature representation and effectively solves occlusion, lighting issues. Furthermore, to address the issues that the segmentation-based network usually suffers a low running speed due to complex convolution calculation, we propose more efficient convolution structures to improve the computational efficiency. Substantial experiments were conducted on the TuSimple lane marking database. The results demonstrate that DALaneNet is robust in various illumination conditions and especially effective in the case of occlusion. Compared with the existing state-of-the-art methods, the DALaneNet exhibits integrated advantages in terms of detection accuracy and speed.

Lane Image Detection Based on Convolution Neural Network Multi-Task Learning

Based on deep neural network multi-task learning technology, lane image detection is studied to improve the application level of driverless technology, improve assisted driving technology and reduce traffic accidents. The lane line database published by Caltech and Tucson company is used to extract the ROI (Region of Interest), scale, and inverse perspective transformation as well as to preprocess the image, so as to enrich the data set and improve the efficiency of the algorithm. In this study, ZFNet is used to replace the basic networks of VPGNet, and their structures are changed to improve the detection efficiency. Multi-label classification, grid box regression and object mask are used as three task modules to build a multi-task learning network named ZF-VPGNet. Considering that neural networks will be combined with embedded systems in the future, the network will be compressed to CZF-VPGNet without excessively affecting the accuracy. Experimental results show that the vision system of driverless technology in this study achieved good test results. In the case of fuzzy lane line and missing lane line mark, the improved algorithm can still detect and obtain the correct results, and achieves high accuracy and robustness. CZF-VPGNet can achieve high real-time performance (26FPS), and a single forward pass takes about 36 ms or less.

Semantic Segmentation of Parking Lot Scene Based on Surround View System

Semantic segmentation of parking lot scenes is the prerequisite of environment perception for automatic parking technology. It provides environmental semantic information for automatic parking of vehicles. However, due to the dim light of the parking lot environment, unclear ground signs, road reflections and other factors, semantic segmentation such as FCN The method is not yet able to segment ground signs such as background and lane lines to meet the perception needs of automatic parking. This paper proposes a parking lot scene semantic segmentation method based on a surround view system. The surround view system consists of 4 fisheye cameras. The images acquired by each camera are subjected to distortion correction, inverse perspective transformation and image stitching fusion to obtain a ring view. Based on the ring view, a semantic segmentation algorithm based on attention and feature fusion is proposed. Experiments are carried out on a self-made parking lot dataset with a size of 1280x960 pixels. The results show that the method proposed in this paper improves the mIoU of the FCN model by 12.3%.

Lane-DeepLab: Lane semantic segmentation in automatic driving scenarios for high-definition maps

Accurate high-definition maps with lane markings are often used as the navigation back-end for commercial autonomous vehicles. Currently, most high-definition maps are manually constructed by human labelling. Therefore, it is urgently required to propose a multi-class lane detection method that can automatically mark the road lanes to assist in generating high-precision maps for autonomous driving. We propose a lane segmentation detection method, named Lane-DeepLab, which is based on semantic segmentation for detecting multi-class lane lines in unmanned driving scenarios. The proposed method is based on the DeepLabv3+ network as the baseline, and we have redesigned the encoder-decoder structure to generate more accurate lane line detection results. More specifically, we restructure the atrous convolution at multi-scale by applying attention mechanism. Subsequently, we employ the Semantic Embedding Branch (SEB) to combine the high-level and low-level semantic information to obtain more abundant features, and use the Single Stage Headless (SSH) context module to obtain multi-scale information. Finally, we fuse the results to generate automatic high-precision mapping results. Our method has improved performance compared with other methods in the ApolloScape part of the dataset. Besides, in the database of Cityscapes, our approach has also achieved good results in semantic segmentation. Experimental results demonstrate that our proposed Lane-DeepLab can provide excellent performance in real traffic scenarios.

Multi-scale spatial convolution algorithm for lane line detection and lane offset estimation in complex road conditions

Deep learning has made remarkable progress in the field of image classification and object detection. Nevertheless, in the autonomous driving research, the real-time lane line detection and lane offset estimation in complex traffic scenes have always been challenging and difficult tasks. Traditional detection methods need manual adjustment of parameters, they face many problems and difficulties and are still highly susceptible to interference caused by obstructing objects, illumination changes, and pavement wear. It is still challenging to design a robust lane detection and lane offset estimation algorithm. In this paper, we propose a convolutional neural network for lane offset estimation and lane line detection in a complex road environment, which transforms the problems of lane line detection into the instance’s segmentation. In response to a change in the method of lane processing, the network will form its example to each line. The global scale perception optimization mechanism is designed to solve the issue, especially where the lane line width is gradually narrowing at the vanishing point of the lane. At the same time, to realize multi-tasking processing and improve performance, and end-to-end lane offset estimation network is used in addition to the lane line detection network.

Lane change strategy analysis and recognition for intelligent driving systems based on random forest

The development of intelligent connected technology provides a platform for multidimensional information interaction and makes the recognition of a specific lane change strategy become a reality. Intelligent driving systems cannot make human-like decisions by only recognizing left or right lane change behaviors; in fact, poor environmental cognitive competence is the main cause of accidents of intelligent driving systems during road tests. Therefore, the recognition of specific lane change strategies can provide support for safety performance improvement. This research aims to identify the different lane change strategies of a subject vehicle with a rear approaching vehicle in the target lane. For the purpose of acquiring different strategies, a total of 42 experienced drivers including 37 males and 5 females participated in the on-road vehicle experiments, and we divided lane change strategies into three categories: mandatory, yielding, and waiting for lane change. The lane change duration time, lane crossing time, and characteristic parameters including distance to lane line, steering wheel angle, relative distance, and relative speed under different lane change strategies were compared and analyzed. Then the random forest classifier was employed to construct a lane change strategy identification model. Results indicated that the maximum value of global recognition accuracy under three input parameters reached 88.78% when the time window was 1.2 s, which was higher than the maximum value of recognition accuracy under four input parameters, but the identification time was 0.1 s later than the model with four input parameters. In addition, the recognition performance of the random forest model, attention-bidirectional long short-term memory (Atten-BiLSTM) model and genetic algorithm-support vector machine (GA-SVM) model was compared. The results demonstrated that the global recognition accuracy and identification time of the proposed model were better than that of GA-SVM model under different input parameters, and the performance was comparable to the Atten-BiLSTM model. The findings provide the basis for the intelligent driving system to make more human-like decisions and improve safety performance.

Detection of lane line based on Robert operator

As autonomous driving technology becomes more and more popular, its safety is also attracting attention. Regarding the automatic driving of vehicles, the detection of road markings is particularly important. This paper improves the lane edge detection part of the Hough transform lane line detection method. Because the traditional Canny operator edge detection method is good for image processing, but the detection time is long, this paper replaces the Canny algorithm with the Robert operator edge detection method. The sub-edge detection method can improve the detection speed of lane line extraction. In MATLAB, by using multiple edge detection operators to perform edge detection on the same image 100 times, and taking the average of the detection time, it is found that the Robert operator takes a shorter time in the detection process than the Canny operator; Then the Robert operator and Canny operator are respectively fused into the Hough transform lane line detection. After 100 times, the same image is detected, and the running time is statistically averaged for comparison and analysis. The Robert operator is better than the Canny operator. The time taken is reduced by 0.15191 s. The simulation results show that the integration of Robert operator in Hough transform lane line detection improves the real-time performance of lane detection.

The geometric attention-aware network for lane detection in complex road scenes.

Lane detection in complex road scenes is still a challenging task due to poor lighting conditions, interference of irrelevant road markings or signs, etc. To solve the problem of lane detection in the various complex road scenes, we proposed a geometric attention-aware network (GAAN) for lane detection. The proposed GAAN adopted a multi-task branch architecture, and used the attention information propagation (AIP) module to perform communication between branches, then the geometric attention-aware (GAA) module was used to complete feature fusion. In order to verify the lane detection effect of the proposed model in this paper, the experiments were conducted on the CULane dataset, TuSimple dataset, and BDD100K dataset. The experimental results show that our method performs well compared with the current excellent lane line detection networks.

Research on visibility detection model optimization based on dark channel prior and image entropy and visibility development trend prediction

In this paper, a visibility detection model based on the dark channel prior and image entropy is established to improve the lane line detection algorithm. Our algorithm does not need to preset the target, nor is it affected by the camera calibration parameters and position. It transforms the visibility calculation problem into the atmosphere transmittance calculation problem and refines the required results through the guided filter, achieving more accurate and stable visibility estimation results. In addition, based on the changing regularity of the visibility over time obtained by the detection model, a mathematical model is established to predict the change of heavy fog. We use ADF to test the visibility obtained in the visibility detection model and calculate the autocorrelation and partial autocorrelation functions. Finding the original sequence non-stationary, we perform the difference on the data, remove all insignificant factors and then incorporate the data into ARIMA model for fitting, finally getting the fitting and prediction results. The results are found similar to the actual situation, indicating that the results obtained by the visibility prediction model are robust and reliable.

End-to-End Autonomous Driving Through Dueling Double Deep Q-Network

Recent years have seen the rapid development of autonomous driving systems, which are typically designed in a hierarchical architecture or an end-to-end architecture. The hierarchical architecture is always complicated and hard to design, while the end-to-end architecture is more promising due to its simple structure. This paper puts forward an end-to-end autonomous driving method through a deep reinforcement learning algorithm Dueling Double Deep Q-Network, making it possible for the vehicle to learn end-to-end driving by itself. This paper firstly proposes an architecture for the end-to-end lane-keeping task. Unlike the traditional image-only state space, the presented state space is composed of both camera images and vehicle motion information. Then corresponding dueling neural network structure is introduced, which reduces the variance and improves sampling efficiency. Thirdly, the proposed method is applied to The Open Racing Car Simulator (TORCS) to demonstrate its great performance, where it surpasses human drivers. Finally, the saliency map of the neural network is visualized, which indicates the trained network drives by observing the lane lines. A video for the presented work is available online, https://youtu.be/76ciJmIHMD8 or https://v.youku.com/v_show/id_XNDM4ODc0MTM4NA==.html.

Vision-Based Lane Line Detection System

Since the 1990s, the rising key issue of the automobile industry is self-driving or driverless vehicles. Apparently, one of the most important challenges for smart self-driving cars comprises lane-detecting and lane-tracking capability to ensure safety and also decrease vehicle accidents for driver assistance systems. Since road lane detection is one of the most challenging tasks, driverless vehicles must learn to observe the road from a visual perspective in order to achieve automatic driving. Most of the research Works done so far can only detect the lanes or vehicles separately. However, in this paper, we propose a method to combine lane information and vehicle/obstacle information that can support the driver assistance system, driver warning system or the lane change assistant system so that it enhances the quality of results. For the lane changing system, the system detects or tracks the lane lines and detects the vehicles on all sides of a test vehicle. In lane detection, line detection algorithms such as the Canny Edge detection algorithm are used to detect the lane edges. Kalman filter will be used to track the vehicle detected from the vehicle detection algorithm. For vehicle detection, we use Otsu’s thresholding, horizontal edge filtering and vertical edge. The vertical edge filter and the Otsu’s thresholding are used to detect the vehicles on all sides of the test vehicles, then the horizontal edge is used to verify the vehicles detected.

Lane line detection and recognition based on dynamic ROI and modified firefly algorithm

Lane line detection and recognition based on dynamic ROI and modified firefly algorithm

Channel Attention in LiDAR-camera Fusion for Lane Line Segmentation

To assess the contributions of the different feature channels of sensors, we introduce a novel multimodal fusion method and demonstrate its practical utility using LiDAR-camera fusion networks. Specifically, a channel attention module that can be easily added to a fusion segmentation network is proposed. In this module, we use the channel attention mechanism to obtain the cross-channel local interaction information, and the weights of feature channels are assigned to represent the contributions of different feature channels. To verify the effectiveness of the proposed method, we conduct experiments on two types of feature fusion with the KITTI benchmark and A2D2 dataset. Our model achieves precise edge segmentation, with a 5.59% gain in precision and a 2.12% gain in F2-score compared to the values of the original fusion method. We believe that we have introduced a new optimization idea for multimodal fusion.