Lane Markings Research Articles

ST-MAE: robust lane detection in continuous multi-frame driving scenes based on a deep hybrid network

Lane detection is one of the key techniques to realize advanced driving assistance and automatic driving. However, lane detection networks based on deep learning have significant shortcomings. The detection results are often unsatisfactory when there are shadows, degraded lane markings, and vehicle occlusion lanes. Therefore, a continuous multi-frame image sequence lane detection network is proposed. Specifically, the continuous six-frame image sequence is input into the network, in which the scene information of each frame image is extracted by an encoder composed of Swin Transformer blocks and input into the PredRNN. Continuous multi-frame of the driving scene is modeled as time-series by ST-LSTM blocks, and then, the shape changes and motion trajectory in the spatiotemporal sequence are effectively modeled. Finally, through the decoder composed of Swin Transformer blocks, the features are obtained and reconstructed to complete the detection task. Extensive experiments on two large-scale datasets demonstrate that the proposed method outperforms the competing methods in lane detection, especially in handling difficult situations. Experiments are carried out based on the TuSimple dataset. The results show: for easy scenes, the validation accuracy is 97.46%, the test accuracy is 97.37%, and the precision is 0.865. For complex scenes, the validation accuracy is 97.38%, the test accuracy is 97.29%, and the precision is 0.859. The running time is 4.4 ms. Experiments are carried out based on the CULane dataset. The results show that, for easy scenes, the validation accuracy is 97.03%, the test accuracy is 96.84%, and the precision is 0.837. For complex scenes, the validation accuracy is 96.18%, the test accuracy is 95.92%, and the precision is 0.829. The running time is 6.5 ms.

Safety evaluation of urban roundabouts in India: a safety performance function-based approach

There is a lack of comprehensive research in roundabout safety under non-lane-based traffic conditions, mainly due to the absence of relevant crash data and effective tools for safety evaluation. Safety Performance Function (SPF) is a suitable tool for providing valuable information related to factors which can potentially contribute to the likelihood of increasing traffic crashes. Currently, very limited studies are available to explain the usefulness of SPF in the vicinity of roundabouts, especially at non-lane-based traffic conditions. This study aims to develop an SPF model for assessing the safety evaluation at roundabouts as a whole (intersection level) and the approach level. Data corresponding to crashes in nineteen roundabouts with different geometric and traffic characteristics was used for model formulation. Crash data for five years (2015–2019) was obtained from the State Crime Records Bureau. An SPF model was developed using a negative binomial model with a log-link function based on the number of crashes, traffic characteristics, and geometry characteristics of the roundabouts. The proportion of powered two-wheelers, percentage of heavy vehicles, entry-angle, and weaving-length were all significantly related with higher crash occurrences at roundabouts, according to the findings. In contrast, the number of circulatory lanes, inscribed circle diameter, and presence of road lane marking were negatively associated with the increased crash occurrences at the roundabout vicinity. In addition to this, the overall crash rate significantly varies across roundabout sections due to the asymmetric effects of geometric and traffic characteristics. The developed SPF would best explain the relationship between geometric and traffic characteristics and the crash occurrence rate in non-lane traffic conditions. The findings of this study support the need to relook at design parameters for better movement at the roundabouts, thereby improving the existing facilities to enhance road users' safety, especially in developing countries. The proposed SPF tool would help engineers examine the safety of roundabouts in terms of design adequacy, quantifying the risk factors, and future crash predictions.

High-Definition Map Representation Techniques for Automated Vehicles

Many studies in the field of robot navigation have focused on environment representation and localization. The goal of map representation is to summarize spatial information in topological and geometrical abstracts. By providing strong priors, maps improve the performance and reliability of automated robots. Due to the transition to fully automated driving in recent years, there has been a constant effort to design methods and technologies to improve the precision of road participants and the environment’s information. Among these efforts is the high-definition (HD) map concept. Making HD maps requires accuracy, completeness, verifiability, and extensibility. Because of the complexity of HD mapping, it is currently expensive and difficult to implement, particularly in an urban environment. In an urban traffic system, the road model is at least a map with sets of roads, lanes, and lane markers. While more research is being dedicated to mapping and localization, a comprehensive review of the various types of map representation is still required. This paper presents a brief overview of map representation, followed by a detailed literature review of HD maps for automated vehicles. The current state of autonomous vehicle (AV) mapping is encouraging, the field has matured to a point where detailed maps of complex environments are built in real time and have been proved useful. Many existing techniques are robust to noise and can cope with a large range of environments. Nevertheless, there are still open problems for future research. AV mapping will continue to be a highly active research area essential to the goal of achieving full autonomy.

Camera–Lidar sensor fusion for drivable area detection in winter weather using convolutional neural networks

Autonomous vehicles rely on perceiving the road environment through a perception pipeline fed by a variety of sensor modalities, including camera, lidar, radar, infrared, gated camera, and ultrasonic. The vehicle computer must perform sensor fusion reliably in various sensor signals degrading environments, which can be performed parametrically or using machine learning. We study sensor fusion of a forward-facing camera and a lidar sensor using convolutional neural networks (CNNs), for drivable area detection in winter driving, where the roads are partially covered with snow, tire tracks obscuring lane markers. Seven fusion models are developed and evaluated for their ability to classify pixels into two classes: drivable and nondrivable. A total of seven models are designed and tested, including camera only, lidar only, early fusion, (three types of) intermediate fusion, and late fusion. The models have accuracies between 84% and 89%, with runtimes between 23 and 61 ms. To select the best model from this group, we introduce a unique metric, named normalized accuracy runtime (NAR) score. A network in a group has a higher NAR score, the more accurate it is, and the shorter its runtime is. The models are evaluated on a winter driving DENSE dataset. Camera and lidar fog noise are synthesized to examine model robustness.

A Comprehensive Review on Lane Marking Detection Using Deep Neural Networks.

Lane marking recognition is one of the most crucial features for automotive vehicles as it is one of the most fundamental requirements of all the autonomy features of Advanced Driver Assistance Systems (ADAS). Researchers have recently made promising improvements in the application of Lane Marking Detection (LMD). This research article has taken the initiative to review lane marking detection, mainly using deep learning techniques. This paper initially discusses the introduction of lane marking detection approaches using deep neural networks and conventional techniques. Lane marking detection frameworks can be categorized into single-stage and two-stage architectures. This paper elaborates on the network’s architecture and the loss function for improving the performance based on the categories. The network’s architecture is divided into object detection, classification, and segmentation, and each is discussed, including their contributions and limitations. There is also a brief indication of the simplification and optimization of the network for simplifying the architecture. Additionally, comparative performance results with a visualization of the final output of five existing techniques is elaborated. Finally, this review is concluded by pointing to particular challenges in lane marking detection, such as generalization problems and computational complexity. There is also a brief future direction for solving the issues, for instance, efficient neural network, Meta, and unsupervised learning.

Illumination-Resilient Lane Detection by Threshold Self-Adjustment Using Newton-Based Extremum Seeking

The ability to detect lane markings under varying lighting conditions is essential for autonomous mobile robots and automatic vehicle driving assistance systems. Because the object color information is subject to illumination variation, this article presents a novel and computationally efficient algorithm based on the extremum-seeking method to achieve illumination-resilient lane detection and path-following tasks for autonomous driving. Lane detection is performed in the hue-saturation-value color space by distinguishing the colored lane marks from the background. The system’s inputs are the upper and lower thresholds in each of the hue, saturation, and value channels. We define a cost function as the combination of detection accuracy and lane coverage to evaluate the algorithm performance. Two extremum-seeking schemes, one with fixed dither amplitudes and another with adaptive dither amplitudes, are designed to adjust the system inputs to minimize the cost function. The two proposed methods are validated and compared through video-based simulation studies and scaled-car field experimental tests.

Tunnel Facility Based Vehicle Localization in Highway Tunnel Using 3D LIDAR

Vehicle localization in highway tunnels is a challenging issue for autonomous vehicle navigation. Since GPS signals from satellites cannot be received inside a highway tunnel, map-aided localization is essential. However, the environment around the tunnel is composed mostly of an elliptical wall. Thereby, the unique feature points for map matching are few unlike the case outdoors. As a result, it is a very difficult condition to perform vehicle navigation in the tunnel with existing map-aided localization. In this paper, we propose tunnel facility-based precise vehicle localization in highway tunnels using 3D LIDAR. For vehicle localization in a highway tunnel, a point landmark map that stores the center points of tunnel facilities and a probability distribution map that stores the probability distributions of the lane markings are used. Point landmark-based localization is possible regardless of the number of feature points, if only representative points of an object can be extracted. Therefore, it is a suitable localization method for highway tunnels where the feature points are few. The tunnel facility points were extracted using 3D LIDAR. Position estimation is conducted using an EKF-based navigation filter. The proposed localization algorithm is verified through experiments using actual highway driving data. The experimental results verify that the tunnel facility-based vehicle localization yields precise results in real time.

Tracking Dependent Extended Targets Using Multi-Output Spatiotemporal Gaussian Processes

In Extended Target Tracking, where estimating the shape is essential as kinematic, exploiting the dependencies between targets is often an excellent way to enhance performance. In a group of dependent targets, sampled features tend to have spatially and temporally correlations inside and between frames. Gaussian process regression has been used as a powerful Bayesian semi-supervised method to describe functions’ spatial and temporal correlation. This paper exploits and models the dependency between extended targets using Gaussian Process. We propose a novel recursive approach called Multi-Output Spatio-Temporal Gaussian Process Kalman Filter (MO-STGP-KF) to estimate and track multiple dependent extended targets that have possibly been degraded or covered with clutter. We used this method for detecting and tracking the group of connected lane markings called “lane-lines”. For detection and clustering, we propose a new Kernel-based Joint Probabilistic Data Association Coupled Filter (K-JPDACF) to cluster point features belonging to each lane-line. Compared to recently published model-based multi-lane tracking, semi-supervised, and fully supervised lane detection methods, our method shows 13 percent 34 percent and 20 percent improvement in accuracy, respectively.

A Deep Learning Instance Segmentation Approach for Lane Marking Detection

A Deep Learning Instance Segmentation Approach for Lane Marking Detection

Cooperative path and trajectory planning for autonomous vehicles on roads without lanes: A laboratory experimental demonstration

Autonomous vehicles that travel without considering the lane marks and utilizing all road width have an opportunity to maximize the use of vehicles’ performance. By taking advantage of the entire width of curvy roads and the cooperative behavior of connected autonomous vehicles, new options for path planning can be implemented while utilizing the existing infrastructure. The proposed cooperative controller uses a nonlinear model predictive control (NMPC) approach for dozens of autonomous vehicles without considering lane marks. This controller maximizes vehicles’ progress on the road with minimal control efforts while complying with design constraints imposed by road geometry, distances between vehicles, and vehicle dynamics. The controller is tested in two simulation case studies. The first examines the performance under two different plant (reality) models. The second considers dozens of vehicles and compares the traffic flow characteristics between the lane-free concept and the lane-based concept within different vehicle densities. The simulation results show that the lane-free concept can improve the traffic flow performance compared with the lane-based road concept, i.e. reducing passengers’ time on the road, reducing energy consumption, and increasing road capacity. These improvements depend on the road density and track layout. In order to demonstrate the proposed controller, three laboratory experiments with several homogeneous and heterogeneous robots were conducted.

Generalized LiDAR Intensity Normalization and Its Positive Impact on Geometric and Learning-Based Lane Marking Detection

Light Detection and Ranging (LiDAR) data collected by mobile mapping systems (MMS) have been utilized to detect lane markings through intensity-based approaches. As LiDAR data continue to be used for lane marking extraction, greater emphasis is being placed on enhancing the utility of the intensity values. Typically, intensity correction/normalization approaches are conducted prior to lane marking extraction. The goal of intensity correction is to adjust the intensity values of a LiDAR unit using geometric scanning parameters (i.e., range or incidence angle). Intensity normalization aims at adjusting the intensity readings of a LiDAR unit based on the assumption that intensity values across laser beams/LiDAR units/MMS should be similar for the same object. As MMS technology develops, correcting/normalizing intensity values across different LiDAR units on the same system and/or different MMS is necessary for lane marking extraction. This study proposes a generalized correction/normalization approach for handling single-beam/multi-beam LiDAR scanners onboard single or multiple MMS. The generalized approach is developed while considering the intensity values of asphalt and concrete pavement. For a performance evaluation of the proposed approach, geometric/morphological and deep/transfer-learning-based lane marking extraction with and without intensity correction/normalization is conducted. The evaluation shows that the proposed approach improves the performance of lane marking extraction (e.g., the F1-score of a U-net model can change from 0.1% to 86.2%).

The Effects of Tactile Display on Automated Vehicle Takeover: A Literature Review

Current autonomous vehicles are still semi-autonomous and require drivers to take over control of the vehicle under various conditions (e.g., encountering erased lane markings). This presents an issue in human-machine interaction, as there is a need for reliable methods to guide drivers through the sudden and complex takeover process. A tactile display that can present multi-dimensional information (e.g., status, direction, and position) may be a good option, especially when the information presented in their visual and auditory channels is overloading. However, limited work has attempted to summarize this topic. Therefore, the goal of this study is to synthesize literature that examined the effects of tactile display on takeover performance in automated vehicles. Findings indicated that tactile displays can be placed in multiple in-vehicle locations to present various vibrotactile patterns helping to improve drivers’ performance during the automated vehicle takeover, which could inform the design of human-machine interfaces for autonomous vehicles.

Creating Semantic HD Maps From Aerial Imagery and Aggregated Vehicle Telemetry for Autonomous Vehicles

We present a new framework for creating lane-level detailed HD-maps at scale for autonomous vehicles (AVs). In order to overcome scaling challenges of ground survey-based HD-map creation, we propose a number of innovations by leveraging three data sources: high-resolution aerial imagery, aggregated vehicle telemetry, and a navigation map. We first divide map creation problem into several categories based on lane configurations. The road category is predicted in a supervised setting using aerial imagery, which is pre-processed by using aggregated vehicle telemetry without supervision. We utilized the navigation map to guide the process along the road network and used aerial imagery and aggregated vehicle telemetry to extract lane level features at each step. We propose a multi-task convolutional neural network (CNN) to predict road polygons, road-way centerline, number of lanes, lane and road edges using both the aerial imagery and the corresponding aggregated vehicle telemetry. The predicted road features for each image are then stitched along a road segment to construct the road and lane edge polylines, which are then used to predict lane marking and road edge types in a sliding window fashion along the road segment. The extracted features are finally utilized to calculate higher level features for each point in the HD-map. Our experimental results show that the proposed framework works well, offering a flexible solution for creating HD-maps for AVs at scale.

Saturation flow rate analysis for special width approach lanes: An empirical study in Karlsruhe, Germany

The special width approach lane (SWAL) is a newly proposed unconventional design, whereby a wide approach lane is divided into two narrower lanes. The design entails the use of a single lane by two passenger cars or one heavy vehicle. Such design has been applicated at signalized intersections of Karlsruhe, Germany. This paper focuses on the saturation flow rate analysis since most existing studies on such design rely on the default highway capacity manual (HCM) values. Saturation flow rate data was collected at four SWAL design based signalized intersections with procedural steps of the HCM 2010 using the video camera. The two-sample t-test was performed to explore the potential influencing factors, and then the non-linear regression analysis was conducted to estimate the saturation flow rate of SWAL. The proposed model can effectively depict the saturation flow rate with lane marking, presence of cyclists, and rainfall being the influencing factors. The overall accuracy of the proposed model is about 95%. The results indicate that the three influencing factors are independent of each other. The existence of cyclists and rainfall lead to a decrease in the saturation flow rate, while the lane markings can improve the saturation flow rate. Moreover, the SWAL works well in Karlsruhe, Germany. The model predicts a base saturation flow rate value of 1652 pcu/h/ln, which is plausible with comparison of the base saturation flow rate recommended in the German Highway Capacity Manual.

Efficient lane marking detection using deep learning technique with differential and cross-entropy loss

<p>Nowadays, researchers are incorporating many modern and significant features on advanced driver assistance systems (ADAS). Lane marking detection is one of them, which allows the vehicle to maintain the perspective road lane. Conventionally, it is detected through handcrafted and very specialized features and goes through substantial post-processing, which leads to high computation, and less accuracy. Additionally, this conventional method is vulnerable to environmental conditions, making it an unreliable model. Consequently, this research work presents a deep learning-based model that is suitable for diverse environmental conditions, including multiple lanes, different daytime, different traffic conditions, good and medium weather conditions, and so forth. This approach has been derived from plain encode-decode E-Net architecture and has been trained by using the differential and cross-entropy losses for the backpropagation. The model has been trained and tested using 3,600 training and 2,700 testing images from TuSimple, a robust public dataset. Input images from very diverse environmental conditions have ensured better generalization of the model. This framework has reached a max accuracy of 96.61%, with an F1 score of 96.34%, a precision value of 98.91%, and a recall of 93.89%. Besides, this model has shown very small false positive and false negative values of 3.125% and 1.259%, which bits the performance of most of the existing state of art models.</p>

Collision Avoidance and Direction Planning for Autonomous Vehicles

The rapid development of information and signal processing technology has contributed significantly to the development of autonomous driving (AD) techniques, which improve driving safety while minimizing human efforts. According to analyst predictions, automatic cars will soon surpass manual cars in number as automatic driving becomes more sophisticated. Ensuring safety and reducing the road accidents is the primary concerns in automated vehicles. Recent research has shown that computer vision, deep learning, and other fields have advanced beyond the imagination. The paper covers the deep learning algorithms and techniques necessary to make a reliable and real-time collision avoidance system using concepts such as Convolutional Neural Networks (CNN), YOLOv4 (You Look Only Once), and other deep learning architectures while also reviewing the current state of the art strategies for Advanced Driver Assistance System (ADAS). By using CNN algorithms, it is possible to detect lane markings and signboards in real time as well as estimate distances between them.

Lane Detection on Roads using Computer Vision

Lane Detection Systems which functions with the purpose to fete the lane borders on road and farther prompts the motorist if he changes and moves to wrong lane markings. Lane detecting system is an essential element of numerous technologically intelligent transport systems. Although it’s a complicate thing to reach because of scrupling road conditions that a person encounters especially while driving at night or indeed in daylight. Lane boundaries are detected using a camera that captures the view of the road, mounted on the front of the vehicle. Given a picture captured from a camera hooked up to a vehicle moving on a road during which captured road could or might not be leveled, or have easily described edges, or some former conceded patterns thereon, also road discovery from an image will be applied to hunt out the road in a picture so it might used as a place in robotization of driving system within the vehicles for moving the vehicle in precise road. In this system of chancing the road within the image prisoner by the vehicle, we're suitable to use some algorithms for vanish point discovery, exploitation Hough Transformation Space, chancing the region of interest, edge discovery exploitation canny edge discovery for road discovery. We've a tendency to use thousands of filmland of colorful roads to trainer our model so the model might notice the road as a result within the new image reused through the vehicle.

Lightweight lane marking detection CNNs by self soft label attention

Lightweight lane marking detection CNNs by self soft label attention

Research on the transition marking between the broken line and solid line of expressway

Drivers often cross the solid line unintentionally during the lane-changing, when the sight distance is blocked by the vehicle ahead. There are significant potential risks to change lanes in solid line area, which would cause traffic flow disruption and traffic safety hazards. If the drivers with limited sight distance can be reminded in advance that the lane marking ahead is a solid line, many drivers will refrain from changing lanes in the restricted area, therefore, it is important to help the drivers with limited sight distance to obtain the information of lane marking ahead. There is limited research on this specific issue presently, nor any relevant regulations or manuals available in China. This paper puts forward a novel concept of transition marking, and a creative approach by adding the transition marking between the broken line and the solid line. The transition marking is considered an information carrier to predict the pass-through right of the road ahead. We have studied the form and length of the transition marking and evaluated its effectiveness through the driving simulation experiment. The form analysis shows that the white mutation broken line has better visibility and legibility. The transition marking length is shown to be closely related to the vehicle speed and distance. The simulation results indicate that the transition marking length is negatively correlated with vehicle distance and positively correlated with speed. Based on the simulation results and detailed statistical analysis, the reference value of transition marking length is proposed. The effectiveness evaluation demonstrates that the transition marking reduces the probability of solid-line crossing during lane-changing dramatically by 92%.

Vehicle trajectory prediction on highways using bird eye view representations and deep learning

This work presents a novel method for predicting vehicle trajectories in highway scenarios using efficient bird’s eye view representations and convolutional neural networks. Vehicle positions, motion histories, road configuration, and vehicle interactions are easily included in the prediction model using basic visual representations. The U-net model has been selected as the prediction kernel to generate future visual representations of the scene using an image-to-image regression approach. A method has been implemented to extract vehicle positions from the generated graphical representations to achieve subpixel resolution. The method has been trained and evaluated using the PREVENTION dataset, an on-board sensor dataset. Different network configurations and scene representations have been evaluated. This study found that U-net with 6 depth levels using a linear terminal layer and a Gaussian representation of the vehicles is the best performing configuration. The use of lane markings was found to produce no improvement in prediction performance. The average prediction error is 0.47 and 0.38 meters and the final prediction error is 0.76 and 0.53 meters for longitudinal and lateral coordinates, respectively, for a predicted trajectory length of 2.0 seconds. The prediction error is up to 50% lower compared to the baseline method.