Lane Markings Research Articles

Fault Injection in Model-Based System Failure Analysis of Highly Automated Vehicles

The active safety control systems of highly automated vehicles for SAE level 3 and higher are still not fully developed and facing some unresolved issues. The deployment of automated driving systems and the functional safety development present challenges in driver – machine control relationship when there is a system failure or malfunction. The current definition of the product development and controllability classes of the road vehicles functional safety (ISO26262) are not feasible in highly automated vehicles (HAV). This research developed an overview of fault or disturbance injection on the steering system of highly automated model to study the impact of steering system sensors malfunction. The approach was to study the fault propagation using a model-based engineering development in a virtual environment of MATLAB. Subsequently, the steering control system of automated vehicle was developed using an adaptive model predictive control structure to study the control system sensors failures on a system-feature level of the vehicle. It was concluded that the steering wheel angle sensor failure has a significant impact on the planned trajectory of the vehicle and thus it was classified as ASIL D, which represents the highest critical safety component and requires comprehensive safety mechanisms to meet the safety goals of the system. The study also introduced a new criterion for controllability classes suitable for highly automated systems based on the global vehicle position relative of the lane marker lines, to deal with the active safety systems and risk handling strategies. The drivers – vehicle control systems are changing significantly in SAE level 3 automated vehicle and above that driving functions are controlled by the vehicle control systems. This presents human factors challenge in this interactive system with moving to SAE levels 4 and 5. Hence, several human machine interfaces and scenario-based testing are introduced to mitigate any risk or safety uncertainty resulting from control handing-over between the driver and the vehicle control system.

Evaluation of Transverse Markings as a Speed Transition Zone Countermeasures in Small, Rural Communities

Small rural communities located along major state or county roadways typically find most of the traffic along their main thoroughfares is pass-through rather than local traffic. Unfortunately, drivers passing through these communities often enter at high rates of speeds, which are often significantly higher than the speed limit of the local segment. Speed management in rural areas requires different considerations compared to urban areas and, within the US, rural speed management is not as advanced with little experience or guidance for agencies to draw on. This paper summarizes the results of a study that evaluated, in part, several different types of transverse pavement markings within the speed transition zones in small rural communities. Three different countermeasures were evaluated: converging chevrons, transverse lane markings, and optical speed bars.

Empirical findings on infrastructure efficiency at a bicycle T-junction

In the pursuit of promoting cycling and providing better cycling infrastructure, its design is of great importance. One of the critical locations in a network are intersections, and specifically T-junctions, due to the inverted perception of priority. In this paper, we investigate the behaviour at T-junctions dedicated to cyclists and the effect on bicycle flow efficiency resulting from the introduction of lane markings that advise through cyclists to shift to the left so that merging cyclists can occupy the space on the right hand side of the cycle path. A comprehensive framework is proposed for the assessment of the T-junction efficiency. Empirical trajectory data from a large-scale cycling experiment are used for the analyses. The findings suggest that cyclist heterogeneity can be even more influential on the efficiency than the infrastructure design. Moreover, a form of self-organisation is observed, as through cyclists are willing to move to the left and allow merging cyclists to fit in the cycle path without the provision of any instructions. This means that the lane marking is both redundant and not improving the efficiency of the T-junction.

Lateral Position Measurement Based on Vehicles' Longitudinal Displacement.

The lateral position of a vehicle in its lane is crucial information required to develop intelligent assistant driving systems. Current studies reveal this information by mixing multiple sources such as cameras, LiDAR or accurate GNSS. Because these systems are not efficient in some degraded weather conditions, a cooperative Vehicle-to-Infrastructure sensor has been developed to help to determine lateral position of a vehicle in its lane. In this paper, the authors propose a completely new and original way to estimate lateral position of the vehicle in its lane using the longitudinal displacement. Using a system based on a hyper-frequency interaction between a transceiver module embedded in the vehicle and passive transponders that can be integrated in the road, for instance under the lane markings, a new signal processing algorithm is presented in order to determine the lateral distance between the vehicle and the transponder axis. The sensor has been tested in an external environment and has shown an estimated lateral distance error of 8 cm at most.

Multitask Attention Network for Lane Detection and Fitting.

Many CNN-based segmentation methods have been applied in lane marking detection recently and gain excellent success for a strong ability in modeling semantic information. Although the accuracy of lane line prediction is getting better and better, lane markings' localization ability is relatively weak, especially when the lane marking point is remote. Traditional lane detection methods usually utilize highly specialized handcrafted features and carefully designed postprocessing to detect the lanes. However, these methods are based on strong assumptions and, thus, are prone to scalability. In this work, we propose a novel multitask method that: 1) integrates the ability to model semantic information of CNN and the strong localization ability provided by handcrafted features and 2) predicts the position of vanishing line. A novel lane fitting method based on vanishing line prediction is also proposed for sharp curves and nonflat road in this article. By integrating segmentation, specialized handcrafted features, and fitting, the accuracy of location and the convergence speed of networks are improved. Extensive experimental results on four-lane marking detection data sets show that our method achieves state-of-the-art performance.

Longitudinal-Scanline-Based Arterial Traffic Video Analytics with Coordinate Transformation Assisted by 3D Infrastructure Data

High-resolution vehicle trajectory data can be used to generate a wide range of performance measures and facilitate many smart mobility applications for traffic operations and management. In this paper, a Longitudinal Scanline LiDAR-Camera model is explored for trajectory extraction at urban arterial intersections. The proposed model can efficiently detect vehicle trajectories under the complex, noisy conditions (e.g., hanging cables, lane markings, crossing traffic) typical of an arterial intersection environment. Traces within video footage are then converted into trajectories in world coordinates by matching a video image with a 3D LiDAR (Light Detection and Ranging) model through key infrastructure points. Using 3D LiDAR data will significantly improve the camera calibration process for real-world trajectory extraction. The pan-tilt-zoom effects of the traffic camera can be handled automatically by a proposed motion estimation algorithm. The results demonstrate the potential of integrating longitudinal-scanline-based vehicle trajectory detection and the 3D LiDAR point cloud to provide lane-by-lane high-resolution trajectory data. The resulting system has the potential to become a low-cost but reliable measure for future smart mobility systems.

Real-World Use of Partially Automated Driving Systems and Driver Impressions

Background: Automated driving systems (ADSs) have the potential to fundamentally transform transportation by reducing crashes, congestion, and cost while improving traffic efficiency and access to mobility for the transportation-challenged population (US DOT, 2020). However, a recent on-road test of five vehicles capable of SAE Level 2 (SAE, 2016) driving automation equipped with forward collision warning (FCW), adaptive cruise control (ACC), lane departure warning (LDW), and lane keep assist (LKA) revealed that ADSs may not work as expected in typical driving situations, such as approaching stopped vehicles and negotiating hills and curves (IIHS, 2018). Even worse, people may not use ADSs as intended due to their misunderstanding of, over-trust, or distrust in such systems’ capabilities and limitations (IIHS, 2019). As Level 2 ADSs have become commercially available, accounts of unintended uses of these systems and fatal consequences have emerged. For example, a recent news article reported a Tesla driver napping behind the wheel (Fox News, 2019). Objective: Given the growing availability of ADSs on public roadways as well as the risk of their unintended use and safety consequences, this work aimed to better understand (1) realworld use of ADSs, (2) prevalence of unintended use of such systems, and (3) driver impressions after prolonged use of such systems. Method: The research team investigated an existing naturalistic driving database collected from the Virginia Connected Corridor Level 2 Naturalistic Driving Study (VCC L2 NDS, Dunn, Dingus, & Soccolich, 2019). The dataset contains data from 50 participants who drove personally owned vehicles for 12 months. Participating vehicles were equipped with longitudinal control systems (e.g., ACC) at the minimum, although most also had lateral control systems (e.g., LKA). Specifically, we investigated safety-critical events (SCEs, Guo & Fang, 2013) of different severity levels (e.g., crashes and near-crashes) captured in the data. We also examined drivers’ responses to a post-study questionnaire that captured drivers’ subjective ratings on the usefulness and usability of the ADS. Results: We found that 47 out of 235 (20%) SCEs involved ADS use. An in-depth analysis of 47 SCEs revealed that people misused ADSs in 57% of SCEs (e.g., engaged in secondary tasks, used the systems not on highways, or with hands off the wheel). During 13% of SCEs, the systems neither reacted to the situation nor warned the driver. A post-study survey showed that people found ADSs useful and usable. However, the more participants were positive to ADS features, the more they felt comfortable engaging in secondary tasks, which is an unintended side effect of Level 2 ADSs as they require the human driver’s supervision. This study also captured some scenarios where the ADSs did not meet driver expectations. Many people reported that the longitudinal control features did not respond well to cutting-in leads (23% of participants) and stopped leads (14%). The lateral control features were often automatically disengaged when encountering blurred lane markings (14% of participants reported) and had difficulties when negotiating curves (9%). Conclusion: This study contributes to a better understanding of the capabilities and limitations of early production SAE L2 vehicles, the prevalence of the unintended use of ADSs, and drivers’ perceptions of these new technologies. Designers of human-machine interfaces (HMIs) for such systems should always consider the possibility of drivers’ overconfidence in the systems. Therefore, it might be better for vehicles to have multimodal HMIs (Large et al., 2019) adaptive to not only the urgency of situations but also driver state by monitoring driver behavior and engagement in the primary task of driving. Application: The findings from this study may inform the development of HMIs, training programs, and owner’s manuals to reduce the unintended use of ADSs and safety consequences. The identified characteristics of situations where the ADSs failed to warn drivers during SCEs will further inform the development of testing scenarios to ensure ADS safety.

How Signs, Markings, and Hazards Impact Motorist Assessment of Cyclist Lane Placement

Human factors research has led to safer interactions between motorists through redesigned signage, roadway designs, and training. Similar efforts are needed to understand and improve interactions between cyclists and motorists. One challenge to safe motorist-cyclist interactions are expectations about where cyclists should be on the road. In this study, we utilize more directive signage and additional lane markings to clarify where cyclists should ride in the travel lane. The impact of these signifiers was examined by having motorists indicate where cyclists should ride in the lane, how difficult it was to determine the correct lane position, and how safe they would feel if they were in that lane position. Results indicate that more directive signage – “bicycles take the lane”-and painted hazard signifiers can change motorists’ expectations, so they are more aligned with safer cyclist positioning in the lane.

Lane departure warning systems and lane line detection methods based on image processing and semantic segmentation: A review

Abstract Recently, the development and application of lane line departure warning systems have been in the market. For any of the systems, the key part of lane line tracking, lane line identification, or lane line departure warning is whether it can accurately and quickly detect lane lines. Since 1990s, they have been studied and implemented for the situations defined by the good viewing conditions and the clear lane markings on road. After then, the accuracy for particular situations, the robustness for a wide range of scenarios, time efficiency and integration into higher-order tasks define visual lane line detection and tracking as a continuing research subject. At present, these kinds of lane marking line detection methods based on machine vision and image processing can be divided into two categories: the traditional image processing and semantic segmentation (includes deep learning) methods. The former mainly involves feature-based and model-based steps, and which can be classified into similarity- and discontinuity-based ones; and the model-based step includes different parametric straight line, curve or pattern models. The semantic segmentation includes different machine learning, neural network and deep learning methods, which is the new trend for the research and application of lane line departure warning systems. This paper describes and analyzes the lane line departure warning systems, image processing algorithms and semantic segmentation methods for lane line detection.

The potential impacts of automated vehicles on pedestrian safety in a four-season country

Automated vehicles (AVs, level 4) are coming on the market in forthcoming years, but it is unsure what is the operational capability of these vehicles in the first place. Especially in four-season countries, winter conditions, such as snowfall and minor amount of daylight hours, increase the operational requirements of AVs. In this study, the potential impacts of AVs on pedestrian safety are evaluated in Finland. The study evaluates which in-depth investigated 40 fatal crashes between pedestrians and driver-managed cars could have been avoided in the best possible situation had the cars been replaced by the AVs. The maximum safety impacts are evaluated in three scenarios. The scenarios are based on the AV’s potentially varied abilities to operate in snowy and low-light conditions, and without lane markings. In addition, a time-to-collision analysis was made for each crash. According to the analysis, 28-73% of the pedestrian crashes are avoidable in the different scenarios. In the basic scenario, in which the operational capability of AV (level 4) is limited, solely 28% of the crashes are avoidable. In a full automation scenario (level 5), 73% of the crashes are potentially avoidable. In reality, some drivers would probably like to drive manually instead of automated driving, which would likely reduce the potential safety impacts. This hypothetical scenario is analyzed and discussed. In the countries with varied weather and road conditions, the operational requirements of the AVs are demanding. In order to ensure the maximum safety potential of the AVs in different countries, the AVs should be able to operate in various conditions and environments.

Driving Behavior Analysis of Intelligent Vehicle System for Lane Detection Using Vision-Sensor

Lane detection on the road is the most essential phase of existing advanced driver assistance systems (ADAS). In the current road scheme, the lanes are parallel tracks which are marked based on the different speed limit for the vehicles. Lane markings are defined as driving regions for the vehicles in order to avoid a collision. In an Intelligent Vehicle System (IVS), these markings are being identified by various massive computing techniques and sensor-based mechanisms like radar, LiDAR, and GPS and have high operational costs. However, vision-based detection methods are gaining popularity due to its low cost and better understanding capability of the scene. The proposed study follows computer vision and image processing techniques to examine and learn significant lane features. The approach demonstrates the practical and effective usage of the lane detection model (LDM). The LDM has been designed with a vehicle prototype using Raspberry Pi, which reveals the intelligent behavior against the detected lane lines. The model is less expensive and power-efficient, which confirms the operational capability of the proposed system under a heterogeneous environment.

Robust Lane Extraction From MLS Point Clouds Towards HD Maps Especially in Curve Road

This article presents a semi-automated method to extract the lane features along the curved roads from mobile laser scanning (MLS) point clouds. The proposed method consists of four steps. After data pre-processing, a road edge detection algorithm is performed to distinguish road curbs and extract road surfaces. Then, textual and directional road markings such as arrows, symbols, and words, to inform drivers in necessary cases, are detected by intensity thresholding and conditional Euclidean clustering algorithms. Furthermore, lane markings are extracted by local intensity analysis and distance thresholding methods according to road design standards, because they are more regular along the road. Finally, centerline points on lanes are estimated based on the coordinates of extracted lane markings. Our method shows strong feasibility and robustness when creating high-definition (HD) maps from MLS data, by increasing the number of blocks in the curve and the distance threshold control in curved lane centerline extraction. Quantitative evaluations show that the average recall, precision, and F1-score obtained from four datasets for road marking extraction are 93.87%, 93.76%, and 93.73%, respectively. The generated lane centerlines are evaluated by overlaying them on manually labeled reference buffers from 4 cm resolution orthoimagery. The comparative study indicates that the proposed methods can achieve higher accuracy and robustness than most state-of-the-art methods.

One stage detector (RetinaNet)-based crack detection for asphalt pavements considering pavement distresses and surface objects

In this study, a supervised machine learning network model is proposed to detect and classify various types of cracks developed in asphalt pavements, including lane markers. Crack images captured from a digital camera are classified into nine categories following the pavement distress identification manual proposed by the Federal Highways Administration (FHWA). These categories are three different types of cracks, such as fatigue, longitudinal, and transverse cracks with three severity levels of the low, medium, and high for each crack type. To establish a training dataset for crack detection, 1000 images with the original size of 3704 × 10,000 pixels are divided into 20,000 smaller images of 1852 × 1000 pixels image size. The training images are labeled based on the nine categories and trained using an updated version of faster R-CNN called RetinaNet. The trained network model is validated using pavement surface images obtained from 2400 m of two road sections. It is observed from the validation study that the detection and classification accuracy of the trained network model is 84.9% considering both the crack type and severity level. When considering the crack type only, the detection accuracy of the network model is 89.1%.

Lane-Level Map Matching Based on HMM

Lane-level map matching is essential for autonomous driving. In this paper, we propose a Hidden Markov Model (HMM) for matching a trajectory of noisy GPS measurements to the road lanes in which the vehicle records its positions. To our knowledge, this is the first time that HMM is used for lane-level map matching. Apart from GPS values, the model is further assisted by yaw rate data (converted to a lane change indicator signal) and visual cues in the form of the left and right lane marking types (dashed, solid, etc.). Having defined expressions for the HMM emission and transition probabilities, we evaluate our model to demonstrate that it achieves 95.1% recall and 3.3% median path length error for motorway trajectories.

Automated Annotation of Lane Markings Using LIDAR and Odometry

Lane markings are a key element for Autonomous Driving. The generation of high definition maps and ground-truth data require extensive manual labor. In this paper, we present an efficient and robust method for the offline annotation of lane markings, using low-density LIDAR point clouds and odometry information. The odometry is used to accumulate the scans and to process them using blocks following the trajectory of the vehicle. At each block, candidate lane marking points are detected by generating virtual scan-lines and applying a dynamically optimized filter function to the LIDAR intensity values. The lane markings are tracked block wise, and their width is estimated and classified as either solid or dashed. The results are lists of connected 3D points that represent the different lane markings. The accuracy of the proposed method was tested against manually labeled recordings. A novel evaluation methodology focused on the lateral precision of detections is presented. Moreover, a web user interface was used to load the produced annotations, achieving a reduction of 60% in the annotation time, as compared to a fully manual baseline.

Identifying safe intersection design through unsupervised feature extraction from satellite imagery

AbstractThe World Health Organization has listed the design of safer intersections as a key intervention to reduce global road trauma. This article presents the first study to systematically analyze the design of all intersections in a large country, based on aerial imagery and deep learning. Approximately 900,000 satellite images were downloaded for all intersections in Australia and customized computer vision techniques emphasized the road infrastructure. A deep autoencoder extracted high‐level features, including the intersection's type, size, shape, lane markings, and complexity, which were used to cluster similar designs. An Australian telematics data set linked infrastructure design to driving behaviors captured during 66 million kilometers of driving. This showed more frequent hard acceleration events (per vehicle) at four‐ than three‐way intersections, relatively low hard deceleration frequencies at T‐intersections, and consistently low average speeds on roundabouts. Overall, domain‐specific feature extraction enabled the identification of infrastructure improvements that could result in safer driving behaviors, potentially reducing road trauma.

Detecting and Tracking Unsafe Lane Departure Events for Predicting Driver Safety in Challenging Naturalistic Driving Data.

Our goal is to improve driver safety predictions in at-risk medical or aging populations from naturalistic driving video data. To meet this goal, we developed a novel model capable of detecting and tracking unsafe lane departure events (e.g., changes and incursions), which may occur more frequently in at-risk driver populations. The model detects and tracks roadway lane markings in challenging, low-resolution driving videos using a semantic lane detection pre-processor (Mask R-CNN) utilizing the driver's forward lane region, demarking the convex hull that represents the driver's lane. The hull centroid is tracked over time, improving lane tracking over approaches which detect lane markers from single video frames. The lane time series was denoised using a Fix-lag Kalman filter. Preliminary results show promise for robust lane departure event detection. Overall recall for detecting lane departure events was 81.82%. The F1 score was 75% (precision 69.23%) and 70.59% (precision 62.07%) for left and right lane departures, respectively. Future investigations include exploring (1) horizontal offset as a means to detect lead vehicle proximity, even when image perspectives are known to have a chirp effect and (2) Long Short Term Memory (LSTM) models to detect peaks instead of a peak detection algorithm.

The impact of a child bike seat and trailer on the objective overtaking behaviour of motorized vehicles passing cyclists

IntroductionThe lateral clearance distance of a motorized vehicle while overtaking a cyclist is a key indicator of safety. This lateral clearance distance has never been measured for cyclists transporting a child. Therefore the aim of this study was to investigate the behaviour of motorized vehicles in overtaking cyclists with and without a child on the same bicycle. MethodsThe lateral clearance distance of the overtaking manoeuvres of motorized vehicles was measured using an instrumented bicycle when performing 19 cycling trips on one single road with two different types of cycling infrastructure (a bike lane and shared lane marking) in the Brussels Capital Region (Belgium). Mixed effect regression was used to examine the effect of cycling condition (cyclist without a child [control], cyclist with a child bike seat and cyclist with a child bike trailer) and secondary independent variables (i.e. cycling infrastructure, peak traffic hours and traffic density) on the lateral clearance distance. ResultsThe mean lateral clearance distance in ‘cyclists without child’ was significantly smaller (117.3 cm) than in ‘cyclists with child’ (128.8 cm) (95%CI [7.2;15.9]). Looking at ‘morning peak traffic hours’ (i.e. 7:00 to 9:00 a.m.) a cyclist with child bike seat was overtaken at greater lateral clearance distances than a cyclist with a child bike trailer or a cyclist without child (p=0.041). Furthermore, the percentage of passing manoeuvres under 100 cm was significantly higher in ‘cyclist without child’ (35.3%) in comparison to ‘cyclist with child bike seat’ (21.8%) and ‘cyclist with child bike trailer’ (21.8%) (Chi2=29.19, p<0.001). No significant differences were found between a shared lane marking and bike lane. ConclusionDrivers of motorized vehicles do adapt their overtaking manoeuvre when they overtake cyclists transporting a child, keeping greater and therefore safer lateral clearance distances. In morning peak traffic hours and/or crowded circumstances the child bike seat can be considered as the safest way to transport a child in terms of lateral clearance distance. In general, with 25.3% overtaking manoeuvres under 100cm, police should monitor compliance with the traffic rules with regard to cyclists, and motorists should be more aware of the overtaking traffic rules.

Intelligent Vehicle Self-Localization Based on Double-Layer Features and Multilayer LIDAR

High-precision self-localization is one of the basic functions of intelligent vehicles. Moreover, location information is necessary prior information for tasks such as behavioural decision-making and path planning. Recently, the technology of map-based localization has been widely concerned due to its accuracy. A prior map for localization stores one or more environmental features extracted by on-board sensors. These features represent fundamental information about the ground environment such as curbs and lane markings, or the vertical environment such as building outlines. In order to enhance the robustness and positioning accuracy, this paper uses two layers of features to express the environment: one is a bottom layer composed of ground and curb features, and the other is an upper layer composed of a 2D point cloud with vertical features. Firstly, a novel collision-based detection method is proposed to extract the curb features, and a vertical-projection-based detection algorithm is used to detect vertical feature points from multilayer LIDAR. Then, the fusion of a multi-frame feature point cloud is used to represent the rich and complete information on the environment of the intelligent vehicle. Finally, the Monte Carlo localization algorithm is used to obtain an optimal estimate of the vehicle position. The experimental results indicate that the proposed localization algorithm can realize localization accuracy.

Gradient-Based Edge Effects on Lane Marking Detection using a Deep Learning-Based Approach

Lane detection is part of the advanced driver assistance system (ADAS) equipped in intelligent vehicles. The system provides the driver with significant geometric information of the road ahead. Numerous deep learning techniques have been employed in lane detection because of the simplicity, ease, and efficiency of these techniques in learning discriminative features from RGB (red, green, and blue) images. However, existing works have rarely considered detecting lane markings during bad weather conditions, which could reduce lane detection performance. Hence, this paper proposed a Fully Convolutional Network (FCN) model with RGB and Canny edge detection used as the model’s spatial input. The proposed platform was developed using two scenarios: FCN-RGB-edge and FCN-edge. The model development was divided into three stages, namely data acquisition, platform development, and benchmarking against existing methods and data. Both scenarios using the proposed method yielded a 4% improvement compared to the original FCN-RGB images (i.e., the previous method). The Canny edge detection method successfully extracted necessary information from the images and neglected the water drops in rainy conditions by treating them as noise. In summary, the proposed method has the potential to boost the performance of the ADAS system in detecting lane markings in rainy conditions.