Road Images Research Articles

CSegNet: a hybrid transformer-CNN network for road crack image segmentation

Detecting cracks from optical images plays a crucial role in road maintenance but its good realisation has many challenges. Road cracks exhibit significant diversity and complexity in terms of shape, size and texture and road images may contain various types of noise and interference, such as lighting variations, shadows and different appearances, due to varying perspectives and scales. To address these challenges, a comprehensive road crack dataset called CRCrack has been constructed, which encompasses various crack characteristics. This study proposes a road crack segmentation network called CSegNet, which combines convolutional neural networks (CNNs) and transformers. The network adopts an encoder-decoder framework, namely DeepLabV3+. In the encoder, leveraging the transformers' flexibility in modelling long-term dependencies and the CNNs' ability to capture local contextual information through local receptive fields, weight sharing and spatial subsampling, a ResNeXt-Transformer (ResNeXTR) feature extraction module is designed as the backbone network to enhance the feature extraction capability for road crack images. To reduce the computational cost in the self-attention (SA) computation of the transformer, an average pooling layer is introduced to downsample the dimensions of the encoded features. In the decoder, to focus on the key information of road cracks under diverse environmental conditions and interferences, an efficient channel attention module (ECAM) and a spatial attention module (SAM) are combined to design an efficient convolutional block attention module (ECBAM) to optimise feature representation. Through comparative experiments on the CRCrack dataset, the results demonstrate that the proposed method outperforms classic networks such as U-Net and DeepLabV3+ in terms of intersection over union (IoU), Dice coefficient and area under the receiver operating characteristic (AUROC) curve evaluation metrics. It exhibits good adaptability to ground crack images from different sources, providing a basis for estimating the degree of road damage.

Enhancing road image clarity with residual neural network dehazing model

Lane markers, or road markers, are the painted lines on a roadway that separate different lanes of traffic. Lane markers guide drivers and ensure orderly vehicle flow. They are essential for advanced driver assistance systems (ADAS), providing reference points for vehicle positioning on the road. These markers enable ADAS to give warnings, assistance, and automation features that enhance driver safety and convenience. However, unpredictable illumination, such as a foggy environment, can suppress marker visibility, impacting ADAS's performance. Deep learning-based methods are well-known for their superiority in handling various haze patterns. This paper presents a residual network (ResNet)-based deep learning model to improve road image clarity impacted by fog. The residual neural network dehaze model (RNN-D) utilises a joint loss function to produce haze-free images with improved lighting conditions and enhanced details. The model was trained, validated, and fine-tuned using hazy and corresponding non-hazy datasets to ensure that the model is quantitatively superior in the peak signal-to-noise ratio (PSNR) and the structural similarity index measure (SSIM). RNN-D achieved an average PSNR of 27.98 and SSIM of 0.8 on multiple open sourced datasets. The proposed algorithm's superior performance and visually appealing results make it a powerful tool for real-world image dehazing applications.

Intelligent Transport System for Lane Detection and Pothole Detection using YOLO

This paper aims to develop an automated system for pothole and lane marking detection on roads using the YOLOv4- tiny model with a supporting database. The traditional methods of detecting road conditions are slow and inefficient, as they are based on manual processing. With computer vision, our system will identify potholes and lanes much more quickly and accurately. The YOLOv4-tiny model is chosen because it provides a good balance between speed and accuracy, making it suitable for real-time applications. The database of images of roads with potholes and lanes will support training and testing of the models to achieve reliable detection. It is anticipated that this paper will be helpful in the improvement of road safety and maintenance as it will be able to provide timely information about road conditions.

Leveraging a deep learning generative model to enhance recognition of minor asphalt defects

Deep learning-based computer vision systems have become powerful tools for automated and cost-effective pavement distress detection, essential for efficient road maintenance. Current methods focus primarily on developing supervised learning architectures, which are limited by the scarcity of annotated image datasets. The use of data augmentation with synthetic images created by generative models to improve these supervised systems is not widely explored. The few studies that do focus on generative architectures are mostly non-conditional, requiring extra labeling, and typically address only road crack defects while aiming to improve classification models rather than object detection. This study introduces AsphaltGAN, a novel class-conditional Generative Adversarial Network with attention mechanisms, designed to augment datasets with various rare road defects to enhance object detection. An in-depth analysis evaluates the impact of different loss functions and hyperparameter tuning. The optimized AsphaltGAN outperforms state-of-the-art generative architectures on public datasets. Additionally, a new workflow is proposed to improve object detection models using synthetic road images. The augmented datasets significantly improve the object detection metrics of You Only Look Once version 8 by 33.0\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document}, 3.8\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document}, 46.3\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document}, and 51.8\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} on the Road Damage Detection 2022 dataset, Crack Dataset, Asphalt Pavement Detection Dataset, and Crack Surface Dataset, respectively.

MonoFG: Monocular 3D Object Detection with Knowledge Distillation for Human-Centric Autonomous Driving Systems

Monocular 3D object detection is essential for identifying objects in road images, thus offering valuable environmental perception data that are crucial for human-centric autonomous driving systems. However, due to the inherent limitations of camera imaging, obtaining precise depth information from images alone is challenging, which hampers the accuracy of within-scene object localization. In this paper, we introduce a monocular 3D object detection method called MonoFG that uses knowledge distillation with separated foreground and background components to improve the accuracy of object localization. First, detached foreground and background distillation processes can strategically leverage the distinct positional information acquired from each location to optimize the produced global distillation effects. This step serves as the foundation for the subsequent feature and response distillation process, which focuses on the distilled foreground and background rather than isolated object distillation. Second, triple attention mechanism-based feature distillation intensifies the feature imitation and feature representation capabilities of the student network. Spatial and channel attention mechanisms encourage the student network to capture crucial pixels and channels from the teacher network, whereas a self-attention mechanism globally transfers the learned relationships between pixels. Third, localization error-based response distillation facilitates a clearer transfer of positional information from the teacher network to the student network. Only when the positioning ability of the teacher network exceeds that of the student network can knowledge be comprehensively distilled across both the foreground and background. Therefore, the distillation process is constrained to specific content, which is delineated by positioning errors that serve as the boundaries. Finally, experiments conducted on the KITTI benchmark dataset demonstrate that our method outperforms many well-known baseline methods in several representative evaluation tasks (e.g., 3D object detection and bird's-eye view (BEV) detection) involving human-centric autonomous driving systems.

Detection of Road Damages Using Machine Learning Methods with Data Collected from Various Geographies: A Study on Türkiye

Road damage seriously affects the comfort and safety of drivers. The detection of road damage is of great importance not only for transportation safety, but also in terms of cost. The detection of road damage is critical for enabling early intervention and repair. In this study, the road damage detection performance of the YOLO (You Only Look Once) v8 algorithm was evaluated using datasets obtained from different geographies, including Czechia -Türkiye, India-Türkiye, USA-Türkiye, and Japan-Türkiye. The findings revealed both the capabilities of the algorithm in damage detection and the challenges it faced in distinguishing certain types of damage. For the creation of the Türkiye dataset, images of roads in the province of Hatay were recorded. These images were labeled using Microsoft's VoTT application. Comparisons and evaluations were made among the developed models. Among these models, the Japan-Türkiye model yielded the best results with a 0.55 mAP and 0.54 F1 score. The results of the models indicated that the appearance of damage varies according to the geographical location and the quality of road data. It was observed that data consisting of local images and uncertain damage types were important in training.

Pavement crack detection based on depth-supervision FRRN model

Timely detection of early pavement distress is crucial for effective pavement maintenance. A promising approach involves the segmenting the road surface cracks area from its images. However, the previously employed full-resolution residual network (FRRN), when employing a large number of full-resolution residual units (FRRUs), is susceptible to challenges such as exploding and vanishing gradients. Addressing these issues, this paper has introduced a depth supervision mechanism into FRRN, leading to the development of the Depth Supervision FRRN (DSFRRN) model, designed specifically for accurate segmentation of cracks in road images. This model enhances nonlinear expressiveness while retaining essential features, achieving high precision in pavement crack segmentation. Through experimental validation using a dataset comprising 1565 images of damaged pavements, the result demonstrate that DSFRRN outperforms the original FRRN in terms of both F1 score and Jaccard index, accurately delineating cracks. Consequently, the automated pavement damage detection method presented herein facilitates cost-effective and efficient pavement inspection, contributing to the safeguarding of roadway surfaces and transportation infrastructure.

A framework for automatically generating composite keywords for geo-tagged street images

Due to the ubiquity of sensor-equipped cameras such as smartphones, images are associated with spatial metadata, including camera’s geographical location and viewing orientation, which can be used for automatically generating better semantic keywords about geo-tagged urban street images in addition to visual keywords extracted from image analysis. This study introduces a novel framework for auto-tagging images that integrates both spatial and visual properties to generate comprehensive and accurate tags. The framework operates through four phases: extraction, abstraction, composition, and assessment. Our research highlights the benefits of combining visual and spatial analyses, demonstrated through a case study using geo-tagged urban street images from Orlando, Pittsburgh, and Manhattan. Experimental results show that the proposed framework significantly enhances the accuracy of keyword-based searches compared to conventional methods. In particular, based on our experiments, image search using the tags generated by our proposed framework, referred to as descriptive tags, achieved an average precision improvement factor of 0.9 compared to conventional tags. Additionally, our proposed ranking algorithm, which extends the term frequency-inverse document frequency (TF-IDF) algorithm, resulted in improvement factors of 0.86 for mean average precision (MAP) and 0.57 for mean reciprocal rank (MRR). Moreover, our framework’s flexibility and robustness make it suitable for diverse applications, from smart cities to online shopping. The paper also includes a detailed evaluation and user study, confirming the precision and reliability of the generated tags.

Object and Pedestrian Detection on Road in Foggy Weather Conditions by Hyperparameterized YOLOv8 Model

Connected cooperative and automated (CAM) vehicles and self-driving cars need to achieve robust and accurate environment understanding. With this aim, they are usually equipped with sensors and adopt multiple sensing strategies, also fused among them to exploit their complementary properties. In recent years, artificial intelligence such as machine learning- and deep learning-based approaches have been applied for object and pedestrian detection and prediction reliability quantification. This paper proposes a procedure based on the YOLOv8 (You Only Look Once) method to discover objects on the roads such as cars, traffic lights, pedestrians and street signs in foggy weather conditions. In particular, YOLOv8 is a recent release of YOLO, a popular neural network model used for object detection and image classification. The obtained model is applied to a dataset including about 4000 foggy road images and the object detection accuracy is improved by changing hyperparameters such as epochs, batch size and augmentation methods. To achieve good accuracy and few errors in detecting objects in the images, the hyperparameters are optimized by four different methods, and different metrics are considered, namely accuracy factor, precision, recall, precision–recall and loss.

Tavern, Road, Moon: A Comparative Study of the Use of Imagery in the Novels of Lu Xun and Wang Zengqi

The imagery in the novels of Lu Xun and Wang Zengqi frequently overlaps. This paper selects the three common images of tavern, road, and moon for comparison, aiming to analyze the unique emotions and deeper meanings each image carries in the works of these two writers. Lu Xuns novel writing is bold in its innovation, breaking free from the constraints of traditional frameworks, with most of his imagery reflecting and bearing the significance of the times, together constructing a literary world that profoundly reflects social realities and human nature. Inspired by Lu Xun, Wang Zengqis novel writing follows a path that returns to the ordinary, pursuing the beauty of harmony. The imagery in Wang Zengqis novels embodies a love for life, fulfilling his creative ideal of bringing warmth to the human world and his persistent pursuit of expressing the beauty of human nature.

New algorithm for road boundary marking and highlighting of road tracks with object removal

In the current era, sensing the surroundings is salient development area in the field of autonomous self- driving cars. Aptness of sensing the surrounding helps to determine the travelable area on the road. Various types of road round boundary extraction techniques were available. However, these are not enough for making road boundaries in the unstructured format. In this study, a robust algorithm to detect the road boundaries for both structured and unstructured in different situations is proposed. The objects on the road are also detected in this method that can assist the driver to ensure safe driving. This method is capable of detecting the road and segment the travelable part of the road along with generation of bounding boxes around the detected road objects. The algorithm proposed in this paper is divided into three modules namely Canny module, YOLO v5 module and Hough Transform (HT) module performing three different tasks. The model is robust in boundary detection and road segmentation because the novel architecture that is used exhibits more efficient and reliable results for driver assistance with high accuracy. The NVIDIA RTX 5500 hardware has been used to build the road boundary detecting system. The proposed technique is trained and validated through different sets of images from a data set. The dataset used in making this model is in high-complex non-linear form contain the road images of various situations at different weather conditions. The experimental mean accuracy achieved in this model is 93.45% on both structured and unstructured road. The experiments show that the proposed algorithm can accurately detect the road boundary and segment the travelable road. Key Words: Road Boundary Detection, Road Segmentation, YOLO5, Hough Transform, Autonomous vehicle

Intelligent road extraction from high resolution remote sensing images based on optimized SVM

Road recognition and extraction based on high-resolution remote sensing images is currently a hot issue at the forefront of image processing and other disciplines, and its results not only help to enrich geographic information, but also have important application value in national defence construction and other aspects. For the intelligent extraction of roads in high-resolution remote sensing images, the study firstly preprocesses these images, and then extracts the roads by using variational function and support vector machine. As the extraction results are not satisfactory enough, the 3D wavelet transform technique is used to extract the spectral features of roads in the frequency domain, combined with the texture in the time domain, and optimized the support vector machine in order to extract the road images more accurately. The outcomes denote that in contrast with other methods, the optimized method can have optimal correctness and quality while guaranteeing higher integrity. The satellite road image of Chengdu A industrial park is rich in extracted information, having the highest completeness of 94.23%, correctness of 60.73%, and quality of 59.42%, and the satellite road image of Chengdu B industrial park is similarly rich in extracted information, having the highest completeness of 83.47%, correctness of 85.31%, and quality of 72.85%. The highest value of CPU usage for road extraction was 2000 kb at 37 s, and the average value of CPU usage for the whole 60 s was 500 kb. It shows that the research method has good results in high resolution remote sensing image road extraction.

Effective lane detection on complex roads with convolutional attention mechanism in autonomous vehicles

Autonomous Vehicles (AV’s) have achieved more popularity in vehicular technology in recent years. For the development of secure and safe driving, these AV’s help to reduce the uncertainties such as crashes, heavy traffic, pedestrian behaviours, random objects, lane detection, different types of roads and their surrounding environments. In AV’s, Lane Detection is one of the most important aspects which helps in lane holding guidance and lane departure warning. From Literature, it is observed that existing deep learning models perform better on well maintained roads and in favourable weather conditions. However, performance in extreme weather conditions and curvy roads need focus. The proposed work focuses on presenting an accurate lane detection approach on poor roads, particularly those with curves, broken lanes, or no lane markings and extreme weather conditions. Lane Detection with Convolutional Attention Mechanism (LD-CAM) model is proposed to achieve this outcome. The proposed method comprises an encoder, an enhanced convolution block attention module (E-CBAM), and a decoder. The encoder unit extracts the input image features, while the E-CBAM focuses on quality of feature maps in input images extracted from the encoder, and the decoder provides output without loss of any information in the original image. The work is carried out using the distinct data from three datasets called Tusimple for different weather condition images, Curve Lanes for different curve lanes images and Cracks and Potholes for damaged road images. The proposed model trained using these datasets showcased an improved performance attaining an Accuracy of 97.90%, Precision of 98.92%, F1-Score of 97.90%, IoU of 98.50% and Dice Co-efficient as 98.80% on both structured and defective roads in extreme weather conditions.

Data tampering detection and recovery scheme based on multi-branch target extraction for internet of vehicles

With the rapid development of new networks such as 5 G/6 G, the Internet of Vehicles has been given features such as hyper-connectivity and hyper-intelligence, promoting the implementation of new scenarios for autonomous vehicles. However, on the Internet of vehicles, vehicles use many cameras, radars and other sensors to sense the environment and execute instructions, facing security issues such as road data tampering and hijacking. Consequently, this paper presents a data tampering detection and recovery scheme based on multi-branch target extraction. Specifically, for road images collected by sensors, this paper presents a target extraction method based on multi-branch spatial feature pyramid blocks to obtain road salient targets. Then, a tamper detection and recovery algorithm based on interval mapping is presented. Once the image road target is tampered with, this method can quickly detect the tampering traces and restore them to achieve the authenticity and integrity of the road image on the Internet of Vehicles. The availability of the proposed scheme is verified through comparative experiments, and the performance is improved satisfactorily compared with other works.

Self-supervised learning unveils urban change from street-level images

Cities around the world are grappling with multiple interconnected challenges, including population growth, shortage of affordable and decent housing, and the need for neighborhood improvements. Despite its critical importance for policy, our ability to effectively monitor and track urban change remains limited. Deep learning-based computer vision methods applied to street-level images have been successful in the measurement of socioeconomic and environmental inequalities but did not fully utilize temporal images to track urban change, as time-varying labels are often unavailable. We used self-supervised methods to measure change in London using 15 million street images taken between 2008 and 2021. Our novel adaptation of Barlow Twins, Street2Vec, embeds urban structure while being invariant to seasonal and daily changes without manual annotations. It outperformed generic pretrained embeddings, successfully identified point-level change in London's housing supply from street-level images, and distinguished between major and minor change. This capability can provide timely information for urban planning and policy decisions towards more liveable, equitable, and sustainable cities.

Vision based nighttime pavement cracks pixel level detection by integrating infrared visible fusion and deep learning

Accurate detection of pavement cracks is an important task in road maintenance and safety management. However, accurate segmentation of road cracks at night is still challenging due to light conditions. In this study, an automatic segmentation method for nighttime road cracks based on infrared-visible fusion and deep learning is proposed. First, a fusion method of infrared and visible light images is proposed to improve the visibility of road cracks under low light conditions. Afterwards, a deep learning network integrating a dynamic sparse attention mechanism is proposed to segment the cracks in the enhanced road images. In this study, a dataset of infrared and visible light images of nighttime road cracks is acquired to test the effectiveness and sophistication of the proposed method. The results show that the proposed method can achieve accurate segmentation of nighttime pavement cracks (77.89 % mIoU、85.68 % mPA、97.74 % Accuracy、87.53 % Precision、81.55 % Recall、84.44 % F1-score) and better than the existing segmentation models (Unet, Pspnet, DeepLabv3+). Integration of the proposed method into an unmanned inspection robot helps to achieve 24/7 pixel-level pavement crack detection.

Monocular Vision Sensor-Based Indoor Road and Stair Detection

Road detection is an essential component of indoor robot navigation. Vision sensors have great advantages in road detection as they can provide rich information in terms of environmental perception. In this paper, a monocular vision sensor-based method for indoor road and stair detection is proposed, which detects feasible areas in indoor environments very fast without paying attention to detailed features of walls or other obstacles. More specifically, for a given indoor road image captured by an on-board vision sensor, the simple linear iterative clustering (SLIC) algorithm-based approach for efficient image segmentation is introduced. Then, according to the DBSCAN algorithm, the generated superpixels are clustered to form large areas of view. The initial road area is obtained through a safe window on the middle bottom of the image. In order to achieve a more accurate road segmentation, the initial image is processed by the binary search, edge detection based on the Canny operator and straight-line detection and location based on the Hough transform, which integrates edge and stair information into road detection. Several experiments are performed to evaluate the performance of the proposed method. The experimental results demonstrate that the proposed method could accurately detect road information and staircase information in images and succeeds in addressing the indoor road-detection problem.

Implementation of wavelet method and backpropagation neural network on road crack detection based on image processing

Road crack detection is critical to road infrastructure maintenance, requiring sophisticated and accurate approaches. This research explores the utilization of a combination of Wavelet and Convolutional Neural Network (CNN) methods to improve efficiency and accuracy in detecting cracks in road images. The wavelet method was chosen for its capability to capture information at different scales, enabling improved feature extraction. Meanwhile, CNN was utilized to comprehend the spatial context and tackle image complexity. The research involves several stages, including data collection, pre-processing, decomposition using the Wavelet method, forming of the CNN architecture model, training, testing, and evaluating the result. The tested images involve three main types of cracks: alligator, linear, and images without cracks. The testing results show that the developed model is capable of classifying cracks with an F1-score of 0.96, recall of 0.96, and precision of 0.96. In real-time detection of road cracks, the testing obtained an F1-score of 0.84, recall of 0.92, and precision of 0.77. This research contributes to the advancement of road crack detection technology by leveraging the capabilities of Wavelet and CNN, enhancing the accuracy and efficiency of crack detection in road maintenance.

An optimized deep belief network based pothole detection model for asphalt road

The poor quality of asphalt roads has a significant impact on driver safety, damages the mechanical structure of vehicles, increases fuel consumption, annoys passengers and is sometimes also responsible for accidents. Further, the poor quality of the road can be described as a rough surface and the presence of potholes. The potholes can be one of the main reasons for accident cause, increased fuel consumption and annoying passengers. Furthermore, the potholes can be of varied size, radiance effect, shadow and scales. Hence, the detection of potholes in asphalt roads can be considered a complex task and one of the serious issues regarding the maintenance of asphalt roads. This work focuses on the detection of the potholes in the asphalt roads. So in this work, a pothole detection model is proposed for accurate detection of potholes in the asphalt roads. The effectiveness of the proposed pothole detection model is tested over a set of real-world image datasets. In this study, the asphalt roads of the Delhi-NCR region are chosen and real-world images of these roads are collected through the smart camera. The final road image dataset consists of a total of 1150 images including 860 pothole images and the rest of are without pothole images. Further, the deep belief network is integrated into a proposed model for the detection of pothole images as a classification task and classified the images as pothole detected and not pothole. The experimental results of the proposed detection model are evaluated using accuracy, precision, recall, F1-Score and AUC parameters. These results are also compared with ANN, SVM, VGG16, VGG19 and InceptionV3 techniques. The simulation results showed that the proposed detection model achieves a 93.04% accuracy rate, 94.30% recall rate, 96.31% precision rate and 96.92% F1-Score rate than other techniques.

Analyzing Road Images for Pothole Detection through Machine Learning Algorithms: A Comprehensive Review

The road accident prediction system leverages deep learning techniques, specifically YOLO (You Only Look Once), for the detection of potholes in road footage. By employing YOLO, the system can efficiently identify and localize potholes within video frames, enabling rapid and accurate detection. Furthermore, the system incorporates a severity prediction module that utilizes the dimensions and characteristics of detected potholes to assess their severity levels. This predictive capability empowers authorities and road maintenance teams to prioritize repair efforts and allocate resources effectively, ultimately contributing to the reduction of road accidents and ensuring safer road conditions for motorists and pedestrians alike. Through the seamless integration of pothole detection and severity prediction functionalities, the road accident prediction system offers a proactive approach to road maintenance and safety management, enhancing overall road infrastructure resilience and public safety.