Road Scene Research Articles

DRMNet: more efficient bilateral networks for real-time semantic segmentation of road scenes

DRMNet: more efficient bilateral networks for real-time semantic segmentation of road scenes

YOLO-APDM: Improved YOLOv8 for Road Target Detection in Infrared Images

A new algorithm called YOLO-APDM is proposed to address low quality and multi-scale target detection issues in infrared road scenes. The method reconstructs the neck section of the algorithm using the multi-scale attentional feature fusion idea. Based on this reconstruction, the P2 detection layer is established, which optimizes network structure, enhances multi-scale feature fusion performance, and expands the detection network’s capacity for multi-scale complicated targets. Replacing YOLOv8’s C2f module with C2f-DCNv3 increases the network’s ability to focus on the target region while lowering the amount of model parameters. The MSCA mechanism is added after the backbone’s SPPF module to improve the model’s detection performance by directing the network’s detection resources to the major road target detection zone. Experimental results show that on the FLIR_ADAS_v2 dataset retaining eight main categories, using YOLO-APDM compared to YOLOv8n, mAP@0.5 and mAP@0.5:0.95 increased by 6.6% and 5.0%, respectively. On the M3FD dataset, mAP@0.5 and mAP@0.5 increased by 8.1% and 5.9%, respectively. The number of model parameters and model size were reduced by 8.6% and 4.8%, respectively. The design requirements of the high-precision detection of infrared road targets were achieved while considering the requirements of model complexity control.

An Algorithmic Study of Transformer-Based Road Scene Segmentation in Autonomous Driving

Applications such as autonomous driving require high-precision semantic image segmentation technology to identify and understand the content of each pixel in the images. Compared with traditional deep convolutional neural networks, the Transformer model is based on pure attention mechanisms, without convolutional layers or recurrent neural network layers. In this paper, we propose a new network structure called SwinLab, which is an improvement upon the Swin Transformer. Experimental results demonstrate that the improved SwinLab model achieves a segmentation accuracy comparable to that of deep convolutional neural network models in applications such as autonomous driving, with an MIoU of 77.61. Additionally, comparative experiments on the CityScapes dataset further validate the effectiveness and generalization of this structure. In conclusion, by refining the Swin Transformer, this paper simplifies the model structure, improves the training and inference speed, and maintains high accuracy, providing a more reliable semantic image segmentation solution for applications such as autonomous driving.

SAFENet: Semantic-Aware Feature Enhancement Network for unsupervised cross-domain road scene segmentation

Unsupervised cross-domain road scene segmentation has attracted substantial interest because of its capability to perform segmentation on new and unlabeled domains, thereby reducing the dependence on expensive manual annotations. This is achieved by leveraging networks trained on labeled source domains to classify images on unlabeled target domains. Conventional techniques usually use adversarial networks to align inputs from the source and the target in either of their domains. However, these approaches often fall short in effectively integrating information from both domains due to Alignment in each space usually leads to bias problems during feature learning. To overcome these limitations and enhance cross-domain interaction while mitigating overfitting to the source domain, we introduce a novel framework called Semantic-Aware Feature Enhancement Network (SAFENet) for Unsupervised Cross-domain Road Scene Segmentation. SAFENet incorporates the Semantic-Aware Enhancement (SAE) module to amplify the importance of class information in segmentation tasks and uses the semantic space as a new domain to guide the alignment of the source and target domains. Additionally, we integrate Adaptive Instance Normalization with Momentum (AdaIN-M) techniques, which convert the source domain image style to the target domain image style, thereby reducing the adverse effects of source domain overfitting on target domain segmentation performance. Moreover, SAFENet employs a Knowledge Transfer (KT) module to optimize network architecture, enhancing computational efficiency during testing while maintaining the robust inference capabilities developed during training. To further improve the segmentation performance, we further employ Curriculum Learning, a self-training mechanism that uses pseudo-labels derived from the target domain to iteratively refine the network. Comprehensive experiments on three well-known datasets, “Synthia→Cityscapes” and “GTA5→Cityscapes”, demonstrate the superior performance of our method. In-depth examinations and ablation studies verify the efficacy of each module within the proposed method.

Millimeter-Wave Radar Clutter Suppression Based on Cycle-Consistency Generative Adversarial Network

Vehicle-mounted millimeter-wave radar is widely used in autonomous driving systems for its ability to observe road scenes at all times and in all weathers. However, the data collected by millimeter-wave radar are seriously affected by the existence of clutter. This clutter will result in false detection during object detection. To address this issue, a feature extraction network with clutter suppression is necessary. This paper proposes a new clutter suppression method for millimeter-wave Range–Angle (RA) images based on a cycle-consistency generative adversarial network (CycleGAN). The generator of the method can be used as the feature extraction network of the object detection. The method aims to convert cluttered images into clutter-free images by unsupervised learning. In this method, an attention gate (AG) is introduced into the generator, a spatial attention mechanism that improves the ability of the model to automatically learn to focus on the features of targets and suppress the clutter of the background. Additionally, the target consistency loss term is added to the loss function to maintain target integrity while suppressing network training overfitting. The public dataset CRUW is utilized to evaluate the performance of the proposed method, which is compared and analyzed with traditional methods and deep learning methods. Experimental results show that the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) of the proposed method reach 39.846 and 0.990, respectively.

Road Boundary Extraction Method from Mobile Laser Scanning Point Clouds

Abstract With the rapid development of mobile laser scanning (MLS) technology, high-precision three-dimensional point cloud data has shown great potential in different fields such as topographic mapping, road asset management and smart city construc-tion. Three-dimensional point cloud data contains not only position information, but also the shape and attributes of the target, which is very convenient for obtaining road information. Accurate extraction of the road boundary is a basic task for obtaining road infrastructural data, which can support the generation of high-precision maps, vehicle navigation and auton-omous driving. However, road boundary extraction in urban environments is easily occluded such as vehicles and pedestri-ans on the road, which leads to problems such as difficulty and incomplete extraction of MLS point cloud road boundaries. To address this problem, this study proposes a road boundary extraction method that integrates pavement edge information, accurately considers the position of the road boundary from two dimensions, eliminates false boundaries, and completes the missing boundary through the extracted boundary spatial relationship. First, grid elevation filtering is used to remove high-level non-ground points. Then the pavement edges and curb stone points are extracted from the preprocessed point cloud, and they are superimposed to remove false boundary points to obtain accurate road boundaries. Finally, based on the spatial relationship of road boundaries, missing parts are detected and repaired to obtain complete road boundaries. Experimental results show that the accuracy on real road scenes exceeds 98%, the completeness rate is above 91%, and the extraction quality is above 90%, which verifies the effectiveness and accuracy of this method.

An Automatic Mapping Method of Navigation Map for Outdoor Road Scenes

Abstract. In the process of autonomous navigation of outdoor mobile robots, four modules are involved: perception, localization, planning, and controlling. The perception module utilizes sensors such as cameras and radars based on various principles to analyze the robot's surroundings in real-time. The localization module uses GPS (Global Positioning System), IMU (Inertial Measurement Unit), and prior maps for real-time positioning analysis. The planning module plans optimal path based on the outputs of the first two modules, and the controlling module directs the robot's chassis to move along the planned optimal path. For the localization module, the accuracy of GPS positioning results heavily depends on weather conditions and GPS signal receptions. Even if the positioning results of imu are integrated, the positioning accuracy still cannot meet the needs of robot navigation. Therefore, using prior maps for repositioning can compensate for this accuracy deficiency. The planning module also requires path planning based on prior maps. If the a priori map storage is large, it will lead to difficulties in usage, maintenance, and updates. Therefore, it is crucial to research lightweight navigation map mapping methods. In this paper, an automatically mapping method of lightweight navigation maps is proposed, combining cameras and LiDAR (Light Detection and Ranging), including semantic informations necessary for outdoor navigation positioning, such as pole-like objects and traffic signs for robot longitudinal positioning, and lane line elements for robot lateral positioning. This method automatic generates robot navigation maps in Lanelet2 format, providing support for subsequent positioning and path planning modules.

AMW-YOLOv8n: Road Scene Object Detection Based on an Improved YOLOv8

This study introduces an improved YOLOv8 model tailored for detecting objects in road scenes. To overcome the limitations of standard convolution operations in adapting to varying targets, we introduce Adaptive Kernel Convolution (AKconv). AKconv dynamically adjusts the convolution kernel’s shape and size, enhancing the backbone network’s feature extraction capabilities and improving feature representation across different scales. Additionally, we employ a Multi-Scale Dilated Attention (MSDA) mechanism to focus on key target features, further enhancing feature representation. To address the challenge posed by YOLOv8’s large down sampling factor, which limits the learning of small target features in deeper feature maps, we add a small target detection layer. Finally, to improve model training efficiency, we introduce a regression loss function with a Wise-IoU dynamic non-monotonic focusing mechanism. With these enhancements, our improved YOLOv8 model excels in road scene object detection tasks, achieving a 5.6 percentage point improvement in average precision over the original YOLOv8n on real road datasets.

Urban Color Perception and Sentiment Analysis Based on Deep Learning and Street View Big Data

The acceleration of urbanization has resulted in a heightened awareness of the impacts of urban environments on residents’ emotional states. This present study focuses on the Lixia District of Jinan City. By using urban street view big data and deep learning methods, we undertook a detailed analysis of the impacts of urban color features on residents’ emotional perceptions. In particular, a substantial corpus of street scene image data was extracted and processed. This was performed using a deep convolutional neural network (DCNN) and semantic segmentation technology (PSPNet), which enabled the simulation and prediction of the subjective perception of the urban environment by humans. Furthermore, the color complexity and coordination in the street scene were quantified and combined with residents’ emotional feedback to carry out a multi-dimensional analysis. The findings revealed that color complexity and coordination were significant elements influencing residents’ emotional perceptions. A high color complexity is visually appealing, but can lead to fatigue, discomfort, and boredom; a moderate complexity stimulates vitality and pleasure; high levels of regional harmony and aesthetics can increase perceptions of beauty and security; and low levels of coordination can increase feelings of depression. The environmental characteristics of different areas and differences in the daily activities of residents resulted in regional differences regarding the impacts of color features on emotional perception. This study corroborates the assertion that environmental color coordination has the capacity to enhance residents’ emotions, thereby providing an important reference point for urban planning. Planning should be based on the functional characteristics of the region, and color complexity and coordination should be reasonably regulated to optimize the emotional experiences of residents. Differentiated color management enhances urban aesthetics, livability, and residents’ happiness and promotes sustainable development. In the future, the influences of color and environmental factors on emotions can be explored in depth, with a view to assist in the formulation of fine urban design.

Domain adaptation for semantic segmentation of road scenes via two-stage alignment of traffic elements

Unsupervised domain adaptation has been used to reduce the domain shift, which would improve the performance of semantic segmentation on unlabeled real-world data. However, existing methodologies fall short in effectively addressing the domain shift issue prevalent in traffic scenarios, leading to less than satisfactory segmentation results. In this paper, we propose a novel domain adaptation method for semantic segmentation via unsupervised alignment of traffic elements. Firstly, we introduce a two-stage self-training framework that leverages a blended set of training samples to enhance the training process. In the first stage, we leverage generated mixup training samples as inputs within our two-stage self-training framework and have developed corresponding loss functions for both the source and target domains to direct the training process. Then, the alignment modules for dynamic and static traffic elements are designed to achieve accurate matching between the source and the target domain images. The cosine similarity maximization is applied to the alignment of dynamic traffic elements, while the prototype learning is utilized for the static traffic elements. Additionally, we present a new technique for reducing noise in pseudo labels by constructing thresholds that adjust to each class. Meanwhile, we formulate the associated target domain loss function for vacant pseudo label pixels. The experimental results demonstrate that the proposed method is superior to the existing methods on five different domain adaptation tasks, which is more applicable to semantic segmentation of road scenes.

Exploring the coherence and divergence between the objective and subjective measurement of streetscape perceptions at the neighborhood level: A case study in Shanghai

Understanding micro-level perceptions of street scenes is highly concerned with residents’ behaviors and socioeconomic outcomes. While many studies rely on objective measures, such as physical features extracted from Street View Imagery (SVI) to proxy perceptions using derived formulas, others employ subjective measures from visual surveys to capture more subtle human perceptions. We argue that the two measurements can diverge significantly over the same perception concept, which might lead to opposite spatial implications in policy if not properly understood. Moreover, as perceptions are often examined individually, few studies have investigated their joint distribution patterns to reflect perceptions’ multi-dimensional nature. To fill the gaps, we collected five pairwise perceptions from SVIs (i.e., complexity, enclosure, greenness, imageability, and walkability) at the neighborhood level in Shanghai. Each perception consists of pairwise values subjectively measured using a GeoAI-based approach and objectively quantified using formulas. We statistically and spatially compared the coherence and divergence of the two measures, further examining the perceptual differences. Advanced techniques including cluster analysis and factor analysis were employed to jointly evaluate their spatial distribution discrepancy. Our results revealed more differences than similarities between the two measures statistically and spatially, confirming any spatial implications concluded from one approach can vary significantly from the other. The joint spatial pattern further corroborated our conclusions. Our study enriches the literature on micro-level street perception measures, uncovers their critical differences to guide future comparative studies, and offers new approaches for urban perception mapping.

LFIR-YOLO: Lightweight Model for Infrared Vehicle and Pedestrian Detection

The complexity of urban road scenes at night and the inadequacy of visible light imaging in such conditions pose significant challenges. To address the issues of insufficient color information, texture detail, and low spatial resolution in infrared imagery, we propose an enhanced infrared detection model called LFIR-YOLO, which is built upon the YOLOv8 architecture. The primary goal is to improve the accuracy of infrared target detection in nighttime traffic scenarios while meeting practical deployment requirements. First, to address challenges such as limited contrast and occlusion noise in infrared images, the C2f module in the high-level backbone network is augmented with a Dilation-wise Residual (DWR) module, incorporating multi-scale infrared contextual information to enhance feature extraction capabilities. Secondly, at the neck of the network, a Content-guided Attention (CGA) mechanism is applied to fuse features and re-modulate both initial and advanced features, catering to the low signal-to-noise ratio and sparse detail features characteristic of infrared images. Third, a shared convolution strategy is employed in the detection head, replacing the decoupled head strategy and utilizing shared Detail Enhancement Convolution (DEConv) and Group Norm (GN) operations to achieve lightweight yet precise improvements. Finally, loss functions, PIoU v2 and Adaptive Threshold Focal Loss (ATFL), are integrated into the model to better decouple infrared targets from the background and to enhance convergence speed. The experimental results on the FLIR and multispectral datasets show that the proposed LFIR-YOLO model achieves an improvement in detection accuracy of 4.3% and 2.6%, respectively, compared to the YOLOv8 model. Furthermore, the model demonstrates a reduction in parameters and computational complexity by 15.5% and 34%, respectively, enhancing its suitability for real-time deployment on resource-constrained edge devices.

Cultural Streetscape: Its Value-System, Space and Significance in Oyo Town, Nigeria

Cultural streetscape is a remarkable street scene that embodies the local culture. It helps in defining and defending urban forms, improves the physical, economic, and socio-cultural mechanisms of urban streets, provides street views for buildings, and shares communal activities and identities. Streetscape is a significant and multi-dynamic component of urban design that contributes to the success of cities and towns. However, as a result of constant urbanization and neglect, African streets that were previously sites of different cultural expressions and local beliefs are quickly turning into areas devoid of local values and meaning. More so, the significance of a cultural streetscape that facilitates ingenious street views, solves ecological challenges, and retains the local memory of places is becoming scarce in the literature. Through an in-depth interview, analysis of pertinent academic outputs, and firsthand observation of the core residential areas of Oyo town, this study revealed the value and significance of cultural streetscape in building, shaping, preserving and sustaining local place identity. Proper archiving and infusion of cultural streetscape elements into the old and new streets design are necessary to ensure Yoruba place identity does not completely become void.

Road-crossing behavior and safety of pedestrians facing autonomous vehicles with an acceleration indicator eHMI in VR traffic flow

To support pedestrian road-crossing behavior in the presence of autonomous vehicles (AVs), we propose a novel external Human-Machine Interface (eHMI) that enhances the vehicle’s inherent motion cues, which pedestrians already use in traditional traffic. Based on the Beta Movement, the eHMI design consists of light bands with multiple arrows moving forward or backward to indicate the vehicle’s accelerating or decelerating motion state. We measured pedestrians’ behavior and perceived safety when faced with AVs equipped with this acceleration indicator eHMI in a virtual reality (VR) street scene, where pedestrians had road-crossing tasks and vehicles did not always yield. We found that pedestrians exhibited crossing behavior according to the motion state of the oncoming vehicle. They almost always crossed if the vehicle yielded but considered the gap size to cross when it did not. With the acceleration indicator eHMI, they crossed more frequently if the vehicle yielded, yet less frequently if it did not. Although risky crossings could not be fully prevented, pedestrians crossed faster and maintained larger safety margins when the non-yielding vehicle used this eHMI. Additionally, pedestrians started to cross earlier when the eHMI indicated the vehicle’s decelerating state. Regarding pedestrians’ perceived safety, the eHMI did not increase perceived safety for decelerating vehicles but effectively decreased the perceived safety for accelerating vehicles. These results demonstrate the potential of enhancing vehicles’ inherent motion cues with eHMIs, but also imply that making vehicles’ motion state more apparent might not simply increase pedestrians’ perceived safety. Overall, a trade-off seems to exist that it is pedestrians’ sense of unsafety that leads to their safe road-crossing behavior. Suggestions for future research and VR experiments were also discussed.

Data science basis and influencing factors for the evaluation of environmental safety perception in Macau parishes

In the context of rapid urbanization, accurately identifying the visual factors that influence environmental safety perception is crucial for improving urban transportation environments and enhancing pedestrian safety. With the increase in urban population density and traffic flow, optimizing urban environmental design to elevate residents’ sense of safety has become a key issue in urban planning and management. However, the existing studies face numerous challenges in conducting large-scale quantitative analysis of environmental safety perception in complex scenarios, such as difficulties in data acquisition and limitations in analytical methods. This study addresses these challenges by applying image semantic segmentation and object detection techniques to extract key visual elements from street view images, combined with manual scoring and deep learning methods, to construct a road safety perception dataset. Using a LightGBM model and the SHAP interpretation framework, in this study, we identify the critical visual factors influencing environmental safety perception. An empirical study was conducted in Macau, a modern city where Eastern and Western cultures intersect, and tourism thrives. The findings reveal that: ① The overall environmental safety perception in the eight parishes and surrounding roads of Macau is relatively high, with significant regional differences in safety perception scores around Macau’s parish roads; ② The proportions of buildings, sidewalks, roads, and trees in images are the four primary factors influencing environmental safety perception; ③ The proportions and quantities of visual elements interact with each other, and their reasonable distribution helps form clear spatial visibility and creates conducive activity spaces, thereby enhancing the perception of environmental safety. Through empirical analysis, this study uncovers the mechanisms by which visual elements in urban street scenes affect environmental safety perception, providing scientific evidence for urban planning and transportation environment improvement. The research holds theoretical significance and offers practical references for urban design and management, demonstrating broad application value.

MSF2Net: multi-stage feature fusion network for real-time semantic segmentation in road scenes

MSF2Net: multi-stage feature fusion network for real-time semantic segmentation in road scenes

Improved Road Target Detection Algorithm for YOLOv7-Tiny

In complex road scenes, we propose an enhanced road target detection algorithm for YOLOv7-Tiny to address issues such as large model size, misclassification, and low localization accuracy. Our approach involves several key modifi-cations. Firstly, we replace the LeakyReLU activation function in YOLOv7-Tiny with H-swish. This replacement not only reduces the number of parameters in the model but also enhances its feature extraction capabilities. Additionally, we replace the ELAN module in the Neck with the DPCH-ELAN module and introduce the darknetblock module along with the pcconv convolution. These modifications improve the network's ability to comprehend complex patterns and semantics, thereby enabling it to capture features at different levels of input data. Moreover, we introduce the pcconv convolution building block at the output side to handle heterogeneous information in complex road scenes, thereby enhancing the network's performance in detecting road targets and abnormalities. In our experiments using a mul-ti-source dataset, the improved model exhibits a reduction in GFLOPs by 20.90% and a decrease in the number of pa-rameters by 24.49%. Furthermore, the mean average precision scores (map) at thresholds of 0.5 and 0.5~0.9 are im-proved to 77.3% and 51.8%, respectively, compared to the original YOLOv7-Tiny model. These experimental results demonstrate that our enhanced model achieves a reduction in model size while simultaneously enhancing detection accuracy, thereby meeting the requirements for real-time detection. To assess the generalizability of our approach, we conducted comparison experiments on the VOC2012 dataset. The results indicate that the improved algorithm exhibits robust generalization capabilities across different datasets.

YOLO-TSF: A Small Traffic Sign Detection Algorithm for Foggy Road Scenes

YOLO-TSF: A Small Traffic Sign Detection Algorithm for Foggy Road Scenes

Real-Time Semantic Segmentation Algorithm for Street Scenes Based on Attention Mechanism and Feature Fusion

In computer vision, the task of semantic segmentation is crucial for applications such as autonomous driving and intelligent surveillance. However, achieving a balance between real-time performance and segmentation accuracy remains a significant challenge. Although Fast-SCNN is favored for its efficiency and low computational complexity, it still faces difficulties when handling complex street scene images. To address this issue, this paper presents an improved Fast-SCNN, aiming to enhance the accuracy and efficiency of semantic segmentation by incorporating a novel attention mechanism and an enhanced feature extraction module. Firstly, the integrated SimAM (Simple, Parameter-Free Attention Module) increases the network’s sensitivity to critical regions of the image and effectively adjusts the feature space weights across channels. Additionally, the refined pyramid pooling module in the global feature extraction module captures a broader range of contextual information through refined pooling levels. During the feature fusion stage, the introduction of an enhanced DAB (Depthwise Asymmetric Bottleneck) block and SE (Squeeze-and-Excitation) attention optimizes the network’s ability to process multi-scale information. Furthermore, the classifier module is extended by incorporating deeper convolutions and more complex convolutional structures, leading to a further improvement in model performance. These enhancements significantly improve the model’s ability to capture details and overall segmentation performance. Experimental results demonstrate that the proposed method excels in processing complex street scene images, achieving a mean Intersection over Union (mIoU) of 71.7% and 69.4% on the Cityscapes and CamVid datasets, respectively, while maintaining inference speeds of 81.4 fps and 113.6 fps. These results indicate that the proposed model effectively improves segmentation quality in complex street scenes while ensuring real-time processing capabilities.

Did you look at the moose? Driver gaze behaviour while searching for hazards in dynamic road scenes

Did you look at the moose? Driver gaze behaviour while searching for hazards in dynamic road scenes