Complex Road Scenes Research Articles

Improved Road Target Detection Algorithm for YOLOv7-Tiny

In complex road scenes, we propose an enhanced road target detection algorithm for YOLOv7-Tiny to address issues such as large model size, misclassification, and low localization accuracy. Our approach involves several key modifi-cations. Firstly, we replace the LeakyReLU activation function in YOLOv7-Tiny with H-swish. This replacement not only reduces the number of parameters in the model but also enhances its feature extraction capabilities. Additionally, we replace the ELAN module in the Neck with the DPCH-ELAN module and introduce the darknetblock module along with the pcconv convolution. These modifications improve the network's ability to comprehend complex patterns and semantics, thereby enabling it to capture features at different levels of input data. Moreover, we introduce the pcconv convolution building block at the output side to handle heterogeneous information in complex road scenes, thereby enhancing the network's performance in detecting road targets and abnormalities. In our experiments using a mul-ti-source dataset, the improved model exhibits a reduction in GFLOPs by 20.90% and a decrease in the number of pa-rameters by 24.49%. Furthermore, the mean average precision scores (map) at thresholds of 0.5 and 0.5~0.9 are im-proved to 77.3% and 51.8%, respectively, compared to the original YOLOv7-Tiny model. These experimental results demonstrate that our enhanced model achieves a reduction in model size while simultaneously enhancing detection accuracy, thereby meeting the requirements for real-time detection. To assess the generalizability of our approach, we conducted comparison experiments on the VOC2012 dataset. The results indicate that the improved algorithm exhibits robust generalization capabilities across different datasets.

Real-Time Semantic Segmentation Algorithm for Street Scenes Based on Attention Mechanism and Feature Fusion

In computer vision, the task of semantic segmentation is crucial for applications such as autonomous driving and intelligent surveillance. However, achieving a balance between real-time performance and segmentation accuracy remains a significant challenge. Although Fast-SCNN is favored for its efficiency and low computational complexity, it still faces difficulties when handling complex street scene images. To address this issue, this paper presents an improved Fast-SCNN, aiming to enhance the accuracy and efficiency of semantic segmentation by incorporating a novel attention mechanism and an enhanced feature extraction module. Firstly, the integrated SimAM (Simple, Parameter-Free Attention Module) increases the network’s sensitivity to critical regions of the image and effectively adjusts the feature space weights across channels. Additionally, the refined pyramid pooling module in the global feature extraction module captures a broader range of contextual information through refined pooling levels. During the feature fusion stage, the introduction of an enhanced DAB (Depthwise Asymmetric Bottleneck) block and SE (Squeeze-and-Excitation) attention optimizes the network’s ability to process multi-scale information. Furthermore, the classifier module is extended by incorporating deeper convolutions and more complex convolutional structures, leading to a further improvement in model performance. These enhancements significantly improve the model’s ability to capture details and overall segmentation performance. Experimental results demonstrate that the proposed method excels in processing complex street scene images, achieving a mean Intersection over Union (mIoU) of 71.7% and 69.4% on the Cityscapes and CamVid datasets, respectively, while maintaining inference speeds of 81.4 fps and 113.6 fps. These results indicate that the proposed model effectively improves segmentation quality in complex street scenes while ensuring real-time processing capabilities.

TTIS-YOLO: a traffic target instance segmentation paradigm for complex road scenarios

Abstract The instance segmentation of traffic targets in complex road scenes is one of the most challenging tasks in autonomous driving. Unlike the bounding box localization for object detection and the category perception mask for semantic segmentation, instance segmentation requires accurate identification of each object under each category and more precise segmentation and positioning of these target objects. Although instance segmentation has apparent advantages, methods, for instance segmentation in complex road scenes, still need to be discovered. In this paper, we proposed an efficient instance segmentation method TTIS-YOLO(Traffic Target Instance Segmentation - YOLO) based on YOLOV5-7.0 for traffic object segmentation of complex road scenes. Our main work is as follows: to propose a MECSP(Multiscale Efficient Cross Stage Partial Network) module, which has fewer parameters, better cross-layer information exchange, and feature representation capabilities. Propose an Efficient Bidirectional Cross Scale Connection optimization method that enables the network to perform more detailed and efficient feature fusion without losing original information, refining the mask flow. WIoU Loss is used as the loss function of positioning and segmentation, and the positioning performance of the model is effectively improved through the strategy of dynamically allocating gradient gains. Experiments have shown that our proposed TTIS-YOLO outperforms baseline models and other mainstream instances segmentation algorithms such as Mask RCNN, YOLACT, SOLO, and SOLOV2 with the highest segmentation accuracy and fastest inference speed. Our proposed TTIS-YOLO-S achieves the best balance between segmentation accuracy and inference speed. Compared to the baseline model, the AP50 and recall values on the Cityscapes validation set increased by 1.7% and 0.9%, respectively, with a parameter reduction of 20.6%, inference speed of 78.1fps on GeForce RTX 3090Ti. Meanwhile, TTIS-YOLO-L achieved the highest segmentation accuracy, with an AP50 value of 27%, and the model parameter quantity decreased by 35.4% compared to the baseline model.

A novel real-time object detection method for complex road scenes based on YOLOv7-tiny

A novel real-time object detection method for complex road scenes based on YOLOv7-tiny

MRD-YOLO: A Multispectral Object Detection Algorithm for Complex Road Scenes

Object detection is one of the core technologies for autonomous driving. Current road object detection mainly relies on visible light, which is prone to missed detections and false alarms in rainy, night-time, and foggy scenes. Multispectral object detection based on the fusion of RGB and infrared images can effectively address the challenges of complex and changing road scenes, improving the detection performance of current algorithms in complex scenarios. However, previous multispectral detection algorithms suffer from issues such as poor fusion of dual-mode information, poor detection performance for multi-scale objects, and inadequate utilization of semantic information. To address these challenges and enhance the detection performance in complex road scenes, this paper proposes a novel multispectral object detection algorithm called MRD-YOLO. In MRD-YOLO, we utilize interaction-based feature extraction to effectively fuse information and introduce the BIC-Fusion module with attention guidance to fuse different modal information. We also incorporate the SAConv module to improve the model’s detection performance for multi-scale objects and utilize the AIFI structure to enhance the utilization of semantic information. Finally, we conduct experiments on two major public datasets, FLIR_Aligned and M3FD. The experimental results demonstrate that compared to other algorithms, the proposed algorithm achieves superior detection performance in complex road scenes.

PODB: A learning-based polarimetric object detection benchmark for road scenes in adverse weather conditions

Due to its insensitivity to light intensity and the capability to capture multidimensional information, polarimetric imaging technology has been proven to have advantages over traditional intensity-based imaging techniques for object detection tasks in adverse environmental conditions, particularly in road traffic scenarios. Recently, with the rapid development of artificial intelligence technology, deep learning (DL)-powered object detection techniques can further enhance recognition accuracy and algorithm robustness. This improvement is made possible by the ability of DL technology to leverage large datasets and extract deeper levels of target-specific features. However, constructing large-scale polarimetric datasets poses challenges as obtaining polarimetric information requires multiple captures of intensity images. In other words, the workload is several times higher compared to traditional imaging techniques. To address the current scarcity of polarimetric datasets and evaluate the practical performance of various algorithms on polarimetric datasets, this paper proposes a Polarimetric Object Detection Benchmark (PODB) dataset. The PODB provides an integrated quality evaluation framework for DL-based object detection algorithms in complex road scenes by incorporating polarimetric imaging. Besides, we conducted extensive object detection experiments using the PODB, which allowed for a comprehensive validation and performance evaluation of effective benchmark algorithms. Furthermore, a physics-based multi-scale image fusion cascaded object detection neural network model is proposed. By combining the multidimensional information provided by polarized images with an adaptive learning multi-decision object detection neural network model, the recognition accuracy of complex road scenes in adverse weather conditions has been improved by approximately 10%. Additionally, we anticipate that PODB will serve as an effective platform for evaluating and comparing the performance of object detection algorithms, as well as providing researchers with a baseline for future studies in developing new DL-based methods.

SNCE-YOLO: An Improved Target Detection Algorithm in Complex Road Scenes

SNCE-YOLO: An Improved Target Detection Algorithm in Complex Road Scenes

Improvement of Road Instance Segmentation Algorithm Based on the Modified Mask R-CNN

Although the Mask region-based convolutional neural network (R-CNN) model possessed a dominant position for complex and variable road scene segmentation, some problems still existed, including insufficient feature expressive ability and low segmentation accuracy. To address these problems, a novel road scene segmentation algorithm based on the modified Mask R-CNN was proposed. The multi-scale backbone network, Res2Net, was utilized to replace the ResNet network, and aimed to improve the feature extraction capability. The soft non-maximum suppression algorithm with attenuation function (soft-NMS) was adopted to improve detection efficiency in the case of a higher overlap rate. The comparison analyses of partition accuracy for various models were performed on the adopted Cityscapes dataset. The results demonstrated that the modified Mask R-CNN effectively increased the segmentation accuracy, especially for small and highly overlapping objects. The adopted Res2Net and soft-NMS can effectively enhance the feature extraction and improve segmentation performance. The average accuracy of the modified Mask R-CNN model reached up to 0.321, and was 0.054 higher than Mask R-CNN. This work provides important guidance to design a more efficient road scene instance segmentation algorithm for further promoting the actual application in automatic driving systems.

PatchAugNet: Patch feature augmentation-based heterogeneous point cloud place recognition in large-scale street scenes

Point Cloud Place Recognition (PCPR) in street scenes is an essential task in the fields of autonomous driving, robot navigation, and urban map updating. However, the domain gap between heterogeneous point clouds and the difficulty of feature characterization in large-scale complex street scenes pose significant challenges for existing PCPR methods. Most PCPR methods only take into account point clouds collected by the same platforms and sensors, thus they are with poor domain transferability. In this paper, we propose PatchAugNet, which utilizes patch feature augmentation and adaptive pyramid feature aggregation to achieve better performance and generalizability for Heterogeneous Point Cloud-based Place Recognition (HPCPR) tasks. Firstly, multi-scale local features are extracted by the pyramid feature extraction module. Secondly, local features are enhanced by the patch feature augmentation module to overcome the domain gap problem and achieve better feature representation as well as network generalizability. Finally, a global feature is generated using an adaptive pyramid feature aggregation module, which automatically adjusts and balances the proportion of intra-scale and inter-scale features according to the scene content. To evaluate the performance of PatchAugNet, a large-scale heterogeneous point cloud dataset consisting of high-precision Mobile Laser Scanning (MLS) point clouds and helmet-mounted Portable Laser Scanning (PLS) point clouds is collected. The dataset covers various street scenes with a length of over 20 km. The comprehensive experimental results indicate that PatchAugNet achieves State-Of-The-Art (SOTA) performance with 83.43 % recall@top1% and 60.34 % recall@top1 on unseen large-scale street scenes, outperforming existing SOTA PCPR methods by + 9.57 recall@top1% and + 15.50 recall@top1, while exhibiting better generalizability. For source code and detailed experimental results, please refer to: https://github.com/WHU-USI3DV/PatchAugNet.

CDAU-Net: A Novel CoordConv-Integrated Deep Dual Cross Attention Mechanism for Enhanced Road Extraction in Remote Sensing Imagery

In the realm of remote sensing image analysis, the task of road extraction poses significant complexities, especially in the context of intricate scenes and diminutive targets. In response to these challenges, we have developed a novel deep learning network, christened CDAU-Net, designed to discern and delineate these features with enhanced precision. This network takes its structural inspiration from the fundamental architecture of U-Net while introducing innovative enhancements: we have integrated CoordConv convolutions into both the initial layer of the U-Net encoder and the terminal layer of the decoder, thereby facilitating a more efficacious processing of spatial information inherent in remote sensing images. Moreover, we have devised a unique mechanism termed the Deep Dual Cross Attention (DDCA), purposed to capture long-range dependencies within images—a critical factor in remote sensing image analysis. Our network replaces the skip-connection component of the U-Net with this newly designed mechanism, dealing with feature maps of the first four scales in the encoder and generating four corresponding outputs. These outputs are subsequently linked with the decoder stage to further capture the remote dependencies present within the remote sensing imagery. We have subjected CDAU-Net to extensive empirical validation, including testing on the Massachusetts Road Dataset and DeepGlobe Road Dataset. Both datasets encompass a diverse range of complex road scenes, making them ideal for evaluating the performance of road extraction algorithms. The experimental results showcase that whether in terms of accuracy, recall rate, or Intersection over Union (IoU) metrics, the CDAU-Net outperforms existing state-of-the-art methods in the task of road extraction. These findings substantiate the effectiveness and superiority of our approach in handling complex scenes and small targets, as well as in capturing long-range dependencies in remote sensing imagery. In sum, the design of CDAU-Net not only enhances the accuracy of road extraction but also presents new perspectives and possibilities for deep learning analysis of remote sensing imagery.

Enhanced YOLOv5: An Efficient Road Object Detection Method.

Accurate identification of road objects is crucial for achieving intelligent traffic systems. However, developing efficient and accurate road object detection methods in complex traffic scenarios has always been a challenging task. The objective of this study was to improve the target detection algorithm for road object detection by enhancing the algorithm's capability to fuse features of different scales and levels, thereby improving the accurate identification of objects in complex road scenes. We propose an improved method called the Enhanced YOLOv5 algorithm for road object detection. By introducing the Bidirectional Feature Pyramid Network (BiFPN) into the YOLOv5 algorithm, we address the challenges of multi-scale and multi-level feature fusion and enhance the detection capability for objects of different sizes. Additionally, we integrate the Convolutional Block Attention Module (CBAM) into the existing YOLOv5 model to enhance its feature representation capability. Furthermore, we employ a new non-maximum suppression technique called Distance Intersection Over Union (DIOU) to effectively address issues such as misjudgment and duplicate detection when significant overlap occurs between bounding boxes. We use mean Average Precision (mAP) and Precision (P) as evaluation metrics. Finally, experimental results on the BDD100K dataset demonstrate that the improved YOLOv5 algorithm achieves a 1.6% increase in object detection mAP, while the P value increases by 5.3%, effectively improving the accuracy and robustness of road object recognition.

ASA-BiSeNet: improved real-time approach for road lane semantic segmentation of low-light autonomous driving road scenes.

The solution to the problem of road environmental perception is one of the essential prerequisites to realizing the autonomous driving of intelligent vehicles, and road lane detection plays a crucial role in road environmental perception. However, road lane detection in complex road scenes is challenging due to poor illumination conditions, the occlusion of other objects, and the influence of unrelated road markings. It also hinders the commercial application of autonomous driving technology in various road scenes. In order to minimize the impact of illumination factors on road lane detection tasks, researchers use deep learning (DL) technology to enhance low-light images. In this study, road lane detection is regarded as an image segmentation problem, and road lane detection is studied based on the DL approach to meet the challenge of rapid environmental changes during driving. First, the Zero-DCE++ approach is used to enhance the video frame of the road scene under low-light conditions. Then, based on the bilateral segmentation network (BiSeNet) approach, the approach of associate self-attention with BiSeNet (ASA-BiSeNet) integrating two attention mechanisms is designed to improve the road lane detection ability. Finally, the ASA-BiSeNet approach is trained based on the self-made road lane dataset for the road lane detection task. At the same time, the approach based on the BiSeNet approach is compared with the ASA-BiSeNet approach. The experimental results show that the frames per second (FPS) of the ASA-BiSeNet approach is about 152.5 FPS, and its mean intersection over union is 71.39%, which can meet the requirements of real-time autonomous driving.

Weakly supervised multi-class semantic video segmentation for road scenes

Weakly supervised multi-class video segmentation is one of the most challenging yet least studied research problems in computer vision. This study aims to investigate two main items: (1) effective feature update for temporal changes combined with feature reuse between temporal frames; and (2) learn object patterns in complex scenes specifically for videos under weak supervision. Associating image tags to visual appearance is not a straightforward learning task, especially for complex scenes. Therefore, in this paper, we present manifold augmentations to obtain reliable pixel labels from image tags. We propose a framework comprised of two key modules: a temporal split module for efficient video processing and a pseudo per-pixel seed generation module for precise pixel-level supervision. Particularly, in our model, we utilize and explore temporal correlations via temporal split module and temporal attention. To reuse the extracted features and incorporate temporal updates for precise and fast computation, a channel-wise temporal split mechanism between successive video frames is presented. Furthermore, we evaluated proposed modules in two additional settings: (1) fully or sparsely supervised road scene video segmentation; and (2) weakly supervised segmentation for complex road scene images. Experiments are conducted on the Cityscapes and CamVid datasets, using DeepLabv3 as segmentation network and LiteFlowNet to compute motion vectors.

A Multi-Scale Traffic Object Detection Algorithm for Road Scenes Based on Improved YOLOv5

Object detection in road scenes is a task that has recently become popular and it is also an important part of intelligent transportation systems. Due to the different locations of cameras in the road scenes, the size of the traffic objects captured varies greatly, which imposes a burden on the network optimization. In addition, in some dense traffic scenes, the size of the traffic objects captured is extremely small and it is easy to miss detection and to encounter false detection. In this paper, we propose an improved multi-scale YOLOv5s algorithm based on the YOLOv5s algorithm. In detail, we add a detection head for extremely small objects to the original YOLOv5s model, which significantly improves the accuracy in detecting extremely small traffic objects. A content-aware reassembly of features (CARAFE) module is introduced in the feature fusion part to enhance the feature fusion. A new SPD-Conv CNN Module is introduced instead of the original convolutional structure to enhance the overall computational efficiency of the model. Finally, the normalization-based attention module (NAM) is introduced, allowing the model to focus on more useful information during training and significantly improving detection accuracy. The experimental results demonstrate that compared with the original YOLOv5s algorithm, the detection accuracy of the multi-scale YOLOv5s model proposed in this paper is improved by 7.1% on the constructed diverse traffic scene datasets. The improved multi-scale YOLOv5s algorithm also maintains the highest detection accuracy among the current mainstream object detection algorithms and is superior in accomplishing the task of detecting traffic objects in complex road scenes.

Research on Infrared and Visible Image Registration Algorithm for Complex Road Scenes

Research on Infrared and Visible Image Registration Algorithm for Complex Road Scenes

Hierarchical Fine Extraction Method of Street Tree Information from Mobile LiDAR Point Cloud Data

The classification and extraction of street tree geometry information in road scenes is crucial in urban forest biomass statistics and road safety. To address the problem of 3D fine extraction of street trees in complex road scenes, this paper designs and investigates a method for extracting street tree geometry and forest parameters from vehicle-mounted LiDAR point clouds in road scenes based on a Gaussian distributed regional growth algorithm and Voronoi range constraints. Firstly, a large number of non-tree and other noise points, such as ground points, buildings, shrubs and vehicle points, are filtered by applying multi-geometric features; then, the main trunk of the street tree is extracted based on the vertical linear features of the tree and the region growth algorithm based on Gaussian distribution; secondly, a Voronoi polygon constraint is established to segment the single tree canopy region with the main trunk center of mass; finally, based on the extracted locations of the street trees and their 3D points, the tree growth parameters of individual trees are obtained for informative management and biomass estimation by combining geometric statistical methods. In this paper, the experimental data from vehicle-borne LiDAR point clouds of different typical areas were selected to verify that the proposed Gaussian-distributed regional growth algorithm can achieve fine classification and extraction of tree growth parameters for different types of roadside trees, with accuracy, recall and F1 values reaching 96.34%, 97.22% and 96.45%, respectively. This research method can be used for the extraction of 3D fine classification of street trees in complex road environments, which in turn can provide support for the safety management of traffic facilities and forest biomass estimation in urban environments.

Curb Detection and Compensation Method for Autonomous Driving via a 3-D-LiDAR Sensor

Effective detection of curbs is crucial for the perception and localization of autonomous vehicles, but, in practical applications, complex road scenes and the occlusion of curbs by various obstacles are two difficult problems to deal with. Therefore, this article proposes a fast curb detection and compensation method. First, the multiblock RANSAC method is used for ground segmentation to distinguish between ground points and nonground points. Second, the candidate curb points are obtained by processing the ground points with the jumping-window algorithm associated with multispatial features. The jumping-window algorithm only processes the region of interest, which greatly improves the operation efficiency of the method. Third, a joint filtering method is used to refine the candidate curb points, mainly to remove obstacles and other interfering points contained in the candidate curb points. Finally, the missing part of the curb caused by the obstruction is interpolated by a piecewise cubic spline to compensate for the curb points and improve the continuity. We use the KITTI dataset to evaluate the proposed method, and the obtained precision, recall, and harmonic mean in various road scenarios are all above 80%, which is sufficient to demonstrate the robustness of our method. At the same time, the average processing time of each frame is only 47 ms, which can meet the real-time requirements of automatic driving.

Anchor-Free Object Detection with Scale-Aware Networks for Autonomous Driving

Current anchor-free object detectors do not rely on anchors and obtain comparable accuracy with anchor-based detectors. However, anchor-free object detectors that adopt a single-level feature map and lack a feature pyramid network (FPN) prior information about an object’s scale; thus, they insufficiently adapt to large object scale variation, especially for autonomous driving in complex road scenes. To address this problem, we propose a divide-and-conquer solution and attempt to introduce some prior information about object scale variation into the model when maintaining a streamlined network structure. Specifically, for small-scale objects, we add some dense layer jump connections between the shallow high-resolution feature layers and the deep high-semantic feature layers. For large-scale objects, dilated convolution is used as an ingredient to cover the features of large-scale objects. Based on this, a scale adaptation module is proposed. In this module, different dilated convolution expansion rates are utilized to change the network’s receptive field size, which can adapt to changes from small-scale to large-scale. The experimental results show that the proposed model has better detection performance with different object scales than existing detectors.

A joint deep learning network of point clouds and multiple views for roadside object classification from lidar point clouds

Urban management and survey departments have begun investigating the feasibility ofacquiring data from various laser scanning systems for urban infrastructure measurements and assessments. Roadside objects such as cars, trees, traffic poles, pedestrians, bicycles and e-bicycles describe the static and dynamic urban information available for acquisition. Because of the unstructured nature of 3D point clouds, the rich targets in complex road scenes, and the varying scales of roadside objects, finely classifying these roadside objects from various point clouds is a challenging task. In this paper, we integrate two representations of roadside objects, point clouds and multiview images to propose a point-group-view network named PGVNet for classifying roadside objects into cars, trees, traffic poles, and small objects (pedestrians, bicycles and e-bicycles) from generalized point clouds. To utilize the topological information of the point clouds, we propose a graph attention convolution operation called AtEdgeConv to mine the relationship among the local points and to extract local geometric features. In addition, we employ a hierarchical view-group-object architecture to diminish the redundant information between similar views and to obtain salient viewwise global features. To fuse the local geometric features from the point clouds and the global features from multiview images, we stack an attention-guided fusion network in PGVNet. In particular, we quantify and leverage the global features as an attention mask to capture the intrinsic correlation and discriminability of the local geometric features, which contributes to recognizing the different roadside objects with similar shapes. To verify the effectiveness and generalization of our methods, we conduct extensive experiments on six test datasets of different urban scenes, which were captured by different laser scanning systems, including mobile laser scanning (MLS) systems, unmanned aerial vehicle (UAV)-based laser scanning (ULS) systems and backpack laser scanning (BLS) systems. Experimental results, and comparisons with state-of-the-art methods, demonstrate that the PGVNet model is able to effectively identify various cars, trees, traffic poles and small objects from generalized point clouds, and achieves promising performances on roadside object classifications, with an overall accuracy of 95.76%. Our code is released on https://github.com/flidarcode/PGVNet.

AN APPROACH FOR VEHICLE TARGET DETECTION USING YOLO V3

In recent years, autonomous driving has become the key development and transformation direction of the traditional automobile industry. In the complex driving environment, automatic driving requires extremely high accuracy and high speed of detection, otherwise, it is easy to lead to traffic accident tragedy. Given the low accuracy of traditional car target detection methods in complex scenes, combined with the current hot development of deep learning, this project applies the YOLO V3 algorithm framework to achieve car target detection. In this project, we will detect cars moving on the road using deep learning techniques. Both detection and tracking vehicles such as cars, trucks, motorcycles, etc are challenging problems, especially in complex real-world scenes that commonly involve multiple vehicles, complicated occlusions, and cluttered or even moving backgrounds. This project mainly focuses on car detection can locate cars even in complex street scenes and it can apply to other vehicles by including other datasets and their class names, but false positives have remained frequent. The approximate articulation of each car is detected using YOLO V3 pretrained detector network and untrained network and their average precision and confidence scores are compared among them. The vehicle dataset having cars is used here. Software Tools: MATLAB 2021a or Above Keywords — Object Detection, Convolutional Neural Network, Deep Learning, YOLO V3