Road Scene Research Articles

Mapping property redevelopment via GeoAI: Integrating computer vision and socioenvironmental patterns and processes

Domain knowledge of social and environmental sciences is generally derived from less structured small data and/or small models. The integration of deep learning with socioenvironmental and geographical patterns and processes is still in an early phase. This study proposes a flexible framework that synthesizes computer vision with patterns and processes of geographical phenomena (e.g., urban redevelopment) via deep convolutional neural networks and spatiotemporal neighborhoods, respectively. Meanwhile, undesirable visual cues (e.g., temporary objects such as cars) cause false alarms in urban change detection. Thus, a masked deep pyramid similarity model (i.e., the computer vision model) is proposed to minimize the negative impact of nonbuilding changes. This pipeline has robustness against obfuscation of undesirable street scene changes and elasticity of geographical knowledge representations in mapping property redevelopment. This model is reproducible and adaptable due to the wide availability of SVI and flexibility of the framework. The results suggest that the domain-driven rules and nonbuilding change masking mechanism can significantly increase the accuracy of a computer vision model. We also find that urban redevelopment should be understood as a locally tuned back-to-the-city process with weak replicability. Hybrid gentrification (i.e., the combination of “seesaw” gentrification and continuous gentrification) is observed globally with local variations. Takeaway for practiceFirst, the population of those affected should be carefully identified and the benefits of gentrification need to be redistributed on a larger scale. For Auckland, decision-makers should not divert attention away from low-income tenants in outer suburbs that slipped below the radar of local government and scholars. Second, more attention should be given to less resourcing neighborhoods in constantly gentrifying neighborhoods when governments formulate and implement housing subsidy policies. Locally, incessant redevelopment in Auckland CBD, New Market and Remuera, etc., should be contained.

Simulation-Based Self-Supervised Line Extraction for LiDAR Odometry in Urban Road Scenes

LiDAR odometry is a fundamental task for high-precision map construction and real-time and accurate localization in autonomous driving. However, point clouds in urban road scenes acquired by vehicle-borne lasers are of large amounts, “near dense and far sparse” density, and contain different dynamic objects, leading to low efficiency and low accuracy of existing LiDAR odometry methods. To address the above issues, a simulation-based self-supervised line extraction in urban road scene is proposed, as a pre-processing for LiDAR odometry to reduce the amount of input and the interference from dynamic objects. A simulated dataset is first constructed according to the characteristics of point clouds in urban road scenes; and then, an EdgeConv-based network, named LO-LineNet, is used for pre-training; finally, a model transferring strategy is adopted to transfer the pre-trained model from a simulated dataset to real-world scenes without ground-truth labels. Experimental results on the KITTI Odometry Dataset and the Apollo SouthBay Dataset indicate that the proposed method can accurately extract reliable lines in urban road scenes in a self-supervised way, and the use of the extracted reliable lines as input for odometry can significantly improve its accuracy and efficiency in urban road scenes.

A Computer Vision-Based Algorithm for Detecting Vehicle Yielding to Pedestrians

Computer vision has made remarkable progress in traffic surveillance, but determining whether a motor vehicle yields to pedestrians still requires considerable human effort. This study proposes an automated method for detecting whether a vehicle yields to pedestrians in intelligent transportation systems. The method employs a target-tracking algorithm that uses feature maps and license plate IDs to track the motion of relevant elements in the camera’s field of view. By analyzing the positions of motor vehicles and pedestrians over time, we predict the warning points of pedestrians and hazardous areas in front of vehicles to determine whether the vehicles yield to pedestrians. Extensive experiments are conducted on the MOT16 dataset, real traffic street scene video dataset, and a Unity3D virtual simulation scene dataset combined with SUMO, which demonstrating the superiority of this tracking algorithms. Compared to the current state-of-the-art methods, this method demonstrates significant improvements in processing speed without compromising accuracy. Specifically, this approach substantially outperforms in operational efficiency, thus catering aptly to real-time recognition requirements. This meticulous experimentation and evaluations reveal a commendable reduction in ID switches, enhancing the reliability of violation attributions to the correct vehicles. Such enhancement is crucial in practical urban settings characterized by dynamic interactions and variable conditions. This approach can be applied in various weather, time, and road conditions, achieving high predictive accuracy and interpretability in detecting vehicle–pedestrian interactions. This advanced algorithm illuminates the viable pathways for integrating technological innovation and sustainability, paving the way for more resilient and intelligent urban ecosystems.

Image representation and reconstruction by compositing Gaussian ellipses

AbstractIn this paper, a method of stroke‐based rendering is proposed for image representation and reconstruction. The proposed method involves compositing a set of ellipses that greatly vary in location, size, rotation angle, color, and opacity. This study proves that alpha compositing is differentiable if a two‐dimensional Gaussian function is used to draw a solid ellipse. The gradient method can then be employed for automatically identifying the parameters of each ellipse such that the difference between the input image and the composited image is minimized. Experimental results indicate that the proposed method can represent various types of images including portraits, landscapes, buildings, street scenes, artificial objects, medical images etc. The proposed method can particularly render the most details and significant features of an input image with fewer strokes compared to other stroke‐based rendering algorithms. This study also demonstrates that the proposed method can be applied in painting style simulation and sparse‐view computed tomography imaging.

A Knowledge-Guided Fusion Visualisation Method of Digital Twin Scenes for Mountain Highways

Informatization is an important trend in the field of mountain highway management, and the digital twin is an effective way to promote mountain highway information management due to the complex and diverse terrain of mountainous areas, the high complexity of mountainous road scene modeling and low visualisation efficiency. It is challenging to construct the digital twin scenarios efficiently for mountain highways. To solve this problem, this article proposes a knowledge-guided fusion expression method for digital twin scenes of mountain highways. First, we explore the expression features and interrelationships of mountain highway scenes to establish the knowledge graph of mountain highway scenes. Second, by utilizing scene knowledge to construct spatial semantic constraint rules, we achieve efficient fusion modeling of basic geographic scenes and dynamic and static ancillary facilities, thereby reducing the complexity of scene modeling. Finally, a multi-level visualisation publishing scheme is established to improve the efficiency of scene visualisation. On this basis, a prototype system is developed, and case experimental analysis is conducted to validate the research. The results of the experiment indicate that the suggested method can accomplish the fusion modelling of mountain highway scenes through knowledge guidance and semantic constraints. Moreover, the construction time for the model fusion is less than 5.7 ms; meanwhile, the dynamic drawing efficiency of the scene is maintained above 60 FPS. Thus, the construction of twinned scenes can be achieved quickly and efficiently, the effect of replicating reality with virtuality is accomplished, and the informatisation management capacity of mountain highways is enhanced.

Multi-object detection for crowded road scene based on ML-AFP of YOLOv5

Aiming at the problem of multi-object detection such as target occlusion and tiny targets in road scenes, this paper proposes an improved YOLOv5 multi-object detection model based on ML-AFP (multi-level aggregation feature perception) mechanism. Since tiny targets such as non-motor vehicle and pedestrians are not easily detected, this paper adds a micro target detection layer and a double head mechanism to improve the detection ability of tiny targets. Varifocal loss is used to achieve a more accurate ranking in the process of non-maximum suppression to solve the problem of target occlusion, and this paper also proposes a ML-AFP mechanism. The adaptive fusion of spatial feature information at different scales improves the expression ability of network model features, and improves the detection accuracy of the model as a whole. Our experimental results on multiple challenging datasets such as KITTI, BDD100K, and show that the accuracy, recall rate and mAP value of the proposed model are greatly improved, which solves the problem of multi-object detection in crowded road scenes.

CDAU-Net: A Novel CoordConv-Integrated Deep Dual Cross Attention Mechanism for Enhanced Road Extraction in Remote Sensing Imagery

In the realm of remote sensing image analysis, the task of road extraction poses significant complexities, especially in the context of intricate scenes and diminutive targets. In response to these challenges, we have developed a novel deep learning network, christened CDAU-Net, designed to discern and delineate these features with enhanced precision. This network takes its structural inspiration from the fundamental architecture of U-Net while introducing innovative enhancements: we have integrated CoordConv convolutions into both the initial layer of the U-Net encoder and the terminal layer of the decoder, thereby facilitating a more efficacious processing of spatial information inherent in remote sensing images. Moreover, we have devised a unique mechanism termed the Deep Dual Cross Attention (DDCA), purposed to capture long-range dependencies within images—a critical factor in remote sensing image analysis. Our network replaces the skip-connection component of the U-Net with this newly designed mechanism, dealing with feature maps of the first four scales in the encoder and generating four corresponding outputs. These outputs are subsequently linked with the decoder stage to further capture the remote dependencies present within the remote sensing imagery. We have subjected CDAU-Net to extensive empirical validation, including testing on the Massachusetts Road Dataset and DeepGlobe Road Dataset. Both datasets encompass a diverse range of complex road scenes, making them ideal for evaluating the performance of road extraction algorithms. The experimental results showcase that whether in terms of accuracy, recall rate, or Intersection over Union (IoU) metrics, the CDAU-Net outperforms existing state-of-the-art methods in the task of road extraction. These findings substantiate the effectiveness and superiority of our approach in handling complex scenes and small targets, as well as in capturing long-range dependencies in remote sensing imagery. In sum, the design of CDAU-Net not only enhances the accuracy of road extraction but also presents new perspectives and possibilities for deep learning analysis of remote sensing imagery.

Retracted: The Segmentation of Road Scenes Based on Improved ESPNet Model

Retracted: The Segmentation of Road Scenes Based on Improved ESPNet Model

Enhanced YOLOv5: An Efficient Road Object Detection Method.

Accurate identification of road objects is crucial for achieving intelligent traffic systems. However, developing efficient and accurate road object detection methods in complex traffic scenarios has always been a challenging task. The objective of this study was to improve the target detection algorithm for road object detection by enhancing the algorithm's capability to fuse features of different scales and levels, thereby improving the accurate identification of objects in complex road scenes. We propose an improved method called the Enhanced YOLOv5 algorithm for road object detection. By introducing the Bidirectional Feature Pyramid Network (BiFPN) into the YOLOv5 algorithm, we address the challenges of multi-scale and multi-level feature fusion and enhance the detection capability for objects of different sizes. Additionally, we integrate the Convolutional Block Attention Module (CBAM) into the existing YOLOv5 model to enhance its feature representation capability. Furthermore, we employ a new non-maximum suppression technique called Distance Intersection Over Union (DIOU) to effectively address issues such as misjudgment and duplicate detection when significant overlap occurs between bounding boxes. We use mean Average Precision (mAP) and Precision (P) as evaluation metrics. Finally, experimental results on the BDD100K dataset demonstrate that the improved YOLOv5 algorithm achieves a 1.6% increase in object detection mAP, while the P value increases by 5.3%, effectively improving the accuracy and robustness of road object recognition.

Road Scene Instance Segmentation Based on Improved SOLOv2

Road instance segmentation is vital for autonomous driving, yet the current algorithms struggle in complex city environments, with issues like poor small object segmentation, low-quality mask edge contours, slow processing, and limited model adaptability. This paper introduces an enhanced instance segmentation method based on SOLOv2. It integrates the Bottleneck Transformer (BoT) module into VoVNetV2, replacing the standard convolutions with ghost convolutions. Additionally, it replaces ResNet with an improved VoVNetV2 backbone to enhance the feature extraction and segmentation speed. Furthermore, the algorithm employs Feature Pyramid Grids (FPGs) instead of Feature Pyramid Networks (FPNs) to introduce multi-directional lateral connections for better feature fusion. Lastly, it incorporates a convolutional Block Attention Module (CBAM) into the detection head for refined features by considering the attention weight coefficients in both the channel and spatial dimensions. The experimental results demonstrate the algorithm’s effectiveness, achieving a 27.6% mAP on Cityscapes, a 4.2% improvement over SOLOv2. It also attains a segmentation speed of 8.9 FPS, a 1.7 FPS increase over SOLOv2, confirming its practicality for real-world engineering applications.

Vision-based investigation of road traffic and violations at urban roundabout in India using UAV video: A case study

In the past decade, the number of vehicles in India has increased exponentially; however, road infrastructure has not scaled proportionately. As a result, road traffic problems such as congestion on urban roads, dangerous traffic violations, and road accidents have increased significantly. Due to limited road infrastructure, traffic violations (human errors) have intensified in densely populated urban areas. This paper presents a case study (at a multi-lane urban roundabout in Ahmedabad city, India) and the methodology based on computer vision to investigate road traffic and violations using drone/UAV-based aerial video. You Only Look Once-YOLOv7 is used for vehicle detection, and Simple Online and Real Time Tracking-SORT for tracking vehicles. Our methodology divides the road scene (roundabout) into certain zones. We then formulated the dictionary, which maps the traffic violations under Motor Vehicle Driving Regulations - MVDR/ Motor Vehicle Act - MVA and the movement of the vehicle (zone traversal sequences). Using the zone-based methodology, we could also probe other road traffic data such as the count of vehicles, speed of vehicles, rate of traffic flow, and congestion. Based on our results, we also infer some of the possible causes of traffic violations in terms of problems/limitations of road infrastructure. As per our analysis, around 23.26% of vehicles committed traffic violations. We detected traffic violations related to lane indiscipline, driving against the authorized flow of traffic, parking violations, and over-speeding within the roundabout. Our methodology of investigating road traffic and violations can be used for road infrastructure improvement, law enforcement drives, and policy making, for road traffic safety, in developing and densely populated countries.

Performance Evaluation of You Only Look Once v4 in Road Anomaly Detection and Visual Simultaneous Localisation and Mapping for Autonomous Vehicles

The proliferation of autonomous vehicles (AVs) emphasises the pressing need to navigate challenging road networks riddled with anomalies like unapproved speed bumps, potholes, and other hazardous conditions, particularly in low- and middle-income countries. These anomalies not only contribute to driving stress, vehicle damage, and financial implications for users but also elevate the risk of accidents. A significant hurdle for AV deployment is the vehicle’s environmental awareness and the capacity to localise effectively without excessive dependence on pre-defined maps in dynamically evolving contexts. Addressing this overarching challenge, this paper introduces a specialised deep learning model, leveraging YOLO v4, which profiles road surfaces by pinpointing defects, demonstrating a mean average precision (mAP@0.5) of 95.34%. Concurrently, a comprehensive solution—RA-SLAM, which is an enhanced Visual Simultaneous Localisation and Mapping (V-SLAM) mechanism for road scene modeling, integrated with the YOLO v4 algorithm—was developed. This approach precisely detects road anomalies, further refining V-SLAM through a keypoint aggregation algorithm. Collectively, these advancements underscore the potential for a holistic integration into AV’s intelligent navigation systems, ensuring safer and more efficient traversal across intricate road terrains.

Task difficulty, risk feeling, and safety margin in the determination of driver behavior to prepare for traffic conflicts

Unpredictable pedestrian and cyclist behavior associated with their appearance on the road in blind spots contributes to traffic near-misses or crashes. When experienced drivers are confronted with uncertainty, they take defensive measures called hazard-anticipatory driving, such as decreasing the vehicle velocity and/or increasing the lateral distance. Our research sought to understand the motivational determinants and perceptual processes that determine driver behavior in preparation for traffic conflicts with covert hazards.This study aimed to investigate the influence of driving experience on drivers’ perceptions and behaviors to prepare for traffic conflicts. Two experiments were designed with 8 experienced and 13 inexperienced participants. In Experiment 1, participants were asked to provide their subjective impressions of task difficulty, feeling of risk, and statistical risk pertaining to assess their perceptions of the separation between task demand and capability after viewing animation clips of road scenes with blind intersections under different forced speeds. In Experiment 2, participants drove using a driving simulator in scenes with blind intersections, similar to those in Experiment 1. We sought to explore the motivational determinants of behavior regarding the relationship between subjective feelings and objective safety margins.The results showed that the driver’s perception of task difficulty correlated with their driving speed, and inexperienced participants tended to underestimate task difficulty compared to experienced participants. The task difficulty and the feeling of risk were strongly correlated regardless of experience, and estimation of statistical risk differed depending on experience. The subjective task difficulty (and/or risk feeling) and objective safety margin were strongly correlated for experienced participants. Experienced participants who perceived a higher degree of difficulty in the forced-paced driving task tended to have greater safety margins in the self-paced driving task. These findings suggest that experienced participants with individually tolerable safety margins adjust their driving velocity and/or lateral distance in the control of task difficulty (and/or risk feeling) to prepare for traffic conflicts. Therefore, the underestimation of task difficulty should be considered when designing effective measures, such as driver assistance systems, to guide inexperienced drivers toward normative behaviors.

Block attention network: A lightweight deep network for real-time semantic segmentation of road scenes in resource-constrained devices

Deep-learning-based semantic segmentation networks typically incorporate object classification networks in their backbone. This leads to a loss of context because classification networks have a smaller field of view. The architecture has been extended to recover context with additional downsampling feature maps, a parallel context branch, or pyramid pooling modules after the backbone. However, these extensions increase multiply–accumulate operations and memory requirements, thus, making them unsuitable for resource-constrained devices. To overcome this limitation, a novel convolutional building block with attention-based context guidance is proposed. The block is repeated to build an efficient encoder–decoder network. Our network runs in real-time, has a lightweight design with only 0.72 Million parameters, and achieves 70.1%, and 66.3% mean intersection-over-union scores on the highly competitive Cityscapes and CamVid datasets, respectively. An efficient decoder is also designed to replace other semantic segmentation network decoders with minimal performance loss. The performance measures on mobile platforms show that our network suits resource-constrained devices. Further, experimental results show that the proposed method can optimally balance the model size-inference speed and segmentation accuracy.

Based on cross-scale fusion attention mechanism network for semantic segmentation for street scenes.

Semantic segmentation, which is a fundamental task in computer vision. Every pixel will have a specific semantic class assigned to it through semantic segmentation methods. Embedded systems and mobile devices are difficult to deploy high-accuracy segmentation algorithms. Despite the rapid development of semantic segmentation, the balance between speed and accuracy must be improved. As a solution to the above problems, we created a cross-scale fusion attention mechanism network called CFANet, which fuses feature maps from different scales. We first design a novel efficient residual module (ERM), which applies both dilation convolution and factorized convolution. Our CFANet is mainly constructed from ERM. Subsequently, we designed a new multi-branch channel attention mechanism (MCAM) to refine the feature maps at different levels. Experiment results show that CFANet achieved 70.6% mean intersection over union (mIoU) and 67.7% mIoU on Cityscapes and CamVid datasets, respectively, with inference speeds of 118 FPS and 105 FPS on NVIDIA RTX2080Ti GPU cards with 0.84M parameters.

Unification of Road Scene Segmentation Strategies Using Multistream Data and Latent Space Attention.

Road scene understanding, as a field of research, has attracted increasing attention in recent years. The development of road scene understanding capabilities that are applicable to real-world road scenarios has seen numerous complications. This has largely been due to the cost and complexity of achieving human-level scene understanding, at which successful segmentation of road scene elements can be achieved with a mean intersection over union score close to 1.0. There is a need for more of a unified approach to road scene segmentation for use in self-driving systems. Previous works have demonstrated how deep learning methods can be combined to improve the segmentation and perception performance of road scene understanding systems. This paper proposes a novel segmentation system that uses fully connected networks, attention mechanisms, and multiple-input data stream fusion to improve segmentation performance. Results show comparable performance compared to previous works, with a mean intersection over union of 87.4% on the Cityscapes dataset.

CAM-FRN: Class Attention Map-Based Flare Removal Network in Frontal-Viewing Camera Images of Vehicles

In recent years, active research has been conducted on computer vision and artificial intelligence (AI) for autonomous driving to increase the understanding of the importance of object detection technology using a frontal-viewing camera. However, using an RGB camera as a frontal-viewing camera can generate lens flare artifacts due to strong light sources, components of the camera lens, and foreign substances, which damage the images, making the shape of objects in the images unrecognizable. Furthermore, the object detection performance is significantly reduced owing to a lens flare during semantic segmentation performed for autonomous driving. Flare artifacts pose challenges in their removal, as they are caused by various scattering and reflection effects. The state-of-the-art methods using general scene image retain artifactual noises and fail to eliminate flare entirely when there exist severe levels of flare in the input image. In addition, no study has been conducted to solve these problems in the field of semantic segmentation for autonomous driving. Therefore, this study proposed a novel lens flare removal technique based on a class attention map-based flare removal network (CAM-FRN) and a semantic segmentation method using the images in which the lens flare is removed. CAM-FRN is a generative-based flare removal network that estimates flare regions, generates highlighted images as input, and incorporates the estimated regions into the loss function for successful artifact reconstruction and comprehensive flare removal. We synthesized a lens flare using the Cambridge-driving Labeled Video Database (CamVid) and Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago (KITTI) datasets, which are road scene open datasets. The experimental results showed that semantic segmentation accuracy in images with lens flare was removed based on CAM-FRN, exhibiting 71.26% and 60.27% mean intersection over union (mIoU) in the CamVid and KITTI databases, respectively. This indicates that the proposed method is significantly better than state-of-the-art methods.

Deep Dual-Resolution Road Scene Segmentation Networks Based on Decoupled Dynamic Filter and Squeeze-Excitation Module.

Image semantic segmentation is an important part of automatic driving assistance technology. The complexity of road scenes and the real-time requirements of application scenes for segmentation algorithm are the challenges facing segmentation algorithms. In order to meet the above challenges, Deep Dual-resolution Road Scene Segmentation Networks based on Decoupled Dynamic Filter and Squeeze-Excitation (DDF&SE-DDRNet) are proposed in this paper. The proposed DDF&SE-DDRNet uses decoupled dynamic filter in each module to reduce the number of network parameters and enable the network to dynamically adjust the weight of each convolution kernel. We add the Squeeze-and-Excitation module to each module of DDF&SE-DDRNet so that the local feature map in the network can obtain global features to reduce the impact of image local interference on the segmentation result. The experimental results on the Cityscapes dataset show that the segmentation accuracy of DDF&SE-DDRNet is at least 2% higher than that of existing algorithms. Moreover, DDF&SE-DDRNet also has satisfactory inferring speed.

Multiple Pedestrian Tracking With Graph Attention Map on Urban Road Scene

Pedestrians are often vulnerable users of urban roads and ensuring their safety is a pressing challenge in the filed of intelligent transportation. Multiple pedestrian tracking is one of the key technologies for traffic statistics and abnormal behavior analysis, etc. Detection-based tracking methods have achieved remarkable results and have become mainstream processing schemes. However, target association is still immature and less effective in complex scenarios. In the proposed tracking system, several candidates surrounding each detected pedestrian are selected sparsely, and the associating relationship between the target and these candidates is determined based on a graph attention map. These graph attention maps contain positional correlations of the matching pairs and are applicable with pedestrians’ posture variations. The weighted correlating value is estimated with the positional weighted matrix and merged attention map. The correlating relationship is confirmed with the weighted correlating value and distance matching loss. To enhance the computation efficiency of the graph attention maps for these tracked targets and candidates, feature extraction is processed separately. Convolutional features extracted from one specific middle layer of the backbone network are used to represent each target or candidate. The experiment results show that the proposed tracker achieves better performance than the other five state-of-the-art trackers on three publicly available databases.

Where was the moose? The time course of dynamic road scene perception

Where was the moose? The time course of dynamic road scene perception