Traffic Scenes Research Articles

Smooth fusion of multi-spectral images via total variation minimization for traffic scene semantic segmentation

Achieving precise semantic segmentation for traffic scenes relies on adopting multi-spectral image fusion techniques to attain high-quality images. Many existing fusion solutions often aim to enhance the similarity between the input and fusion results at the pixel intensity and texture details stage. However, this can result in smoothness issues that limit semantic segmentation performance. To address these issues, we present a smooth representation learning optimization mechanism (SFLM) that conducts image fusion on two dimensions: inter- and intra-image levels. The former overcomes over- or under-smoothing problems via the mutual information maximization between the fusion result and image samples (i.e., negative and positive). The latter balances under and over-smoothing for fusion results by minimizing the total variation in pixel space and maximizing the total variation in gradient space based on contrast learning. In this way, the proposed method effectively overcomes the fusion quality issues, providing better feature representations for semantic segmentation in autonomous vehicles. Experimental results on four public datasets validate our method’s effectiveness, robustness, and overall superiority.

Query-Informed Multi-Agent Motion Prediction.

In a dynamic environment, autonomous driving vehicles require accurate decision-making and trajectory planning. To achieve this, autonomous vehicles need to understand their surrounding environment and predict the behavior and future trajectories of other traffic participants. In recent years, vectorization methods have dominated the field of motion prediction due to their ability to capture complex interactions in traffic scenes. However, existing research using vectorization methods for scene encoding often overlooks important physical information about vehicles, such as speed and heading angle, relying solely on displacement to represent the physical attributes of agents. This approach is insufficient for accurate trajectory prediction models. Additionally, agents' future trajectories can be diverse, such as proceeding straight or making left or right turns at intersections. Therefore, the output of trajectory prediction models should be multimodal to account for these variations. Existing research has used multiple regression heads to output future trajectories and confidence, but the results have been suboptimal. To address these issues, we propose QINET, a method for accurate multimodal trajectory prediction for all agents in a scene. In the scene encoding part, we enhance the feature attributes of agent vehicles to better represent the physical information of agents in the scene. Our scene representation also possesses rotational and spatial invariance. In the decoder part, we use cross-attention and induce the generation of multimodal future trajectories by employing a self-learned query matrix. Experimental results demonstrate that QINET achieves state-of-the-art performance on the Argoverse motion prediction benchmark and is capable of fast multimodal trajectory prediction for multiple agents.

ETSR-YOLO: An improved multi-scale traffic sign detection algorithm based on YOLOv5.

In the application of driverless technology, current traffic sign recognition methods are susceptible to the influence of ambient light interference, target size changes and complex backgrounds, resulting in reduced recognition accuracy. To address these challenges, this study introduces an optimisation algorithm called ETSR-YOLO, which is based on the YOLOv5s algorithm. First, this study improves the path aggregation network (PANet) of YOLOv5s to enhance multi-scale feature fusion by generating an additional high-resolution feature layer to improve the recognition of YOLOv5s for small-sized objects. Second, the study introduces two improved C3 modules that aim to suppress background noise interference and enhance the feature extraction capabilities of the network. Finally, the study uses the Wise-IoU (WIoU) function in the post-processing stage to improve the learning ability and robustness of the algorithm to different samples. The experimental results show that ETSR-YOLO improves mAP@0.5 by 6.6% on the Tsinghua-Tencent 100K (TT100K) dataset and by 1.9% on the CSUST Chinese Traffic Sign Detection Benchmark 2021 (CCTSDB2021) dataset. In the experiments conducted on the embedded computing platform, ETSR-YOLO demonstrates a short average inference time, thereby affirming its capability to deliver dependable traffic sign detection for intelligent vehicles operating in real-world traffic scenes. The source code and test results of the models used in this study are available at https://github.com/cbrook16/ETSR-YOLO.

Mobip: a lightweight model for driving perception using MobileNet.

The visual perception model is critical to autonomous driving systems. It provides the information necessary for self-driving cars to make decisions in traffic scenes. We propose a lightweight multi-task network (Mobip) to simultaneously perform traffic object detection, drivable area segmentation, and lane line detection. The network consists of a shared encoder for feature extraction and two decoders for handling detection and segmentation tasks collectively. By using MobileNetV2 as the backbone and an extremely efficient multi-task architecture to implement the perception model, our network has great advantages in inference speed. The performance of the multi-task network is verified on a challenging public Berkeley Deep Drive(BDD100K) dataset. The model achieves an inference speed of 58 FPS on NVIDIA Tesla V100 while still maintaining competitive performance on all three tasks compared to other multi-task networks. Besides, the effectiveness and efficiency of the multi-task architecture are verified via ablative studies.

A Deep Reinforcement Learning Algorithm Suitable for Autonomous Vehicles: Double Bootstrapped Soft-Actor–Critic-Discrete

With the rapid advancement of modern society, autonomous systems have been broadly applied in people’s daily lives. Under the guidance of this trend, autonomous vehicles have gradually become popular. However, due to some adverse factors(such as insufficient computing force and limited communication bandwidth) in edge computing scenarios and the lack of autonomous decision-making ability, the safety of autonomous vehicles is not enough. It is a good solution to use a deep reinforcement learning(DRL) algorithm, which combines deep learning(DL) and reinforcement learning(RL), to provide a fast convergence speed and an appropriate decision-making ability. In this paper, based on Soft-Actor-Critic(SAC) and Soft-Actor-Critic-Discrete(SAC-D), we propose a Double Bootstrapped Soft-Actor-Critic-Discrete(DBSAC-D) algorithm. By introducing bootstrap, the ability to explore in action space is enhanced, the value of each action is accurately judged, the convergence process is accelerated and the consumption of the computing force is reduced. In addition, we also propose a novel sampling strategy, which balances the novelty and importance of the sampled data, and improves the training value of the sampled data to the network model. The experimental results show that our proposed algorithm achieves good performances in several traffic scenes and has a fast convergence speed.

Automatic Roadside Camera Calibration with Transformers

Previous camera self-calibration methods have exhibited certain notable shortcomings. On the one hand, they either exclusively emphasized scene cues or solely focused on vehicle-related cues, resulting in a lack of adaptability to diverse scenarios and a limited number of effective features. Furthermore, these methods either solely utilized geometric features within traffic scenes or exclusively extracted semantic information, failing to comprehensively consider both aspects. This limited the comprehensive feature extraction from scenes, ultimately leading to a decrease in calibration accuracy. Additionally, conventional vanishing point-based self-calibration methods often required the design of additional edge-background models and manual parameter tuning, thereby increasing operational complexity and the potential for errors. Given these observed limitations, and in order to address these challenges, we propose an innovative roadside camera self-calibration model based on the Transformer architecture. This model possesses a unique capability to simultaneously learn scene features and vehicle features within traffic scenarios while considering both geometric and semantic information. Through this approach, our model can overcome the constraints of prior methods, enhancing calibration accuracy and robustness while reducing operational complexity and the potential for errors. Our method outperforms existing approaches on both real-world dataset scenarios and publicly available datasets, demonstrating the effectiveness of our approach.

Quantitative Analysis of Dangerous Driving Behavior through Drone Footage using Mathematical Modeling

This research paper presents a method for analyzing driving using aerial drone images of traffic scenes. By applying principles of motion and calculus we introduce an approach that provides insights into driving dynamics without relying on complex vehicle markings or annotations. This method offers a perspective on road behaviors, allowing for an understanding of potential risks and ultimately enhancing road safety. In our experiment we examined a series of drone images capturing traffic scenarios. We provided a procedure and demonstration of applications of physics and calculus to reconstruct traffic scenes. We constructed a trajectory mechanism that uses all of said concepts to form a concrete plan to reconstruct scenes, using real drone footage with bird’s eye view. The combination of concepts with aerial image analysis showcases the potential for objective and data driven assessments of reckless driving behaviors. Overall this paper serves as a starting point for advancements, in utilizing models to analyze traffic patterns. It contributes to promoting road practices and improving transportation management through evidence based approaches.

Moving Target Detection Algorithm for Millimeter Wave Radar Based on Keystone-2DFFT

Millimeter wave radar has the advantage of all-day and all-weather capability for detection, speed measurement. It plays an important role in urban traffic flow monitoring and traffic safety monitoring. The conventional 2-dimensional Fast Fourier Transform (2DFFT) algorithm is performed target detection in the range-Doppler domain. However, the target motion will induce the range walk phenomenon, which leads to a decrease in the target energy and the performance of the target detection and speed measurement. To solve the above problems, this paper proposes a moving vehicle detection algorithm based on Keystone-2DFFT for a traffic scene. Firstly, this paper constructs and analyzes the Frequency Modulated ContinuousWave (FMCW) moving target signal model under traffic monitoring scenario’s radar observation geometry. The traditional 2DFFT moving target detection algorithm is briefly introduced. Then, based on mentioned signal model, an improved moving vehicle detection algorithm based on Keystone-2DFFT transform is proposed. The method first input the echo, then the range walk is removed by keystone transformation. the keystone transformation is achieved via Sinc interpolation. Next is transform data into range-Doppler domain to perform detection and speed estimation. The algorithm is verified by simulation data and real data.

Convolutional Neural Network-Based Vehicle Classification in Low-Quality Imaging Conditions for Internet of Things Devices

Vehicle classification has an important role in the efficient implementation of Internet of Things (IoT)-based intelligent transportation system (ITS) applications. Nowadays, because of their higher performance, convolutional neural networks (CNNs) are mostly used for vehicle classification. However, the computational complexity of CNNs and high-resolution data provided by high-quality monitoring cameras can pose significant challenges due to limited IoT device resources. In order to address this issue, this study aims to propose a simple CNN-based model for vehicle classification in low-quality images collected by a standard security camera positioned far from a traffic scene under low lighting and different weather conditions. For this purpose, firstly, a new dataset that contains 4800 low-quality vehicle images with 100 × 100 pixels and a 96 dpi resolution was created. Then, the proposed model and several well-known CNN-based models were tested on the created dataset. The results demonstrate that the proposed model achieved 95.8% accuracy, outperforming Inception v3, Inception-ResNet v2, Xception, and VGG19. While DenseNet121 and ResNet50 achieved better accuracy, their complexity in terms of higher trainable parameters, layers, and training times might be a significant concern in practice. In this context, the results suggest that the proposed model could be a feasible option for IoT devices used in ITS applications due to its simple architecture.

TREYESCAN: configuration of an eye tracking test for the measurement of compensatory eye movements in patients with visual field defects

The Traffic Eye Scanning and Compensation Analyzer (TREYESCAN) is introduced as an innovative eye tracking test designed to measure compensatory eye movements in individuals with visual field defects. The primary objective of the test is to quantitatively assess and analyze the compensatory eye movements employed by patients with visual field defects while viewing videos of various traffic scenes from the viewpoint of a driver of a passenger car. The filming process involved capturing a wide range of driving conditions and hazards, aiming to replicate real-world scenarios. Specific dynamic areas of interest within these scenes were selected and assessed by a panel of experts on medical and practical fitness to drive. Pilot measurements were conducted on a sample of 20 normally-sighted individuals during two different measurement sessions. The results provide valuable insights into how individuals without visual impairment view the dynamic scenes presented in the test. Moving forward, the TREYESCAN will be used in a case–control study involving glaucoma patients and control subjects, with the goal of further investigating and understanding the mechanisms employed by individuals with glaucoma to compensate for their visual field defects.

Preprocessing Effects in Road Traffic Scene Analysis Using Machine Learning Models

Image-based real-time object detection is a key issue in integrated automatic traffic monitoring and control systems and is one of the main research topics in Intelligent Transportation Systems (ITS). Despite the growing interest in the application of image processing techniques for traffic scene analysis, vehicle detection algorithms are still unreliable in complex real-world backgrounds. Machine learning models are increasingly being applied to automatic video-based object detection and traffic scene analysis due to their excellent performance in real-time pattern recognition. However, the performance of machine learning models is highly dependent on the properties of the input vectors. Studies have shown that the preprocessing of raw image data obtained from digital video can improve the predictive performance of machine learning models. The purpose of this study is to investigate the predictive performance of machine learning models using preprocessing methods such as image size reduction and image filtering. The experiment was performed with the Backpropagation model, which is one of the most popular machine learning models, and the predictive performance was compared to see how it could be affected by different preprocessing techniques. Deteksi objek real-time berbasis gambar adalah isu kunci dalam sistem pemantauan dan kontrol lalu lintas otomatis terintegrasi dan merupakan salah satu topik penelitian utama dalam Sistem Transportasi Cerdas. Meskipun minat yang meningkat dalam penerapan teknik pemrosesan gambar untuk analisis scene lalu lintas, algoritma deteksi kendaraan masih belum dapat diandalkan dalam latar belakang dunia nyata yang kompleks. Model pembelajaran mesin semakin diterapkan pada deteksi objek berbasis video otomatis dan analisis scene lalu lintas karena kinerja mereka yang sangat baik dalam pengenalan pola real-time. Namun, kinerja model pembelajaran mesin sangat bergantung pada sifat vektor masukan. Studi telah menunjukkan bahwa pra-pemrosesan data gambar mentah yang diperoleh dari video digital dapat meningkatkan kinerja prediktif dari model pembelajaran mesin. Tujuan dari penelitian ini adalah untuk menyelidiki kinerja prediktif dari model pembelajaran mesin menggunakan metode pra-pemrosesan seperti reduksi ukuran gambar dan penyaringan gambar. Percobaan dilakukan dengan model Backpropagation, yang merupakan salah satu model pembelajaran mesin paling populer, dan kinerja prediktifnya dibandingkan untuk melihat bagaimana hal itu dapat dipengaruhi oleh berbagai teknik pra-pemrosesan.

Study on driver’s active emergency response in dangerous traffic scenes based on driving simulator

Objective A driver’s active emergency response in dangerous traffic scenes consists of two parts, including reaction behavior and physiological state. In dangerous traffic scenes, the driver’s active emergency response has an important impact on human collision injury. Clarifying the driver’s active emergency response is an important prerequisite for the study of human collision injury under nonstandard posture. Therefore, this study investigates the driver’s active emergency response in different inevitable collision scenes using driving simulator. Methods A driving simulator with a high-speed camera system and human physiological signal acquisition system was first built. Then, three typical vehicle dangerous collision scenes were developed, including frontal collision, side collision, and rear-end collision. Finally, twenty participants (15 males and 5 females) were recruited for a driving experiment, and their active emergency responses were recorded and analyzed. Results All subjects would rotate the steering wheel to the left or right in the active emergency state, and the rotation of the hand would also cause the subject’s upper body to tilt in the same direction. The maximum angle for male subjects to rotate the steering wheel was 59.98°, while for the female subjects, it was 44.28°. In addition, the maximum grip force between the male subjects and the steering wheel was 280.5 N, compared to 192.5 N for female subjects. Compared to the female participants, the male participants not only have a greater rotation angle and a greater grip force on the steering wheel, but also have greater pressure on the brake pedal, and the foot moves quickly from the accelerator pedal to the brake pedal and presses the brake pedal. Conclusions Drivers have different active emergency responses to different vehicle collision scenes. Quantitative statistics of driver’s active emergency response will have important guiding significance for the analysis of the impact of human active emergency response on human injury characteristics in subsequent vehicle collision experiments.

Direction-Oriented Topic Modeling with Applications in Traffic Scene Analysis

Direction-Oriented Topic Modeling with Applications in Traffic Scene Analysis

A Review on Cross Weather Traffic Scene Understanding Using Transfer Learning for Intelligent Transport System

Intelligent transport systems (ITS) have revolutionized the transportation industry by integrating cutting-edge technologies to enhance road safety, reduce traffic congestion and optimize the transportation network. Scene understanding is a critical component of ITS that enables real-time decision-making by interpreting the environment's contextual information. However, achieving accurate scene understanding requires vast amounts of labeled data, which can be costly and time-consuming. It is quite challenging to Understand traffic scene captured from vehicle mounted cameras. In recent times, the combination of road scene-graph representations and graph learning techniques has demonstrated superior performance compared to cutting-edge deep learning methods across various tasks such as action classification, risk assessment, and collision prediction. It's a grueling problem due to large variations under different illumination conditions. Transfer learning is a promising approach to address this challenge. Transfer learning involves leveraging pre-trained deep learning models on large-scale datasets to develop efficient models for new tasks with limited data. In the context of ITS, transfer learning can enable accurate scene understanding with less data by reusing learned features from other domains.
 This paper presents a comprehensive overview of the application of transfer learning for scene understanding in cross domain. It highlights the benefits of transfer learning for ITS and presents various transfer learning techniques used for scene understanding. This survey paper provides systematic review on cross domain outdoor scene understanding and transfer learning approaches from different perspective, presents information on current state of art and significant methods in choosing the right transfer learning model for specific scene understanding applications.

GSAA: A Novel Graph Spatiotemporal Attention Algorithm for Smart City Traffic Prediction

With the development of 5G and Internet of Things technologies, the application process of smart transportation in smart cities continues to advance. Sensors are a key source of information for smart transportation, and their data commonly includes complicated traffic scene information. Urban traffic scheduling and efficiency can be significantly increased by deploying data from smart sensors to forecast traffic flows. Despite the fact that some related works have focused on the prediction task of traffic flows, they have not completely mined the traffic spatiotemporal information present in smart sensor data. We offer a novel graph spatio-temporal attention algorithm (GSAA) for traffic prediction in this paper. To completely exploit the geographical and temporal correlations among complicated roadways for traffic forecast, the algorithm combines a spatiotemporal attention mechanism with a graph neural network.To take full advantage of how much effect various hyperparameters provide, deep reinforcement learning is used to obtain the optimal hyperparameters while the predictive model is trained. Experimental results on real-world public datasets show that the algorithm proposed in this paper achieves performance improvements of about 5.47% and 13.10% over the MAE (mean absolute error) than the best baseline strategies for short-term and long-term traffic forecasting, respectively.

Towards explainable motion prediction using heterogeneous graph representations

Motion prediction systems play a crucial role in enabling autonomous vehicles to navigate safely and efficiently in complex traffic scenarios. Graph Neural Network (GNN)-based approaches have emerged as a promising solution for capturing interactions among dynamic agents and static objects. However, they often lack transparency, interpretability and explainability — qualities that are essential for building trust in autonomous driving systems. In this work, we address this challenge by presenting a comprehensive approach to enhance the explainability of graph-based motion prediction systems. We introduce the Explainable Heterogeneous Graph-based Policy (XHGP) model based on an heterogeneous graph representation of the traffic scene and lane-graph traversals. Distinct from other graph-based models, XHGP leverages object-level and type-level attention mechanisms to learn interaction behaviors, providing information about the importance of agents and interactions in the scene. In addition, capitalizing on XHGP’s architecture, we investigate the explanations provided by the GNNExplainer and apply counterfactual reasoning to analyze the sensitivity of the model to modifications of the input data. This includes masking scene elements, altering trajectories, and adding or removing dynamic agents. Our proposal advances towards achieving reliable and explainable motion prediction systems, addressing the concerns of users, developers and regulatory agencies alike. The insights gained from our explainability analysis contribute to a better understanding of the relationships between dynamic and static elements in traffic scenarios, facilitating the interpretation of the results, as well as the correction of possible errors in motion prediction models, and thus contributing to the development of trustworthy motion prediction systems.The code to reproduce this work is publicly available at https://github.com/sancarlim/Explainable-MP/tree/v1.1.

Real-time Lightweight Target Detection Network under Autonomous Driving

Target detection in traffic scenes has been a focal point concerning computer vision, especially in the context of autonomous driving, including the recognition of vehicles, pedestrians, and traffic signs. However, the computational power of devices operating in traffic scenarios is limited, placing stringent demands on computational resources and latency. To address these challenges, this study proposed a lightweight detection algorithm, GV2-YOLO, which prioritizes both speed and accuracy. First, we apply the GhostNetv2 architecture and incorporate the Ghost module to decrease the parameters of the backbone feature extraction network. Second, the algorithm also integrates SPPF and Slim-neck by GSConv techniques, which effectively reduces the computational complexity while ensuring the resulting significant reduction in detection accuracy. Our proposed algorithm achieves a mAP of 84.7% on the CCTSDB dataset. The algorithm has a total parameter count of 9.1 M, making it ideal for deployment in embedded autonomous driving platforms.

Analysis of pedestrian crossing behavior based on Centralized Unscented Kalman Filter and pedestrian awareness based social force model

With the development of autonomous driving, ensuring pedestrian safety has become a hot research topic. The traditional pedestrian crossing behavior research mostly track and predict pedestrian crossing from the view of vehicles, but do not have in-depth research on the impact of other elements in the traffic scene. This paper proposes a method for analyzing pedestrian crossing behavior based on roadside equipment for specific road sections. First, a pedestrian awareness-based social force model (PASFM) is proposed by introducing the factors that influence pedestrian crossing decisions in traffic scenarios into the social force model (SFM), including zebra crossings, fellow walkers and vehicles. To explore the psychological state of vehicles and pedestrians when they interact, evolutionary game theory is used to simulate the psychological characteristics of pedestrians when they encounter vehicles, and then reflect in the changes of the desired speed. Then Centralized Unscented Kalman Filter (CUKF) is proposed to complete the whole process of tracking and predicting the motion state of pedestrian crossing behavior, which uses PASFM as its prediction process. Finally, simulations and experiments are designed to show the effectiveness of our methods. Results show that the proposed method performs better during pedestrian tracking and can predict their crossing behavior to a certain extent.

Real-time risk assessment of road vehicles based on inverse perspective mapping

Pan/Tilt/Zoom (PTZ) cameras play an important role in traffic scenes due to their wide monitoring fields and high flexibility. However, since the focal length and angle of PTZ cameras change irregularly with the monitoring needs, it is difficult to obtain accurate physical information about the real world from the image information of PTZ cameras. Aiming to address the need for real-time risk assessment of road vehicles in traffic monitoring scenarios, a vehicle position and velocity measurement scheme based on camera inverse perspective transformation is proposed, along with a method for real-time risk assessment based on the position and velocity. Specifically, Firstly, the vehicle target in the video is detected and tracked by deep learning YOLO detection algorithm and optical flow tracking algorithm. According to the obtained trajectory set, the vanishing points in the road direction are calculated by Cascade Hough Transform and the road marking lines are detected. Then, according to the vanishing point and marking line, the camera calibration task is accomplished via exploratory focal length. After camera calibration, the camera-to-road inverse perspective transformation is applied to project the image plane onto the road surface and obtain, the actual position information of vehicles. Finally, the vehicle speed measurement and real-time road risk assessment are achieved by calculating the average of instantaneous velocities across multiple frames. Simulation experiment results in a traffic monitoring scenario demonstrate that this perspective-based method for real-time road vehicle risk assessment achieves good stability and practicality, which meets the requirements for vehicle speed measurement and real-time road risk assessment.

Al-yousofi: A Novel Benchmark For Multi-vehicle Detection And Tracking

Vehicle detection in aerial imagery has been instrumental in a wide range of applications. Lately, as a result of the robust feature representations, convolutional neural networks (CNN) based detection methods have achieved prodigious performance in computer vision. The diversity of dataset sources that relate to vehicle images is numerous, but it is not sufficient in some cases due to the different types of vehicles from one country to another. In this paper, we propose a new dataset of vehicle images called AL-YOUSOFI taken in the Hashemite Kingdom of Jordan, Irbid city. The AL-YOUSOFI benchmark dataset consists of 40 challenging videos captured from real-world traffic scenes (over 158,000 frames with rich annotations, including vehicle type and vehicle bounding boxes) for multi-object detection and tracking. The state-of-the-art algorithms that have been previously trained on AL-YOUSOFI, has been evaluated. The results showed a variation in efficiency between the algorithms, and this is due to how each works. The full dataset is available at this URL.