Traffic Scene Images Research Articles

A Small-Scale Object Detection Algorithm in Intelligent Transportation Scenarios

In response to the problem of poor detection ability of object detection models for small-scale targets in intelligent transportation scenarios, a fusion method is proposed to enhance the features of small-scale targets, starting from feature utilization and fusion methods. The algorithm is based on the YOLOv4 tiny framework and enhances the utilization of shallow and mid-level features on the basis of Feature Pyramid Network (FPN), improving the detection accuracy of small and medium-sized targets. In view of the problem that the background of the intelligent traffic scene image is cluttered, and there is more redundant information, the Convolutional Block Attention Module (CBAM) is used to improve the attention of the model to the traffic target. To address the problem of data imbalance and prior bounding box adaptation in custom traffic data sets that expand traffic images in COCO and VOC, we propose a Copy-Paste method with an improved generation method and a K-means algorithm with improved distance measurement to enhance the model’s detection ability for corresponding categories. Comparative experiments were conducted on a customized 260-thousand traffic data set containing public traffic images, and the results showed that compared to YOLOv4 tiny, the proposed algorithm improved mAP by 4.9% while still ensuring the real-time performance of the model.

Res2Net-based multi-scale and multi-attention model for traffic scene image classification.

With the increasing applications of traffic scene image classification in intelligent transportation systems, there is a growing demand for improved accuracy and robustness in this classification task. However, due to weather conditions, time, lighting variations, and annotation costs, traditional deep learning methods still have limitations in extracting complex traffic scene features and achieving higher recognition accuracy. The previous classification methods for traffic scene images had gaps in multi-scale feature extraction and the combination of frequency domain, spatial, and channel attention. To address these issues, this paper proposes a multi-scale and multi-attention model based on Res2Net. Our proposed framework introduces an Adaptive Feature Refinement Pyramid Module (AFRPM) to enhance multi-scale feature extraction, thus improving the accuracy of traffic scene image classification. Additionally, we integrate frequency domain and spatial-channel attention mechanisms to develop recognition capabilities for complex backgrounds, objects of different scales, and local details in traffic scene images. The paper conducts the task of classifying traffic scene images using the Traffic-Net dataset. The experimental results demonstrate that our model achieves an accuracy of 96.88% on this dataset, which is an improvement of approximately 2% compared to the baseline Res2Net network. Furthermore, we validate the effectiveness of the proposed modules through ablation experiments.

Integrating AI-Technology with Unmanned Aerial Vehicles for Traffic Congestion Recognition

This article explores the integration of unmanned aerial vehicles (UAVs) and artificial intelligence (AI) technology to address traffic congestion. By leveraging UAVs equipped with route-planning capabilities, aerial images of traffic scenes are captured and analyzed using convolutional neural networks (CNNs). The processed data is then transmitted to a traffic management center, enabling timely recognition and response to congestion. This framework offers a practical approach to alleviate traffic delays, reduce fuel consumption, mitigate pollution, and enhance overall road safety

TSIC-CLIP: Traffic Scene Image Captioning Model Based on Clip

Image captioning in traffic scenes presents several challenges, including imprecise caption generation, lack of personalization, and an unwieldy number of model parameters. We propose a new image captioning model for traffic scenes to address these issues. The model incorporates an adapter-based fine-tuned feature extraction part to enhance personalization and a caption generation module using global weighted attention pooling to reduce model parameters and improve accuracy. The proposed model consists of four main stages. In the first stage, the Image-Encoder extracts the global features of the input image and divides it into nine sub-regions, encoding each sub-region separately. In the second stage, the Text-Encoder encodes the text dataset to obtain text features. It then calculates the similarity between the image sub-region features and encoded text features, selecting the text features with the highest similarity. Subsequently, the pre-trained Faster RCNN model extracts local image features. The model then splices together the text features, global image features, and local image features to fuse the multimodal information. In the final stage, the extracted features are fed into the Captioning model, which effectively fuses the different features using a novel global weighted attention pooling layer. The Captioning model then generates natural language image captions. The proposed model is evaluated on the MS-COCO dataset, Flickr 30K dataset, and BUUISE-Image dataset, using mainstream evaluation metrics. Experiments demonstrate significant improvements across all evaluation metrics on the public datasets and strong performance on the BUUISE-Image traffic scene dataset.

Automatic Outer Contour Detection and Quantification of Vehicles Using Monocular Vision

Oversized vehicles have the potential to collide with walls or ceilings when passing through tunnels and bridges, posing a serious threat to the health of transportation infrastructure and public safety. Hence, it is crucial to accurately and immediately detect vehicle dimensions, including length, width, and height, to avoid such accidents. Using computer vision and view geometry, this study presents a framework for automatically detecting and quantifying the outer contours of vehicles through monocular vision. First, traffic scene images are captured which are then used to create a transformation matrix of the ground surface. Second, a modified Mask region-based convolutional neural network (Mask R-CNN) is constructed to detect and segment the vehicle instances from the video frames. Finally, a view geometry-based algorithm was developed to detect the outer contours of passing vehicles and quantify their dimensions. In the field test, the accuracy of the vehicle segmentation and the identified vehicle dimensions was validated. In addition, the proposed method’s superiority was confirmed by comparing it with two other existing approaches. The comparison results show that the proposed method has better accuracy and is more convenient to use since it does not require a premeasured reference. In addition, the developed method can accurately identify not only the dimensions of vehicles parallel to the road but also vehicles that are changing lanes or making a U-turn.

Intelligent road segmentation and obstacle detection for autonomous railway vehicle

For autonomous railway vehicle with complex crisscrossed tracks, it is a huge challenge to intelligently detect the trespassers lying in the possible track regions where the train will move along. In order to solve the issue that the existing object detection algorithms detect all obstacles in traffic scene images, a novel strategy YOLOSEG is proposed for intelligent road segmentation and obstacle detection of railway trespasser. Unet is firstly trained to intelligently segment the railway track region where the train is likely to move on, and then the generated region mask is introduced into object detection network for recognizing obstacle within the mask area. The real video of the obstacle emerging in front of the train is difficult to record, therefore the traffic scenes taken from drivers’ perspectives are randomly combined with various obstacles to create the synthetic training dataset which covers various railway traffic scenarios and lighting conditions, and at the same time the label file is automatically generated. Furthermore, a random brightness strategy is proposed for dataset enhancement. By the performance evaluation comparison of FLOPs, Top-1 Accuracy, and mAP@0.5/%, abundant trespasser detection experiments based on synthetic dataset and real images verify the accuracy and effectiveness of the proposed method.

MRCNet: Multi-Level Residual Connectivity Network for Image Classification

Computer vision obtains object and environment information by simulating human visual senses and borrowing human sensory activity. As one of the main tasks of computer vision, image classification can be used not only for face recognition, traffic scene recognition, image retrieval, and automatic photo categorization but also as a theoretical basis for target detection and image segmentation. In this paper, we use the existing CNN architecture network-ConvNeXt. By adapting and modifying the residual connectivity and convolutional structure of the network, we achieve a balance between classification accuracy and inference speed. These modifications are able to reduce both computation and memory consumption while keeping accuracy largely unchanged, thus better facilitating network lightweighting.

Quantitative Analysis of Dangerous Driving Behavior through Drone Footage using Mathematical Modeling

This research paper presents a method for analyzing driving using aerial drone images of traffic scenes. By applying principles of motion and calculus we introduce an approach that provides insights into driving dynamics without relying on complex vehicle markings or annotations. This method offers a perspective on road behaviors, allowing for an understanding of potential risks and ultimately enhancing road safety. In our experiment we examined a series of drone images capturing traffic scenarios. We provided a procedure and demonstration of applications of physics and calculus to reconstruct traffic scenes. We constructed a trajectory mechanism that uses all of said concepts to form a concrete plan to reconstruct scenes, using real drone footage with bird’s eye view. The combination of concepts with aerial image analysis showcases the potential for objective and data driven assessments of reckless driving behaviors. Overall this paper serves as a starting point for advancements, in utilizing models to analyze traffic patterns. It contributes to promoting road practices and improving transportation management through evidence based approaches.

Complex Traffic Scene Image Classification Based on Sparse Optimization Boundary Semantics Deep Learning

With the rapid development of intelligent traffic information monitoring technology, accurate identification of vehicles, pedestrians and other objects on the road has become particularly important. Therefore, in order to improve the recognition and classification accuracy of image objects in complex traffic scenes, this paper proposes a segmentation method of semantic redefine segmentation using image boundary region. First, we use the SegNet semantic segmentation model to obtain the rough classification features of the vehicle road object, then use the simple linear iterative clustering (SLIC) algorithm to obtain the over segmented area of the image, which can determine the classification of each pixel in each super pixel area, and then optimize the target segmentation of the boundary and small areas in the vehicle road image. Finally, the edge recovery ability of condition random field (CRF) is used to refine the image boundary. The experimental results show that compared with FCN-8s and SegNet, the pixel accuracy of the proposed algorithm in this paper improves by 2.33% and 0.57%, respectively. And compared with Unet, the algorithm in this paper performs better when dealing with multi-target segmentation.

Predicting perceived risk of traffic scenes using computer vision

Perceived risk, or subjective risk, is an important concept in the field of traffic psychology and automated driving. In this paper, we investigate whether perceived risk in images of traffic scenes can be predicted from computer vision features that may also be used by automated vehicles (AVs). We conducted an international crowdsourcing study with 1378 participants, who rated the perceived risk of 100 randomly selected dashcam images on German roads. The population-level perceived risk was found to be statistically reliable, with a split-half reliability of 0.98. We used linear regression analysis to predict (r = 0.62) perceived risk from two features obtained with the YOLOv4 computer vision algorithm: the number of people in the scene and the mean size of the bounding boxes surrounding other road users. When the ego-vehicle’s speed was added as a predictor variable, the prediction strength increased to r = 0.75. Interestingly, the sign of the speed prediction was negative, indicating that a higher vehicle speed was associated with a lower perceived risk. This finding aligns with the principle of self-explaining roads. Our results suggest that computer-vision features and vehicle speed contribute to an accurate prediction of population subjective risk, outperforming the ratings provided by individual participants (mean r = 0.41). These findings may have implications for AV development and the modeling of psychological constructs in traffic psychology.

Traffic Sign Detection and Classification on the Austrian Highway Traffic Sign Data Set

Advanced Driver Assistance Systems rely on automated traffic sign recognition. Today, Deep Learning methods outperform other approaches in terms of accuracy and processing time; however, they require vast and well-curated data sets for training. In this paper, we present the Austrian Highway Traffic Sign Data Set (ATSD), a comprehensive annotated data set of images of almost all traffic signs on Austrian highways in 2014, and corresponding images of full traffic scenes they are contained in. Altogether, the data set consists of almost 7500 scene images with more than 28,000 detailed annotations of more than 100 distinct traffic sign classes. It covers diverse environments, ranging from urban to rural and mountainous areas, and includes many images recorded in tunnels. We further evaluate state-of-the-art traffic sign detectors and classifiers on ATSD to establish baselines for future experiments. The data set and our baseline models are freely available online.

Research on Traffic Scene Recognition Algorithm Combining Object Detection and Semantic Segmentation

At present, the detection of instances in traffic scenes based on deep learning is mainly divided into two mainstream directions: object detection and semantic segmentation. Among them, object detection realizes the specific location of a single object in the road scene, and semantic segmentation realizes the pixel level classification of objects and background categories in the road scene. However, when pedestrian and other objects have occlusion problems, semantic segmentation is difficult to directly separate a single instance, and the anchor frame generated by object detection contains redundant information. To solve this problem, this paper proposes a method combining target detection and semantic segmentation. This method first uses YOLOv5 model to complete the target detection, and detects people, vehicles and other objects in the captured traffic scene image. At the same time, the improved DeepLabv3+ network model is used to capture the semantic and regional information of roads in the captured traffic scene image. Finally, the prediction results of the output of the two task branches are drawn in the image to be detected, and finally the drawing results are combined and output uniformly. This method can effectively distinguish different people, vehicles, roads and other information in the traffic scene, "complement each other" to understand the driverless road scene, and improve the detection accuracy. The experimental results show that the average mAP of this method is 79.11%, and the segmentation accuracy is high, which is suitable for the unmanned driving scene on real urban roads.

Two-Stage Merging Network for Describing Traffic Scenes in Intelligent Vehicle Driving System

Intelligent vehicle driving systems aim to control the driving behavior of a vehicle in real time without human intervention by perceiving and monitoring the surrounding environment. Describing images of traffic scenes automatically, which is one of the key problems of intelligent vehicle driving technology, has drawn attention since its inception. In recent years, a variety of automatic image description technologies have been proposed, among which the attention-based encoder-decoder framework achieved good results. In this paper we will discuss the fusing of a variety of information from multiple aspects of the images of traffic scenes. First, we will introduce visual attention, text attention and image topics attention which generates the weighted visual features, the attentive text information and the global image topics information respectively. We will then propose an adaptive two-stage merging network based on an encoder-decoder framework, which can fully integrate the three kinds of information in two stages, while automatically calculating the proportions of the information at each time step. Numerous experiments conducted on COCO2014 and Flickr30K datasets have demonstrated the effectiveness and advantages of the proposed method.

EFASPP U-Net for semantic segmentation of night traffic scenes using fusion of visible and thermal images

The development of self-driving cars increases driving safety and accelerates urban transportation. These systems must have robust and real-time understanding of traffic conditions and surroundings, both at day and night. Many semantic image segmentation techniques have been proposed based on deep neural networks to partition the traffic scene images as a substantial step. However, the proposed algorithms and public datasets are mostly based on visible images during the daytime. Also, most of these algorithms are computationally intensive. However, little research has been done to date to address the application of the fusion of thermal and visible images and the high-performance low-volume deep convolutional networks. In this paper, a multispectral Encoder Fused Atrous Spatial Pyramid Pooling (EFASPP) U-Net deep network is proposed to merge the features of the visible and thermal images recorded at night traffic scenes. The proposed network is designed based on the structure of the U-Net, due to its high accuracy and speed of processing, as well as no need for large training datasets. The fusion of visible and thermal features in the encoders of EFASPP U-Net network is performed using standard and atrous convolution layers. Also, a new multispectral dataset is developed in this work for night-time traffic scenes due to the lack of sufficient public dataset in this field. The major contributions of this work include a low-volume high-performance multispectral semantic segmentation network for smart vehicles and a new dataset for this application. The experimental results show the high accuracy and speed of the proposed method.

ALODAD: An Anchor-Free Lightweight Object Detector for Autonomous Driving

Vision-based object detection is an essential component of autonomous driving. Because vehicles typically have limited on-board computing resources, a small-sized detection model is required. Simultaneously, high object detection accuracy and real-time inference detection speeds are required to ensure safety while driving. In this paper, an anchor-free lightweight object detector for autonomous driving called ALODAD is proposed. ALODAD incorporates an attention scheme into the lightweight neural network GhostNet and builds an anchor-free detection framework to achieve lower computational costs and provide parameters with high detection accuracy. Specifically, the lightweight backbone neural network integrates a convolutional block attention model that analyzes the valuable features from traffic scene images to generate an accurate bounding box, and then constructs feature pyramids for multi-scale object detection. The proposed method adds an intersection over union (IoU) branch to the decoupled detector to rank the vast number of candidate detections accurately. To increase the data diversity, data augmentation was used during training. Extensive experiments based on benchmarks demonstrate that the proposed method offers improved performance compared to the baseline. The proposed method can achieve an increased detection accuracy while meeting the real-time requirements of autonomous driving. The proposed method was compared with the YOLOv5 and RetinaNet models and 98.7% and 94.5% were obtained for the average precision metrics AP50 and AP75, respectively, on the BCTSDB dataset.

A Comparison of Bottom-Up Models for Spatial Saliency Predictions in Autonomous Driving.

Bottom-up saliency models identify the salient regions of an image based on features such as color, intensity and orientation. These models are typically used as predictors of human visual behavior and for computer vision tasks. In this paper, we conduct a systematic evaluation of the saliency maps computed with four selected bottom-up models on images of urban and highway traffic scenes. Saliency both over whole images and on object level is investigated and elaborated in terms of the energy and the entropy of the saliency maps. We identify significant differences with respect to the amount, size and shape-complexity of the salient areas computed by different models. Based on these findings, we analyze the likelihood that object instances fall within the salient areas of an image and investigate the agreement between the segments of traffic participants and the saliency maps of the different models. The overall and object-level analysis provides insights on the distinctive features of salient areas identified by different models, which can be used as selection criteria for prospective applications in autonomous driving such as object detection and tracking.

Detection and Classification of Small Traffic Signs Based on Cascade Network

Research on the traffic sign detection is significant for driverless technology, which provides useful navigation information. Existing object detection methods are only applicable to large-size objects or small-scale specific types of traffic signs， and the performance of detecting traffic signs in street views is not adequate. In this regard, we propose a method to detect and classify small traffic signs by constructing a cascaded network. Specifically, the RetinaNet network is adopted firstly to integrate multi-layer information to identify small traffic signs in traffic scene images. The focal loss function is used to balance the biased distribution of traffic sign categories. Then, a two-class network is cascaded after the RetinaNet, which helps identify valid traffic signs from the first-stage prediction results. Experiments show that our cascaded network structure could achieve the balance of different categories of predictions and an improvement in precision and recall.

Social and Non-social Brain Areas in Risk Behaviour: The Role of Social Context

The human brain contains social areas that become active when interacting with another human. These are located in the ventral prefrontal and mediodorsal cortices, adjacent to areas involved in reward processing and cognitive control. Human behaviour is strongly influenced by the social context. This is particularly evident when observing greater risk propensity in the presence of a peer, particularly during adolescence and emerging adulthood. We explored the widely held view that enhanced risk propensity is the consequence of weak cognitive control. We used brain activity, estimated from EEG recordings in a sample of 114 emerging adult dyads whilst performing a risk perception task, to predict risk behaviour in a subsequent driving simulation task. Being with a peer reduced the ability to discriminate riskiness in images of traffic scenes, biased responses towards the perception of no-risk, and increased the rate of accidents in the driving simulation. Risk perception involved three sets of clusters showing activity only when being with a peer, only when being alone, and in both social contexts. Functional connectivity between the clusters accounted for the later driving simulation performance depending on the peer’s presence. In the light of our findings, greater risk-taking, when a peer is present, seems to be triggered by the activation of a different, less efficient brain network for risk-processing.

The Traffic Scene Understanding and Prediction Based on Image Captioning

The traffic scene understanding is the core technology in Intelligent Transportation Systems (ITS) and Advanced Driver Assistance System (ADAS), and it is becoming increasingly important for smart or autonomous vehicles. The recent methods for traffic scene understanding, such as Traffic Sign Recognition (TSR), Pedestrian Detection, and Vehicle Detection, have three major shortcomings. First, most models are customized for recognizing a specific category of traffic target instead of general traffic targets. Second, as for these recognition modules, the task of traffic scene understanding is to recognize objects rather than make driving suggestions or strategies. Third, numerous independent recognition modules disadvantage to fusing multi-modal information to make a comprehensive decision for driving operation in accordance with complicated traffic scenes. In this paper, we first introduce the image captioning model to alleviate the aforementioned shortcomings. Different from existing methods, our primary idea is to accurately identify all categories of traffic objects and understand traffic scenes by making full use of all information, and making the suggestions or strategy for driving operation in natural language by using Long Short Term Memory network (LSTM) rather than keywords. The proposed solution naturally solves the problems of feature fusion, general object recognition, and low-level semantic understanding. We tested the solution on our created traffic scene image dataset for evaluation of image captioning. Extensive experiments including quantitative and qualitative comparisons demonstrate that the proposed solution can identify more objects and produce higher-level semantic information than the state-of-the-arts.

Multi-scale traffic vehicle detection based on faster R–CNN with NAS optimization and feature enrichment

It well known that vehicle detection is an important component of the field of object detection. However, the environment of vehicle detection is particularly sophisticated in practical processes. It is comparatively difficult to detect vehicles of various scales in traffic scene images, because the vehicles partially obscured by green belts, roadblocks or other vehicles, as well as influence of some low illumination weather. In this paper, we present a model based on Faster R–CNN with NAS optimization and feature enrichment to realize the effective detection of multi-scale vehicle targets in traffic scenes. First, we proposed a Retinex-based image adaptive correction algorithm (RIAC) to enhance the traffic images in the dataset to reduce the influence of shadow and illumination, and improve the image quality. Second, in order to improve the feature expression of the backbone network, we conducted Neural Architecture Search (NAS) on the backbone network used for feature extraction of Faster R–CNN to generate the optimal cross-layer connection to extract multi-layer features more effectively. Third, we used the object Feature Enrichment that combines the multi-layer feature information and the context information of the last layer after cross-layer connection to enrich the information of vehicle targets, and improve the robustness of the model for challenging targets such as small scale and severe occlusion. In the implementation of the model, K-means clustering algorithm was used to select the suitable anchor size for our dataset to improve the convergence speed of the model. Our model has been trained and tested on the UN-DETRAC dataset, and the obtained results indicate that our method has art-of-state detection performance.