Complex Traffic Scenes Research Articles

Towards accurate tiny vehicle detection in complex scenes

Large variance of scales is still a notable challenge for vehicle detection in complex traffic scenes. Though Faster RCNN based two-stage detectors have demonstrated superiority on generic object detection, its effectiveness on vehicle detection in real applications is not aware of, especially for vehicles of tiny scales. In this paper, a novel two-stage detector is proposed to perform tiny vehicle detection with high recall. It consists of a backward feature enhancement network (BFEN) and a spatial layout preserving network (SLPN). At the first stage, less attention has been paid for generating proposals with high recall, which, however, plays a significant role in recalling vehicles of tiny scales. By fully exploiting the feature representation power of a backbone network, the BFEN aims at generating high-quality region proposals for vehicles of various scales. Even with only 100 proposals, the proposed BFEN achieves an encouraging recall rate over 99%. For a better localization of tiny vehicles, we argue that the spatial layout of the ROI features also plays a significant role in the second stage. Accordingly, a light-weight detection sub-network called SLPN is designed to progressively integrate ROI features, while preserving the spatial layouts. Experiments done on the challenging DETRAC vehicle detection dataset show that the proposed method significantly improves a competitive baseline (ResNet-50 based Faster RCNN) by 16.5% mAP. Comparative performance to the state of the arts are also achieved on both the DETRAC and KITTI benchmarks.

Brain-Inspired Cognitive Model With Attention for Self-Driving Cars

The perception-driven approach and end-to-end system are two major vision-based frameworks for self-driving cars. However, it is difficult to introduce attention and historical information into the autonomous driving process, which are essential for achieving human-like driving in these two methods. In this paper, we propose a novel model for self-driving cars called the brain-inspired cognitive model with attention. This model comprises three parts: 1) a convolutional neural network for simulating the human visual cortex; 2) a cognitive map to describe the relationships between objects in a complex traffic scene; and 3) a recurrent neural network, which is combined with the real-time updated cognitive map to implement the attention mechanism and long-short term memory. An advantage of our model is that it can accurately solve three tasks simultaneously: 1) detecting the free space and boundaries for the current and adjacent lanes; 2) estimating the distances to obstacles and vehicle attitude; and 3) learning the driving behavior and decision-making process of a human driver. Importantly, the proposed model can accept external navigation instructions during an end-to-end driving process. To evaluate the model, we built a large-scale road-vehicle dataset containing over 40 000 labeled road images captured by three cameras placed on our self-driving car. Moreover, human driving activities and vehicle states were recorded at the same time.

Model Predictive Trajectory Planning for Automated Driving

In order to enable automated driving systems on the road, several key challenges need to be solved. One of these issues is real-time maneuver decision and trajectory planning. This paper introduces a general framework for maneuver and trajectory planning with model predictive methods. It discusses several representations of this framework in distinct complexity levels. In general, sophisticated models require to nonquadratic objective functions with nonlinear constraints, leading to increased computational complexities during calculation. Yet, oversimplified models can neither cope with vehicle dynamics in critical maneuvers nor do they represent complex traffic scenes with several maneuver options appropriately. A scheme to partition the trajectory space into homotopy regions is proposed. In each homotopy class, linearization about a trajectory from this class is applied. Demonstrations by simulation and with experimental vehicles show the capability of the proposed method in selecting optimal maneuvers and trajectories. This is even valid during extreme maneuvers, such as in last moment collision avoidance.

Employing Topical Relations in Semantic Analysis of Traffic Videos

Motion patterns in traffic video can be directly exploited to generate high-level descriptions of video content, which can be used for rule mining and abnormal event detection. The most recent and successful unsupervised methods for complex traffic scene analysis are based on topic models. In this paper, a topic related sparse topical coding framework is proposed for more effectively discovering motion patterns in traffic videos.

Image Caption Description of Traffic Scene Based on Deep Learning

It is a hard issue to describe the complex traffic scene accurately in computer vision. The traffic scene is changeable, which causes image captioning easily interfered by light changes and object occlusion. To solve this problem, we propose an image caption generation model based on attention mechanism. Combining convolutional neural network (CNN) and recurrent neural network (RNN) to generate an end-to-end description for traffic images. To generate a semantic description with distinct degree of discrimination, the attention mechanism is applied to language model. Using Flickr8K、Flickr30K and MS COCO benchmark datasets to validate the effectiveness of our method. The accuracy is promoted maximally by 8.6%, 12.4%, 19.3% and 21.5% in different evaluation metrics. Experiments show that our algorithm has good robustness in four different complex traffic scenarios, such as light change, abnormal weather environment, road marked target and various kinds of transportation tools.

Understanding Pedestrian Behavior in Complex Traffic Scenes

Designing autonomous vehicles for urban environments remains an unresolved problem. One major dilemma faced by autonomous cars is understanding the intention of other road users and communicating with them. To investigate one aspect of this, specifically pedestrian crossing behavior, we have collected a large dataset of pedestrian samples at crosswalks under various conditions (e.g., weather) and in different types of roads. Using the data, we analyzed pedestrian behavior from two different perspectives: the way they communicate with drivers prior to crossing and the factors that influence their behavior. Our study shows that changes in head orientation in the form of looking or glancing at the traffic is a strong indicator of crossing intention. We also found that context in the form of the properties of a crosswalk (e.g., its width), traffic dynamics (e.g., speed of the vehicles) as well as pedestrian demographics can alter pedestrian behavior after the initial intention of crossing has been displayed. Our findings suggest that the contextual elements can be interrelated, meaning that the presence of one factor may increase/decrease the influence of other factors. Overall, our work formulates the problem of pedestrian-driver interaction and sheds light on its complexity in typical traffic scenarios.

Real‐time vehicle detection and counting in complex traffic scenes using background subtraction model with low‐rank decomposition

Real-time vehicle counting can efficiently improve traffic control and management. Aiming to efficiently collect the real-time traffic information, the authors propose an effective vehicle counting system for detecting and tracking vehicles in complex traffic scenes. The proposed algorithm detects moving vehicles based on background subtraction method with ‘low-rank + sparse’ decomposition. For accurately counting vehicles, an online Kalman filter algorithm is used to track the multiple moving objects and avoid counting one vehicle repeatedly. The proposed method is evaluated on three publicly available datasets, which include seven video sequences with various challenging scenes for detection performance evaluation, and another two video sequences for vehicle counting evaluation. The experimental results demonstrate a good performance of the proposed method in terms of both qualitative and quantitative evaluations.

Reliable vehicle type recognition based on information fusion in multiple sensor networks

With the purpose to improve the real-time performance and accuracy of vehicle type recognition in complex traffic scenes, this paper proposed an information fusion based vehicle type recognition method. Firstly, the multiple sensor network built of geomagnetic sensors is described, whose characteristics include easy deployment and being budget friendly. Then, the vehicle type recognition method for the proposed multiple sensor network is applied. The main innovation of the method is matching the data collected by multiple sensors to form a fusion feature waveform for a single vehicle. Finally, the experimental results showed that the proposed method had 95% accuracy for small vehicles (Passenger vehicles and Sport utility vehicles) and 90% accuracy for large vehicles (Buses and Vans) approximately, which can provide a reference for multiple sensor network design and traffic data collection.

Development of a tracking-based system for automated traffic data collection for roundabouts

Traffic data collection is essential for performance assessment, safety improvement and road planning. While automated traffic data collection for highways is relatively mature, that for roundabouts is more challenging due to more complex traffic scenes, data specifications and vehicle behavior. In this paper, the authors propose an automated traffic data collection system dedicated to roundabout scenes. The proposed system has mainly four steps of processing. First, camera calibration is performed for roundabout traffic scenes with a novel circle-based calibration algorithm. Second, the system uses enhanced Mixture of Gaussian algorithm with shaking removal for video segmentation, which can tolerate repeated camera displacements and background movements. Then, Kalman filtering, Kernel-based tracking and overlap-based optimization are employed to track vehicles while they are occluded and to derive the complete vehicle trajectories. The resulting vehicle trajectory of each individual vehicle gives the position, size, shape and speed of the vehicle at each time moment. Finally, a data mining algorithm is used to automatically extract the interested traffic data from the vehicle trajectories. The overall traffic data collection system has been implemented in software and runs on regular PC. The total processing time for a 3-hour video is currently 6 h. The automated traffic data collection system can significantly reduce cost and improve efficiency compared to manual data collection. The extracted traffic data have been compared to accurate manual measurements for 29 videos recorded on 29 different days, and an accuracy of more than 90% has been achieved.

Hierarchical and Networked Vehicle Surveillance in ITS: A Survey

Traffic surveillance has become an important topic in intelligent transportation systems (ITSs), which is aimed at monitoring and managing traffic flow. With the progress in computer vision, video-based surveillance systems have made great advances on traffic surveillance in ITSs. However, the performance of most existing surveillance systems is susceptible to challenging complex traffic scenes (e.g., object occlusion, pose variation, and cluttered background). Moreover, existing related research is mainly on a single video sensor node, which is incapable of addressing the surveillance of traffic road networks. Accordingly, we present a review of the literature on the video-based vehicle surveillance systems in ITSs. We analyze the existing challenges in video-based surveillance systems for the vehicle and present a general architecture for video surveillance systems, i.e., the hierarchical and networked vehicle surveillance, to survey the different existing and potential techniques. Then, different methods are reviewed and discussed with respect to each module. Applications and future developments are discussed to provide future needs of ITS services.

Max-covering scheme for gesture recognition of Chinese traffic police

We present a method to recognize gestures made by Chinese traffic police in complex scenes based on a max-covering scheme for driver assistance systems and intelligent vehicles. Gesture recognition is made possible by upper-body-part detection with a five-part body model. First, the police's torso and arms are extracted from a complex traffic scene as the foreground region by using dark channel prior and kernel density estimation. Then the coordinates of pixels in the upper arms and forearms are determined using the proposed max-covering scheme, which is based on a key observation that body-part tiles maximally cover the foreground region and satisfy a body plan. Finally, the rotation joint angle or Gabor feature-based two-dimensional principal component analysis is used to recognize the gestures made by Chinese traffic police. A comparative study is proposed with other human pose estimation methods, which demonstrates that better recognition results can be obtained using the proposed method on a number of video sequences.

Does driving experience in video games count? Hazard anticipation and visual exploration of male gamers as function of driving experience

Risk perception and distribution of visual attention while driving are crucial elements for accident prevention and new-driver improvement. This study investigates how racing videogames could shape the visual exploration of virtual and real road in male pre-drivers. The visual performance of players of racing video games with and without driver’s license was tested in virtual vs. real scenarios. Attention to specific elements of different types of road interactions was monitored using an eye-tracking system. Results showed that habitual use of racing video games was not found to foster a positive effect on users’ distribution of visual attention, supporting visual patterns typical of novice drivers. Gamers without driving experience replicated the same patterns in a real road scenario, ignoring road signs and potential areas of interactions with other drivers, while experienced drivers gamers explored video games roads like real roads. The fact that the gamers’ driving performance was not comparable to drivers in the virtual scenario suggests that there are other variables in the gameplay that create a less complex traffic scene, still the visual complexity of different real road interactions is kept in video game interactions, opening new perspectives towards gamers’ visual exploration of the road.

Contour Extraction Based on Improved Snake Model and its Application in Vehicle Identification

Contour curve is an important shape feature for vehicle recognition and it is a hard work to extraction it from complex dynamic traffic video for in-vehicle detection system. Snake Model is used to automatically extract the object contour curve proposed by Kass et al, but it is inability for traffic objects. Presented here is a novel approach for extracting vehicle contour curve by combining stereo vision with Snake Model. In this paper, Stereo vision is first used to segment vehicle from traffic background, then Snake Model is adopted to obtain complete contour curve. In view of classical Snake model is easily affected by noise, here we propose a improved Snake model by combining corner detection technology with Distance Potential Snake Model. Moreover, a vehicle identification method based on contour curve is presented. The method presented here was tested on complex traffic scenes and the corresponding results prove the efficiency of our proposed method.

Automatic multi-vehicle tracking using video cameras: An improved CAMShift approach

Video cameras play an important role in achieving the potentials promised by Intelligent Transportation Systems. In particular, video tracking systems have been widely applied in traffic flow data collection and incident detection by tracking and analyzing vehicle trajectories through video image processing. Focusing on current vehicle tracking methods that are limited to complex traffic scenes such as target non-uniqueness and color distraction in real traffic situations, this paper proposes an improved Continuously Adaptive Mean Shift (CAMShift) method for automatic multi-vehicle tracking using video cameras. The proposed method firstly uses a background subtraction method for automatically detecting, selecting, and initializing targets, i.e., vehicles. Moreover, a motion estimation method is proposed for estimating vehicle motion states and predicting the new locations of the vehicles. Finally, both the color and edge feature distributions of the vehicles are extracted and a mixed model is established for searching the matching vehicles in the next image frame. The proposed method was comparatively evaluated with the traditional CAMShift method using the same video sequences captured from real traffic situations. Evaluation results showed that the proposed method is efficient for accurately tracking vehicle movements, and is promising for practical applications.

Counting Vehicles from Semantic Regions

Automatically counting vehicles in complex traffic scenes from videos is challenging. Detection and tracking algorithms may fail due to occlusions, scene clutters, and large variations of viewpoints and vehicle types. We propose a new approach of counting vehicles through exploiting contextual regularities from scene structures. It breaks the problem into simpler problems, which count vehicles on each path separately. The model of each path and its source and sink add strong regularization on the motion and the sizes of vehicles and can thus significantly improve the accuracy of vehicle counting. Our approach is based on tracking and clustering feature points and can be summarized in threefold. First, an algorithm is proposed to automatically learn the models of scene structures. A traffic scene is segmented into local semantic regions by exploiting the temporal cooccurrence of local motions. Local semantic regions are connected into global complete paths using the proposed fast marching algorithm. Sources and sinks are estimated from the models of semantic regions. Second, an algorithm is proposed to cluster trajectories of feature points into objects and to estimate average vehicle sizes at different locations from initial clustering results. Third, trajectories of features points are often fragmented due to occlusions. By integrating the spatiotemporal features of trajectory clusters with contextual models of paths and sources and sinks, trajectory clusters are assigned into different paths and connected into complete trajectories. Experimental results on a complex traffic scene show the effectiveness of our approach.

Measuring Driving Behaviors from Live Video

Because of the technical and cost constraints on traditional measurement methods, there is a lack of long-term driving behavior data from natural traffic scenes, and this situation has been hindering research progress into driving behavior modeling and other related topics. Thanks to high-definition cameras and advanced visual measurement methods, traffic visual detection is entering a new stage of traffic visual measurement, and thus we can expect to achieve accurate segmentation, positioning, and measurement for road vehicles from live video to meet the requirement for field test data in behavior modeling. To measure driving behaviors in a cost-effective manner, the authors propose a comprehensive visual measurement approach that could perform well in complex traffic scenes. Specifically, they describe a procedure for traffic visual measurement, some preliminary algorithms, and some representative experimental results. Comparisons between the proposed method and three traditional ones (driving simulator, in-vehicle data recorder, and remote-sensing camera) indicate that the biggest advantage of the proposed method is it can measure driving behaviors from live video. Hence, the ongoing research will greatly benefit cognition in driving behavior models.

One-scan rule extraction to explain significant vehicle interactions with guaranteed error value

Counting frequent itemsets allows us to compute the importance of items over a stream of data. Translating this concept to video streams imposes the need of representing activities as a sequence of activities over a video stream. In this paper, we present a model to find approximate co-occurring associations between activities from video stream data considering an unsupervised clustering of activities. We show that a hierarchical Topic Model of two stochastic processes is needed to jointly learn both an unknown number of activities in the video and the visual features that positively correlate for each activity. Unlike most of previous works, we decouple the analysis of associations between multiple moving objects from the discovery of activities. While the discovery of activities is an off-line process in which event distributions are grouped, the discovery of rules is an on-line process that approximates the importance of each rule with guaranteed error value. Our method reduces space complexity by adapting the algorithm to the amount of memory available before any process to update frequency values for itemsets is incrementally performed. The most visible aspect of this effort is the incremental generation of rules that discover the interaction of frequent activities for current scenes. Our experimental results show that our approach efficiently and automatically discovers sets of activities in a video stream coming from surveillance videos containing complex traffic scenes governed by multiple semaphores, while evaluating their frequent occurrence and co-occurring relationships.

Multi-Vehicle Tracking Based Traffic Illegal Parking Detection

Vehicle detection and tracking are the foundations of traffic violation punishment. The current methods have more parameters to be adjusted and can not adapt to the complex traffic scenes. For the purpose of practice application, we propose an illegal parking detection algorithm based on vehicle tracking theory. In order to filter the dynamic interferences such as vehicles parking for a long time, moving vehicles overlapping, and the vehicle losing momentarily from the video, a Gaussian mixture model is combined with the correlations among adjacent pixels to update background and eliminate noises. In allusion to complex traffic scenes, a fast multi-object tracking algorithm is presented to track occluded vehicles effectively. This method predicts vehicle trajectory by Kalman filter and achieves real-time object tracking state judgment by using the feature matching matrix. The experimental results show that this method can track multiple moving vehicles effectively and has high detection precision for illegal parking incidents.

Object-Oriented Bayesian Networks for Detection of Lane Change Maneuvers

This article introduces a novel approach towards the recognition of typical driving maneuvers in structured highway scenarios and shows some key benefits of traffic scene modeling with object-oriented Bayesian networks (OOBNs). The approach exploits the advantages of an introduced lane-related coordinate system together with individual occupancy schedule grids for all modeled vehicles. This combination allows an efficient classification of the existing vehicle-lane and vehicle- vehicle relations in traffic scenes and thus substantially improves the understanding of complex traffic scenes. Probabilities and variances within the network are propagated systematically which results in probabilistic sets of the modeled driving maneuvers. Using this generic approach, the network is able to classify a total of 27 driving maneuvers including merging and object following.

Vehicle detection algorithm based on light pairing and tracking at nighttime

Detecting a vehicle to obtain traffic information at nighttime is difficult. This study proposes a vehicle detection algorithm, called the headlight extraction, pairing, and tracking (HLEPT) algorithm, which can acquire traffic information in the rain at nighttime by identifying vehicles through the location of their headlights and other indicative lights. A knowledge-based connected-component procedure, in which vehicles are located by grouping their lights and estimating their features, is proposed. The features of a complex nighttime traffic scene were also analyzed. The HLEPT algorithm includes a headlight extraction algorithm, as well as regulations for the pairing and grouping of lights and light tracking using a Kanade-Lucas-Tomasi tracker to measure traffic flow and velocity. Experimental results demonstrate the feasibility and effectiveness of the proposed approach on vehicle detection in the rain at nighttime.