Traffic Video Sequences Research Articles

V-LPDR: Towards a unified framework for license plate detection, tracking, and recognition in real-world traffic videos

License plate detection and recognition (LPDR) has attracted considerable attention in recent years, and many algorithms have presented the competitive performance on several datasets. However, there are still three significant issues to be addressed in this field. Firstly, most methods have poor detection performance in unconstrained scenarios with moving vehicles and highly distracting background objects. Secondly, existing systems generally focus on single image-based algorithms, yet traffic video sequences provide more effective information than individual frames for LPDR tasks. Thirdly, images and videos captured in complex environments may be adversely affected by distortions and low resolution, causing sensitive recognition performance and reduced robustness. To remedy these issues, we propose to automatically perform license plate detection, tracking, and recognition in real-world traffic videos and integrate them into a unified end-to-end framework via deep learning. The contributions of this paper are threefold: 1) A deep flow-guided spatiotemporal license plate detector is proposed to model the video contextual information by introducing optical flow and a novel spatiotemporal attention mechanism; 2) An online license plate tracker is developed to bridge video-based detection and recognition which utilizes both motion and deep appearance information, and innovatively, it can be end-to-end trained with the detector via multi-task learning; 3) The efficient quality-guided license plate recommender and recognizer are proposed to jointly perform stream recognition. The former recommends high-quality frames from video streams while the latter generates recognition results. We evaluate the proposed method on three traffic video-based license plate datasets, and ablation studies have been presented to verify the effectiveness of each component mentioned above. Moreover, extensive experiments are conducted for comparison with other approaches in different scenarios, and the results have demonstrated that our method achieves state-of-the-art performance on all datasets.

Pixel matching search algorithm for counting moving vehicle in highway traffic videos

Traffic monitoring through video processing is one of the hot research areas in the Intelligent Transportation System (ITS). Vehicle counting systems should be simple enough to be applied in real-time circumstances. A novel and fast algorithm for vehicle counting from a traffic video sequence is proposed in this paper where the vehicle tracking step is not necessary. A reference model is only created in the video frames for a narrow area. When going through this narrow area, the moving vehicles are identified as foreground objects. Detection of moving vehicles is achieved by integrating approximated median filter based background subtraction with binary integral projection. The detected candidates are counted as a vehicle using a novel pixel matching search algorithm. The proposed algorithm does not rely on every video frame. It only requires every third frame for processing and thus increases the computation speed by three times compared to existing techniques. The proposed algorithm is tested and validated on a standard data set as well as a custom data set. Two parameters such as accuracy and processing time are used for the system evaluation where an overall accuracy of 96.84% is achieved. The processing time results show that the proposed system can perform in real-time with an average real-time processing speed of 93.92%.

A new cast shadow detection method for traffic surveillance video analysis using color and statistical modeling

In traffic surveillance video analysis systems, the cast shadows of vehicles often have a negative effect on video analysis results. A novel cast shadow detection framework, which consists of a new foreground detection method and a cast shadow detection method, is presented in this paper to detect and remove the cast shadows from the foreground. The new foreground detection method applies an innovative Global Foreground Modeling (GFM) method, a Gaussian mixture model or GMM, and the Bayes classifier for foreground and background classification. While the GFM method is for global foreground modeling, the GMM is for local background modeling, and the Bayes classifier applies both the foreground and the background models for foreground detection. The rationale of the GFM method stems from the observation that the foreground objects often appear in recent frames and their trajectories often lead them to different locations in these frames. As a result, the statistical models used to characterize the foreground objects should not be pixel based or locally defined. The cast shadow detection method contains four hierarchical steps. First, a set of new chromatic criteria is presented to detect the candidate shadow pixels in the HSV color space. Second, a new shadow region detection method is proposed to cluster the candidate shadow pixels into shadow regions. Third, a statistical shadow model, which uses a single Gaussian distribution to model the shadow class, is presented for classifying shadow pixels. Fourth, an aggregated shadow detection method is presented for final shadow detection. Experiments using the public video data ‘Highway-1’ and ‘Highway-3’, and the New Jersey Department of Transportation (NJDOT) real traffic video sequences show the feasibility of the proposed method. In particular, the proposed method achieves better shadow detection performance than the popular shadow detection methods, and is able to improve the traffic video analysis results.

Automatic vehicle detection and counting approach using low-rank representation and locality-constrained linear coding

PurposeVehicles estimation can be used in evaluating traffic conditions and facilitating traffic control, which is an important task in intelligent transportation system. The paper aims to propose a vehicle-counting method based on the analysis of surveillance videos.Design/methodology/approachThe paper proposes a novel two-step method using low-rank representation (LRR) detection and locality-constrained linear coding (LLC) classification to count the number of vehicles in traffic video sequences automatically. The proposed method is based on an offline training to understand an LLC-based classifier with extracted features for vehicle and pedestrian classification, followed by an online counting algorithm to count the number of vehicles detected from the image sequence.FindingsThe proposed method allows delivery estimation (counting the number of vehicles at each frame only) and total number estimation of vehicles shown in the scene. The paper compares the proposed method with other similar methods on three public data sets. The experimental results show that the proposed method is competitive and effective in terms of computational speed and evaluation accuracy.Research limitations/implicationsThe proposed method does not consider illumination. Hence, the results might be unsatisfactory under low-lighting condition. Therefore, researchers are encouraged to add a term that controls the illumination changes into the energy function of vehicle detection in future work.Originality/valueThe paper bridges the gap between LRR detection and vehicle counting by taking advantage of existing LLC classification algorithm to distinguish different moving objects.

Real-time framework for image dehazing based on linear transmission and constant-time airlight estimation

The haze phenomenon exerts a degrading effect that decreases contrast and causes colour shifts in outdoor images. The presence of haze in digital images is bothersome, unpleasant, and, occasionally, even dangerous. The atmospheric light scattering (ALS) model is widely used to restore hazy images. In this model, two unknown parameters should be estimated: airlight and scene transmission. The quality of dehazed images depends considerably on the accuracy of both estimates. Classic methods typically determine airlight based on the brightest pixels in an image. However, in the traffic scene context, this estimate is compromised when other light sources, such as vehicle headlights from the opposite direction, are present. Transmission estimation is usually more complicated. Hence, the complexity of the overall dehazing process is dependent on this estimate. To address this issue, this study proposes a framework for constant-time airlight and linear transmission estimation. This framework consists of two methods: airlight by image integrals (ALII), which is utilized to estimate the airlight value in real time with high accuracy, and bounded transmission (BT), which is proposed for the linear and simplified estimation of transmission maps. To evaluate the proposed framework, three image datasets are used: (1) seven images that are gathered from the works of existing methods (called the global dataset); (2) the synthetic foggy road image database (FRIDA), which is a synthetically generated dataset for simulating different bad weather conditions; and (3) a dataset of images that were extracted from videos in Malaysia (IV-M), which consists of images that were extracted from traffic video sequences, which were captured in various weather conditions from 2014 to 2016 in Malaysia. Experimental results show that the proposed framework is at least seven times faster than existing methods. In addition, the ANOVA test proves that the quality of the dehazed images is statistically similar to or better than the image quality that was achieved using existing methods.

Vision Based Vehicle Detection Using Hybrid Algorithm

Moving vehicle detection remain very critical and thus intended for Video-based solution, comparing to other techniques and by considering the traffic video sequence recorded from a video camera, this paper presents a video-based solution applied with adaptive subtracted background technology in combination with virtual detector and blob tracking technologies. This paper provides Experimental results moving vehicle detection which is implemented in Visual C++ code with OpenCV, thus the proposed method used for detection.

A Growing Neural Gas Approach to Classify Vehicles in Traffic Environments

Automated video surveillance presents a great amount of applications and one of them is traffic monitoring. Vehicle type detection can provide information about the characteristics of the traffic flow to human traffic controllers in order to facilitate their decision-making process. A video surveillance system is proposed in this work to execute such classification. First of all, a foreground detection and tracking object process has been carried out. Once the vehicles are detected, a feature extraction method obtains the most significant features of this detected vehicles. When the extraction process is done, the vehicle types are determined by employing a set of Growing Neural Gas neural networks. The performance of the proposal has been analyzed from a qualitative and quantitative point of view by using a set of benchmark traffic video sequences, with acceptable results.

A Quick Extraction Algorithm Using Window-Scanning for Moving Vehicles

Aiming at the particularity of traffic monitoring video sequences and the regularity of vehicle movement, a quick extraction algorithm using window-scanning for moving vehicles in traffic monitoring videos is proposed in this paper. This algorithm uses hypothesis testing to higher order statistics of frame differences to achieve the rough separation of moving vehicles and background. Then obtain the length of the vehicle and extract the vertical coordinates of the initial point of moving vehicle by setting a static window with a stationary location, combining with the velocity of the vehicle and the moving pixel distribution probability in the window. And obtain the width of the vehicle the horizontal coordinates of the initial point of moving vehicle by setting a dynamic window, combining with the distribution probability of moving pixels in the window. Finally this algorithm achieved the quick extraction of vehicles with the window obtained by length and width, combining with the coordinates of the initial point of moving vehicle. Experiments show the feasibility of the algorithm that the time and space efficiency is relatively high.

Shape Features of Overlapping Boundary for Classification of Moving Vehicles

In this paper, the authors propose a model for classification of moving vehicles in traffic videos. A corner-based tracking method is presented to track and detect moving vehicles. The authors propose to overlap the boundary curves of each of the detected moving vehicles while tracking in a sequence of frames to reconstruct a complete boundary shape of the vehicle. The reconstructed boundary shape is normalized and then shape features are extracted. Vehicles are categorized into 4 different types of vehicle classes using KNN rule, the weighted KNN, PNN, and SVM classifiers. Experiments are conducted on traffic video sequences captured in an uncontrolled environment during daytime.

Trajectory clustering in CCTV traffic videos using probability product kernels with hidden Markov models

This article presents a method for clustering the trajectories obtained by tracking vehicles in traffic videos, recorded from CCTV cameras in public spaces. The proposed method employs a model-based approach in which (1) each trajectory (position and velocity) is modelled using a hidden Markov model (HMM), and (2) the distance between two trajectories is computed as the probabilistic similarity between HMMs, by means of the probability product kernel. Experiments on a set of real traffic video sequences reveal very good results of the proposed approach, outperforming two non-trivial baselines. To the best of the authors' knowledge, this approach is novel for trajectory grouping in traffic videos.

Image Segmentation in Video Sequences Using Modified Background Subtraction

In computer vision, Background is a technique for finding moving objects in a video sequences for example vehicle driving on a freeway. For to detect non stationary (dynamic) objects, it is necessary to subtracting current image from a time-averaged background image. There are various background subtraction algorithms for detecting moving vehicles or any moving object(s) like pedestrians in urban traffic video sequences. A crude approximation to the task of classifying each pixel on the frame of current image, locate slow-moving objects or in poor image qualities of videos and distinguish shadows from moving objects by using modified background subtraction method. While classifying each pixel on the frame of the current image, it is to be detect the moving object at foreground and background conditional environment that we can classify each pixel using a model of how that pixel looks when it is part of video frame classes. A mixture of Gaussians classification model for each pixel using an unsupervised technique is an efficient, incremental version of Expectation Maximization (EM) is used for the purpose. Unlike standard image-averaging approach, this method automatically updates the mixture component for each video frame class according to likelihood of membership; hence slowmoving objects and poor image quality of videos are also being handled perfectly. Our approach identifies and eliminates shadows much more effectively than other techniques like thresholding.

Length-based vehicle classification in multi-lane traffic flow

For the realtime classification of moving vehicles in the multi-lane traffic video sequences, a length-based method is proposed. To extract the moving regions of interest, the difference image between the updated background and current frame is obtained by using background subtraction, and then an edge-based shadow removal algorithm is implemented. Moreover, a thresholding segmentation method for the region detection of moving vehicle based on location search is developed. At the estimation stage, a registration line is set up in the detection area, then the vehicle length is estimated with the horizontal projection technique as soon as the vehicle leaves the registration line. Lastly, the vehicle is classified according to its length and the classification threshold. The proposed method is different from traditional methods that require complex camera calibrations. It calculates the pixel-based vehicle length by using uncalibrated traffic video sequences at lower computational cost. Furthermore, only one registration line is set up, which has high flexibility. Experimental results of three traffic video sequences show that the classification accuracies for the large and small vehicles are 97.1% and 96.7% respectively, which demonstrates the effectiveness of the proposed method.

Learning sequences of sparse correlated patterns using small-world attractor neural networks: An application to traffic videos

The goal of this work is to learn and retrieve a sequence of highly correlated patterns using a Hopfield-type of attractor neural network (ANN) with a small-world connectivity distribution. For this model, we propose a weight learning heuristic which combines the pseudo-inverse approach with a row-shifting schema. The influence of the ratio of random connectivity on retrieval quality and learning time has been studied. Our approach has been successfully tested on a complex pattern, as it is the case of traffic video sequences, for different combinations of the involved parameters. Moreover, it has demonstrated to be robust with respect to highly variable frame activity.

Estimation of Dynamic Background and Object Detection in Noisy Visual Surveillance

Dynamic background subtraction in noisy environment for detecting object is a challenging process in computer vision. The proposed algorithm has been used to identify moving objects from the sequence of video frames which contains dynamically changing backgrounds in the noisy atmosphere. There are many challenges in achieving a robust background subtraction algorithm in the external noisy environment. In connection with our previous work, in this paper, we have proposed a methodology to perform background subtraction from moving vehicles in traffic video sequences that combines statistical assumptions of moving objects using the previous frames in the dynamically varying noisy situation. Background image is frequently updated in order to achieve reliability of the motion detection. For that, a binary moving objects hypothesis mask is constructed to classify any group of lattices as being from a moving object based on the optimal threshold. Then, the new incoming information is integrated into the current background image using a Kalman filter. In order to improve the performance, a post-processing has been done. It has been accomplished by shadow and noise removal algorithms operating at the lattice which identifies object-level elements. The results of post-processing can be used to detect object more efficiently. Experimental results and analysis show the prominence of the proposed approach which has achieved an average of 94% accuracy in real-time acquired images.

Approximate reasoning and finite state machines to the detection of actions in video sequences

In this paper a novel approach for recognizing actions in video sequences is presented, where the information obtained from the segmentation and tracking algorithms is used as input data. First of all, the fuzzification of input data is done and this process allows to successfully manage the uncertainty inherent to the information obtained from low-level and medium-level vision tasks, to unify the information obtained from different vision algorithms into a homogeneous representation and to aggregate the characteristics of the analyzed scenario and the objects in motion. Another contribution is the novelty of representing actions by means of an automaton and the generation of input symbols for the finite automaton depending on the comparison process between objects and actions, i.e., the main reasoning process is based on the operation of automata with capability to manage fuzzy representations of all video data. The experiments on several real traffic video sequences demonstrate encouraging results, especially when no training algorithms to obtain predefined actions to be identified are required.

Robust background subtraction in traffic video sequence

For intelligent transportation surveillance, a novel background model based on Marr wavelet kernel and a background subtraction technique based on binary discrete wavelet transforms were introduced. The background model kept a sample of intensity values for each pixel in the image and used this sample to estimate the probability density function of the pixel intensity. The density function was estimated using a new Marr wavelet kernel density estimation technique. Since this approach was quite general, the model could approximate any distribution for the pixel intensity without any assumptions about the underlying distribution shape. The background and current frame were transformed in the binary discrete wavelet domain, and background subtraction was performed in each sub-band. After obtaining the foreground, shadow was eliminated by an edge detection method. Experimental results show that the proposed method produces good results with much lower computational complexity and effectively extracts the moving objects with accuracy ratio higher than 90%, indicating that the proposed method is an effective algorithm for intelligent transportation system.

Time efficient ghost removal for motion detection in visual surveillance systems

The problem of 'ghosts' that appear on foreground/background separation methods is investigated. The contribution provided is a fast and effective algorithm for ghost detection and removal, that can be integrated to any motion detection system that estimates a difference map between the current frame and a background model. The performance of the ghost removal algorithm is assessed using a series of urban traffic video sequences that would normally cause ghost problems.

Learning-based spatio-temporal vehicle tracking and indexing for transportation multimedia database systems

One key technology of intelligent transportation systems is the use of advanced sensor systems for on-line surveillance to gather detailed information on traffic conditions. Traffic video analysis can provide a wide range of useful information to traffic planners. In this context, the object-level indexing of video data can enable vehicle classification, traffic flow analysis, incident detection and analysis at intersections, vehicle tracking for traffic operations, and update of design warrants. In this paper, a learning-based automatic framework is proposed to support the multimedia data indexing and querying of spatio-temporal relationships of vehicle objects in a traffic video sequence. The spatio-temporal relationships of vehicle objects are captured via the proposed unsupervised image/video segmentation method and object tracking algorithm, and modeled using a multimedia augmented transition network model and multimedia input strings. An efficient and effective background learning and subtraction technique is employed to eliminate the complex background details in the traffic video frames. It substantially enhances the efficiency of the segmentation process and the accuracy of the segmentation results to enable more accurate video indexing and annotation. The paper uses four real-life traffic video sequences from several road intersections under different weather conditions in the study experiments. The results show that the proposed framework is effective in automating data collection and access for complex traffic situations.

A Multimedia Data Mining Framework: Mining Information from Traffic Video Sequences

The analysis and mining of traffic video sequences to discover important but previously unknown knowledge such as vehicle identification, traffic flow, queue detection, incident detection, and the spatio-temporal relations of the vehicles at intersections, provide an economic approach for daily traffic monitoring operations. To meet such demands, a multimedia data mining framework is proposed in this paper. The proposed multimedia data mining framework analyzes the traffic video sequences using background subtraction, image/video segmentation, vehicle tracking, and modeling with the multimedia augmented transition network (MATN) model and multimedia input strings, in the domain of traffic monitoring over traffic intersections. The spatio-temporal relationships of the vehicle objects in each frame are discovered and accurately captured and modeled. Such an additional level of sophistication enabled by the proposed multimedia data mining framework in terms of spatio-temporal tracking generates a capability for automation. This capability alone can significantly influence and enhance current data processing and implementation strategies for several problems vis-a-vis traffic operations. Three real-life traffic video sequences obtained from different sources and with different weather conditions are used to illustrate the effectiveness and robustness of the proposed multimedia data mining framework by demonstrating how the proposed framework can be applied to traffic applications to answer the spatio-temporal queries.

Combination of Edge Element and Optical Flow Estimates for 3D-Model-Based Vehicle Tracking in Traffic Image Sequences

A model-based vehicle tracking system for the evaluation of inner-city traffic video sequences has been systematically tested on about 15 minutes of real world video data. Methodological improvements during preparatory test phases affected—among other changes—the combination of edge element and optical flow estimates in the measurement process and a more consequent exploitation of background knowledge. The explication of this knowledge in the form of models facilitates the evaluation of video data for different scenes by exchanging the scene-dependent models. An extensive series of experiments with a large test sample demonstrates that the current version of our system appears to have reached a relative optimum: further interactive tuning of tracking parameters does no longer promise to improve the overall system performance significantly. Even the incorporation of further knowledge regarding vehicle and scene geometry or illumination has to cope with an increasing level of interaction between different knowledge sources and system parameters. Our results indicate that model-based tracking of rigid objects in monocular image sequences may have to be reappraised more thoroughly than anticipated during the recent past.