Traffic Surveillance Applications Research Articles

Real-Time Vehicle Accident Recognition from Traffic Video Surveillance using YOLOV8 and OpenCV

The automatic detection of traffic accidents is a significant topic in traffic monitoring systems. It can reduce irresponsible driving behavior, improve emergency response, improve traffic management, and encourage safer driving practices. Computer vision can be a promising technique for automatic accident detection because it provides a reliable, automated, and speedy accident detection system that can improve emergency response times and ultimately save lives. This paper proposed an ensemble model that uses the YOLOv8 approach for efficient and precise event detection. The model framework's robustness is evaluated using YouTube video sequences with various lighting circumstances. The proposed model has been trained using the open-source dataset Crash Car Detection Dataset, and its produced precision, recall, and mAP are 93.8% and 98%, 96.1%, respectively, which is a significant improvement above the prior precision, recall, and mAP figures of 91.3%, 87.6%, and 93.8%. The effectiveness of the proposed approach in real-time traffic surveillance applications is proved by experimental results using actual traffic video data.

An efficient automated vehicle license plate recognition system under image processing

<span lang="EN-US">An automated vehicle license plate recognition system using image processing techniques identifies vehicle numbers without human interference. This system has significant impact because of its good application in various fields like car parking, access control, speed control, automatic toll collection, border security, traffic violence detection and surveillance applications. This paper presents a methodology that is quite simple but at the same time very much efficient and this system consists of four sequential modules which are preprocessing, number plate extraction, number plate character segmentation and character recognition. Preprocessing aims to improve the image quality that is captured in various illumination conditions and stick out outstanding information that we need, which is favorable to subsequent processing including extraction, segmentation and recognition. After preprocessing various morphological operations are applied to extract the desired license plate region. Then for segmentation the bounding box method is applied that segments each letter and number present on the license plate region. Finally, template matching is applied in identifying all segmented characters present in the license plate image. The experimental results showed that the proposed system can recognize license plate characters efficiently with higher accuracy. Using MATLAB software, the proposed method attains recognition accuracy of 94.17%.</span>

SRODNet: Object Detection Network Based on Super Resolution for Autonomous Vehicles

Object detection methods have been applied in several aerial and traffic surveillance applications. However, object detection accuracy decreases in low-resolution (LR) images owing to feature loss. To address this problem, we propose a single network, SRODNet, that incorporates both super-resolution (SR) and object detection (OD). First, a modified residual block (MRB) is proposed in the SR to recover the feature information of LR images, and this network was jointly optimized with YOLOv5 to benefit from hierarchical features for small object detection. Moreover, the proposed model focuses on minimizing the computational cost of network optimization. We evaluated the proposed model using standard datasets such as VEDAI-VISIBLE, VEDAI-IR, DOTA, and Korean highway traffic (KoHT), both quantitatively and qualitatively. The experimental results show that the proposed method improves the accuracy of vehicular detection better than other conventional methods.

Stereo camera pair calibration for traffic surveillance applications

We proposed an innovative stereo camera pair calibration method suitable for traffic surveillance applications. In the method, we first split the sought parameters into two groups. The first group contains parameters that can be computed prior to the device being installed in the desired location, and we determine their values using already established methods. The second group contains parameters that need to be evaluated after the device is installed on-site, and we utilize calibration vehicles for their evaluation. We first localize the calibration vehicles by detecting and tracking their license plates in a series of stereo images; afterward, we exploit the known information about their speed and acceleration to compute the distances that they traveled between two frames; finally, we determine the internal and external stereo camera pair parameters by minimizing the difference between the computed and estimated distances while preserving the epipolar geometry. We evaluated the presented method on a task of measuring the distances that vehicles traveled between two consecutive frames. For this evaluation, we recorded a dataset containing almost 700 vehicles with trajectories that were recorded using prototype hardware and ground truth distance measurements that were obtained from a pair of single-beam LIDARs. The evaluation results were compared with the current state-of-the-art methods for long-distance stereo camera pair calibration and used in the application of the proposed method on the vehicle speed measurement task.

A deep neural framework for real-time vehicular accident detection based on motion temporal templates

Vehicular accident prediction and detection has recently garnered curiosity and large amounts of attention in machine learning applications and related areas, due to its peculiar and fascinating application potentials in the development of Intelligent Transportation Systems (ITS) that play a pivotal role in the success of emerging smart cities. In this paper, we present a new vision-based framework for real-time vehicular accident prediction and detection, based on motion temporal templates and fuzzy time-slicing. The presented framework proceeds in a stepwise fashion, starting with automatically detecting moving objects (i.e., on-road vehicles or roadside pedestrians), followed by dynamically keep tracking of the detected moving objects based on temporal templates, clustering and supervised learning. Then, an extensive set of local features is extracted from the temporal templates of moving objects. Finally, an effective deep neural network (DNN) model is trained on the extracted features to detect abnormal vehicle behavioral patterns and thus predict an accident just before it occurs. The experiments on real-world vehicular accident videos demonstrate that the framework can yield mostly promising results by achieving a hit rate of 98.5% with a false alarm rate of 4.2% that compare very favorably to those from existing approaches, while still being able to deliver delay guarantees for realtime traffic monitoring and surveillance applications.

Deep Learning-Based Traffic Behavior Analysis under Multiple Camera Environment

In a video surveillance system, tracking multiple moving objects using a single camera feed is having numerous challenges. A multi-camera system increases the output image quality in both overlapping and non-overlapping environment. Traffic behavior analysis is an intensified demand in a recent topic of research. Due to increasing traffic in intercity roads, interstate, and national highways. Automated traffic visual surveillance applications with the multi-camera are a topic of research in computer vision. This paper, present a multi-camera system study for the overlapping area of the road for traffic analysis in three sections. The second section represents the thorough literature survey on the multi-camera system. Here, the third section is our proposed system using a dual-camera experimental setup with their coordination. A deep neural network is used in the experiments for traffic behavior analysis. The emphasis of this paper is on the physical arrangement of the multi-camera system, calibration, and advantages- disadvantages. On a conclusion note, future development and advancement in traffic analysis using a multi-camera system is discussed.

A Novel AMS-DAT Algorithm for Moving Vehicle Detection in a Satellite Video

Satellite videos have recently served as a new data source for a wide range of applications in traffic management and military surveillance. Due to its wider coverage, satellite videos show more advantages in large-scale monitoring than ground surveillance videos. However, pseudomotion background and low-resolution targets pose new challenges to moving vehicle detection in satellite videos, resulting in poor performance of conventional target detection methods when applied to satellite videos. To overcome this difficulty, we propose a novel moving vehicle detection approach using adaptive motion separation and difference accumulated trajectory. Specifically, a new indicator is designed to assist adaptive separation of moving targets and background, considering the scale invariance of vehicles in satellite videos. Meanwhile, we offer a vehicle discrimination algorithm based on a differential accumulated trajectory to distinguish the moving vehicles from the pseudomotion background. Experimental results on two satellite video data sets demonstrate that the proposed approach achieves better detection performance over the state-of-the-art moving vehicle detection methods.

Video Foreground Extraction Using Multi-View Receptive Field and Encoder–Decoder DCNN for Traffic and Surveillance Applications

The automatic detection of foreground (FG) objects in videos is a demanding area of computer vision, with essential applications in video-based traffic analysis and surveillance. New solutions have attempted exploiting deep neural network (DNN) for this purpose. In DNN, learning agents, i.e., features for video FG object segmentation is nontrivial, unlike image segmentation. It is a temporally processed decision-making problem, where the agents involved are the spatial and temporal correlations of the FG objects and the background (BG) of the scene. To handle this and to overcome the conventional DL models’ poor delineation at the borders of FG regions due to fixed-view receptive filed-based learning, this work introduces a Multi-view Receptive Field Encoder-Decoder Convolutional Neural Network called MvRF-CNN. The main contribution of the model is harnessing multiple views of convolutional (conv) kernels with residual feature fusions at early, mid and late stages in an encoder-decoder (EnDec) architecture. It enhances the ability of the model to learn condition-invariant agents resulting in highly delineated FG masks when compared to the existing approaches from heuristic- to DL-based techniques. The model is trained with sequence-specific labeled samples to predict scene-specific pixel-level labels of FG objects in near static scenes with a minute dynamism. The experimental study on 37 video sequences from traffic and surveillance scenarios that include complex environments, viz. dynamic background, camera jittery, intermittent object motion, scenes with cast shadows, night videos, and lousy weather proves the effectiveness of the model. The study covers two input configurations: a 3-channel (RGB) single frame and a 3-channel double-frame with a BG such that two consecutive grayscale frames stacked with a prior BG model. The ablation investigations are also conducted to show the importance of transfer learning (TL) and mid-fusion approaches for enhancing the segmentation performance and the model's robustness on failure modes: when there is lack of manually annotated hard ground truths (HGT) and testing the model under non-scene-specific videos. In overall, the model achieves a figure-of-merit of ${95\%}$ and 42 $FPS$ of mean average performance.

Real-time visual detection and tracking system for traffic monitoring

Computer vision systems for traffic monitoring represent an essential tool for a broad range of traffic surveillance applications. Two of the most noteworthy challenges for these systems are the real-time operation with hundreds of vehicles and the total occlusions which hinder the tracking of the vehicles. In this paper, we present a traffic monitoring approach that deals with these two challenges based on three modules: detection, tracking and data association. First, vehicles are identified through a deep learning based detector. Second, tracking is performed with a combination of a Discriminative Correlation Filter and a Kalman Filter. This permits to estimate the tracking error in order to make tracking more robust and reliable. Finally, the data association through the Hungarian algorithm combines the information of the previous steps. The contributions are: (i) a real-time traffic monitoring system robust to occlusions that can process more than four hundred vehicles simultaneously; and (ii) the application of the system to anomaly detection in traffic and roundabout input/output analysis. The system has been evaluated with more than two thousand vehicles in real-life videos.

Efficient Scale-Adaptive License Plate Detection System

License plate detection is a common problem in traffic surveillance applications. Although some solutions have been proposed in the literature, their success is usually restricted to very specific scenarios, with their performance dropping in more demanding conditions. One of the main challenges to be addressed for this kind of systems is the varying scale of the license plates, which depends on the distance between the vehicles and the camera. Traditionally, systems have handled this issue by sequentially running single-scale detectors over a pyramid of images. This approach, although simplifies the training process, requires as many evaluations as considered scales, which leads to running times that grow linearly with the number of scales considered. In this paper, we propose a scale-adaptive deformable part-based model which, based on a well-known boosting algorithm, automatically models scale during the training phase by selecting the most prominent features at each scale and notably reduces the test detection time by avoiding the evaluation at different scales. In addition, our method incorporates an empirically constrained-deformation model that adapts to different levels of deformation shown by distinct local features within license plates. As shown in the experimental section, the proposed detector is robust and scale and perspective independent and can work in quite diverse scenarios. Experiments on two datasets show that the proposed method achieves a significantly better performance in comparison with other methods of the state of the art.

Photonic approach for on‐board and ground radars in automotive applications

While automotive radars are driving the development of high-performance technologies for remote sensing, pushing radiofrequency systems to higher frequencies, photonics is gradually changing the approach to micro- and millimetre wave RF generation and distribution. With its unique features, photonics can extend the potential of radars, in particular for ground-based traffic surveillance and on-board automotive applications, enhancing traffic safety and enabling the deployment of smart driver-less vehicles. In fact, microwave photonics offers unprecedented flexibility and stability, such as −113 dBc/Hz (at 100 kHz offset frequency) at 80 GHz, with the capability of generating an extremely broad range of carrier frequencies. Moreover, it can employ signals which span up to several GHz of bandwidth, thus allowing higher precision in target detection and discrimination. This study compares photonic and electronic technologies, and it demonstrates, through simulation results, the benefits of a multiple input, multiple output photonic radar when applied to automotive case-study scenarios.

Vehicle Detection and Tracking for Traffic Surveillance Applications A Review Paper

International Journal of Computer Sciences and Engineering (A UGC Approved and indexed with DOI, ICI and Approved, DPI Digital Library) is one of the leading and growing open access, peer-reviewed, monthly, and scientific research journal for scientists, engineers, research scholars, and academicians, which gains a foothold in Asia and opens to the world, aims to publish original, theoretical and practical advances in Computer Science,Information Technology, Engineering (Software, Mechanical, Civil, Electronics & Electrical), and all interdisciplinary streams of Computing Sciences. It intends to disseminate original, scientific, theoretical or applied research in the field of Computer Sciences and allied fields. It provides a platform for publishing results and research with a strong empirical component. It aims to bridge the significant gap between research and practice by promoting the publication of original, novel, industry-relevant research.

Accelerating Target Detection in Passive Radar Sensors: Delay-Doppler-Acceleration Estimation

Passive bistatic radars (PBRs) need to use long integration times to achieve ranges of interest for air traffic surveillance applications. However, with the increase of integration time, classical target detection algorithms used in passive radars encounter some limitations in detecting accelerating targets. This is mainly due to a simplified target’s motion model used for detector design. In such cases, the mismatch between the target’s simplified motion model and that of the actual target motion limits any improvement in signal-to-noise ratio. To cure this especially in the presence of the accelerating targets, we obtain the received signal model and formulate target detection problem as an M-ary hypothesis testing problem. Then, we apply generalized likelihood ratio principle to derive a new 3-D sequential detection algorithm. In the proposed detector, range-Doppler-acceleration coordinates of detectable targets are estimated as compared to the classical target detection algorithms with range-Doppler estimations. To effectively implement the proposed detector over the desired range-Doppler-acceleration map, we use a modified version of chirp fast Fourier transform. We also obtain a closed-form expression for false alarm probability to adjust the detection threshold. Simulation results are provided to illustrate the superiority of the proposed 3-D-detection algorithm over state-of-the-art classical target detection algorithm given in the context of FM-based PBR systems.

StreamTune: dynamic resource scheduling approach for workload skew in video data center

Video surveillance applications need video data center to provide elastic virtual machine (VM) provisioning. However, the workloads of the VMs are hardly to be predicted for online video surveillance service. The unknown arrival workloads easily lead to workload skew among VMs. In this paper, we study how to balance the workload skew on online video surveillance system. First, we design the system framework for online surveillance service which consists of video capturing and analysis tasks. Second, we propose StreamTune, an online resource scheduling approach for workload balancing, to deal with irregular video analysis workload with the minimum number of VMs. We aim at timely balancing the workload skew on video analyzers without depending on any workload prediction method. Furthermore, we evaluate the performance of the proposed approach using a traffic surveillance application. The experimental results show that our approach is well adaptive to the variation of workload and achieves workload balance with less VMs.

Lagrange-Polynomial-Interpolation-Based Keystone Transform for a Passive Radar

In this paper, we address the problem of target's range migration in passive bistatic radar exploiting long coherent integration times with fairly wideband signals of opportunity. We resort to the well-known keystone transform (KT) to compensate for the range walk effect and to take advantage of a higher coherent integration gain against targets with nonnegligible radial velocity. Specifically, an efficient implementation of the KT is proposed, based on the Lagrange polynomial interpolation, in order to reduce the computational load of the method that mostly depends on the required slow-time interpolation stage. The analysis conducted against simulated data shows that the conceived approach allows us to achieve theoretical performance, while further reducing the KT complexity with respect to alternative solutions based on cardinal sine functions or chirp-Z transforms. Moreover, the application against experimental datasets collected by a DVB-T-based passive radar proves the practical effectiveness of the proposed algorithm and highlights its suitability for real-time air traffic surveillance applications.

BoxCars: Improving Fine-Grained Recognition of Vehicles Using 3-D Bounding Boxes in Traffic Surveillance

In this paper, we focus on fine-grained recognition of vehicles mainly in traffic surveillance applications. We propose an approach that is orthogonal to recent advancements in fine-grained recognition (automatic part discovery and bilinear pooling). In addition, in contrast to other methods focused on fine-grained recognition of vehicles, we do not limit ourselves to a frontal/rear viewpoint, but allow the vehicles to be seen from any viewpoint. Our approach is based on 3-D bounding boxes built around the vehicles. The bounding box can be automatically constructed from traffic surveillance data. For scenarios where it is not possible to use precise construction, we propose a method for an estimation of the 3-D bounding box. The 3-D bounding box is used to normalize the image viewpoint by "unpacking" the image into a plane. We also propose to randomly alter the color of the image and add a rectangle with random noise to a random position in the image during the training of convolutional neural networks (CNNs). We have collected a large fine-grained vehicle data set BoxCars116k, with 116k images of vehicles from various viewpoints taken by numerous surveillance cameras. We performed a number of experiments, which show that our proposed method significantly improves CNN classification accuracy (the accuracy is increased by up to 12% points and the error is reduced by up to 50% compared with CNNs without the proposed modifications). We also show that our method outperforms the state-of-the-art methods for fine-grained recognition.

Energy Interpolated Template Coding for Video Compression in Traffic Surveillance Application

In video coding, exploitation of temporal correlation between frames is an important step for reduction of redundant data in successive video frames. However, dynamic nature of video content introduces difficulty in finding temporal correlation. In this paper we propose a novel template coding approach to compress the video data for traffic surveillance which addresses above said difficulty. In this work, the conventional approach of the template coding, wherein two successive frames are considered, is improved by a ‘dynamic model’ of the template. The dynamism of template selection is achieved through energy interpolation of successive frame data over some time period, rather than only two successive frame data.Acoherent histogram model is developed to build accurate template to achieve improvement in compression. The proposed efficient template matching approach predicts exact template thereby minimizing the processing overheads and reduction in processing time. The obtained simulation result unveils that, the proposed approach results in accurate template localization, thereby improving the accuracy in coding and the coding speed in comparison to conventional template based compression approaches.

Parallel Continuous Preference Queries over Out-of-Order and Bursty Data Streams

Techniques to handle traffic bursts and out-of-order arrivals are of paramount importance to provide real-time sensor data analytics in domains like traffic surveillance, transportation management, healthcare and security applications. In these systems the amount of raw data coming from sensors must be analyzed by continuous queries that extract value-added information used to make informed decisions in real-time. To perform this task with timing constraints, parallelism must be exploited in the query execution in order to enable the real-time processing on parallel architectures. In this paper we focus on continuous preference queries, a representative class of continuous queries for decision making, and we propose a parallel query model targeting the efficient processing over out-of-order and bursty data streams. We study how to integrate punctuation mechanisms in order to enable out-of-order processing. Then, we present advanced scheduling strategies targeting scenarios with different burstiness levels, parameterized using the index of dispersion quantity. Extensive experiments have been performed using synthetic datasets and real-world data streams obtained from an existing real-time locating system. The experimental evaluation demonstrates the efficiency of our parallel solution and its effectiveness in handling the out-of-orderness degrees and burstiness levels of real-world applications.

VHF Cross-Range Profiling of Aerial Targets Via Passive ISAR: Signal Processing Schemes and Experimental Results

The possibility of obtaining cross-range profiles of aerial targets is contemplated in this paper by applying inverse synthetic aperture radar (ISAR) techniques against FM-based passive radar data. The considered application is shown to pose a number of limitations, mostly related to the very long coherent processing interval required to attain a reasonable cross-range resolution. To mitigate such limitations, a suitable two-stage ISAR processing scheme is proposed for the estimation of the target motion induced phase history and its compensation. The results are shown against experimental data sets collected by a very high frequency (VHF) band passive radar receiver developed at Sapienza University of Rome. This allows to demonstrate the effectiveness of the proposed approach that provides the FM-based passive radar with advanced capabilities for air traffic surveillance applications.

UAV-Based Traffic Analysis: A Universal Guiding Framework Based on Literature Survey

Abstract: The Unmanned Aerial Vehicles (UAVs) commonly also known as drones are considered as one of the most dynamic and multi-dimensional emerging technologies of the modern era. Recently, this technology has found multiple applications in the transportation field as well; ranging from the traffic surveillance applications to the traffic network analysis for the overall improvement of the traffic flow and safety conditions. However, in order to conduct a UAV-based traffic study, an extremely diligent planning and execution is required followed by an optimal data analysis and interpretation procedure. This paper presents a universal guiding framework for ensuring a safe and efficient execution of a UAV-based study. It also explores the analysis steps that follow the execution of a drone flight. The framework based on the existing studies, is classified into the following seven components: (i) scope definition, (ii) flight planning, (iii) flight implementation, (iv) data acquisition, (v) data processing and analysis, (vi) data interpretation and (vii) optimized traffic application. The proposed framework provides a comprehensive guideline for an efficient conduction and completion of a drone-based traffic study. It gives an overview of the management in the context of the hardware and the software entities involved in the process. In this paper, an extensive yet systematic review of the existing traffic-related UAV studies is presented by moulding them in a step-by-step framework. With the significant increase in the number of UAV studies expected in the coming years, this literature review could become a useful resource for future researchers. The future research will mainly focus on the practical applications of the proposed guiding framework of the UAV-based traffic monitoring and analysis study.