Urban Surveillance Systems Research Articles

DiffRank: Enhancing efficiency in discontinuous frame rate analysis for urban surveillance systems

Urban public safety management relies heavily on video surveillance systems, which provide crucial visual data for resolving a wide range of incidents and controlling unlawful activities. Traditional methods for target detection predominantly employ a two-stage approach, focusing on precision in identifying objects such as pedestrians and vehicles. These objects, typically sparse in large-scale, lower-quality surveillance footage, induce considerable redundant computation during the initial processing stage. This redundancy constrains real-time detection capabilities and escalates processing costs. Furthermore, transmitting raw images and videos laden with superfluous information to centralized back-end systems significantly burdens network communications and fails to capitalize on the computational resources available at diverse surveillance nodes. This study introduces DiffRank, a novel preprocessing method for fixed-angle video imagery in urban surveillance. The method strategically generates candidate regions during preprocessing, thereby reducing redundant object detection and improving the efficiency of the detection algorithm. Drawing upon change detection principles, a background feature learning approach utilizing shallow features has been developed. This approach prioritizes learning the characteristics of fixed-area backgrounds over direct background identification. As a result, alterations in ROI are efficiently discerned using computationally efficient shallow features, markedly accelerating the generation of proposed Regions of Interest (ROIs) and diminishing the computational demands for subsequent object detection and classification. Comparative analysis on various public and private datasets illustrates that DiffRank, while maintaining high accuracy, substantially outperforms existing baselines in terms of speed, particularly with larger image sizes (e.g., an improvement exceeding 300 % at 1920×1080 resolution). Moreover, the method demonstrates enhanced robustness compared to baseline methods, efficiently disregarding static targets like mannequins in display windows. The advancements in candidate area preprocessing enable a balanced approach between detection accuracy and overall detection speed, making the algorithm highly applicable for real-time on-site analysis in edge computing scenarios and cloud-edge collaborative computing environments.

A novel approach for reliable pedestrian trajectory collection with behavior-based trajectory reconstruction for urban surveillance systems

Collecting reliable pedestrian trajectories in pedestrian behavior analysis, trajectories broken by frame sampling and trajectories crossing in multi-object conditions often hinder their performance of existing pedestrian tracking models. Despite attempts to address these issues by performing detection and tracking simultaneously using deep learning algorithms, previous methods still struggle with errors such as mistaking a single pedestrian for multiple pedestrians. We propose a novel approach to efficiently collect and correct pedestrian trajectories with minimized practical errors in multi-object conditions for urban surveillance systems. Our system utilizes a single vision sensor to automatically collects trajectories of multiple pedestrians and employ simple, low-computational algorithms, particularly the Deep simple online real-time tracking (Deep SORT) method, to calibrate the trajectories from tracking-by-detection models. Additionally, our system identifies and merges broken pedestrian trajectories, treating them as potential single trajectories, while considering their spatiotemporal ranges. We evaluate the proposed system by implementing it on real testbed video footage. Our method significantly improves practical errors and achieves more accurate pedestrian trajectories compared to existing models, and exhibits robust characteristics, effectively handling complex situations such as occlusions and crowds.

How do children move and behave on streets? Vision-based movement behavior analysis using children's trajectories in urban surveillance systems

Understanding how children use streets will allow urban planners to create streets that are livelier and more child-friendly. However, it is still challenging to investigate detailed information on children's movement behaviors on individual streets due to labor-intensive observations, the difficulty of integrating qualitative data, and privacy concerns. With the wide application of vision-based technologies, Researchers can analyze the movements and behaviors of large groups of children on streets automatically and continuously. This study aims to analyze children's movement behaviors on streets utilizing computer vision techniques in urban surveillance systems. The proposed methods automatically extract children's trajectories, calculate movement behavioral features, and classify the movement behavioral characteristics of children. Our results identify five movement behaviors of children on streets: walking, staying, running, accelerating, and decelerating, demonstrating their use of streets. In addition, the results showed the feasibility of computer vision techniques to identify differences in children's movement behaviors under varying conditions, such as day and time, and the potential of street environments to positively influence children's use of streets from the daily street lives of children. Utilizing vision-based analysis of children's movement behaviors provides meaningful information for improving streets for children toward a better understanding of the use of streets.

IoT-based Automated Vehicle Plate Detection Algorithm for Urban Surveillance Systems by A New Hybrid Optimized CNN

IoT-based Automated Vehicle Plate Detection Algorithm for Urban Surveillance Systems by A New Hybrid Optimized CNN

Detecting the Severity of Socio-Spatial Conflicts Involving Wild Boars in the City Using Social Media Data.

The encroachment of wild boars into urban areas is a growing problem. The occurrence of wild boars in cities leads to conflict situations. Socio-spatial conflicts can escalate to a varied degree. Assessments of these conflicts can be performed by analyzing spatial data concerning the affected locations and wild boar behaviors. The collection of spatial data is a laborious and costly process that requires access to urban surveillance systems, in addition to regular analyses of intervention reports. A supporting method for assessing the risk of wild boar encroachment and socio-spatial conflict in cities was proposed in the present study. The developed approach relies on big data, namely, multimedia and descriptive data that are on social media. The proposed method was tested in the city of Olsztyn in Poland. The main aim of this study was to evaluate the applicability of data crowdsourced from a popular social networking site for determining the location and severity of conflicts. A photointerpretation method and the kernel density estimation (KDE) tool implemented in ArcGIS Desktop 10.7.1 software were applied in the study. The proposed approach fills a gap in the application of crowdsourcing data to identify types of socio-spatial conflicts involving wild boars in urban areas. Validation of the results with reports of calls to intervention services showed the high coverage of this approach and thus the usefulness of crowdsourcing data.

Deep Siamese Cross-Residual Learning for Robust Visual Tracking

The sixth-generation (6G) wireless technology contributes to the establishment of the Internet of Things (IoT). Recently, the IoT has become popular because of its smart architectures and various applications. Among these applications, intelligent urban surveillance systems for smart cities are becoming more and more important. Therefore, designing a robust visual tracking method has become an urgent task. Deep Siamese convolutional neural networks have been applied to visual tracking recently because of their advantageous abilities to learn a matching function between the template and the target candidate. Unlike traditional Siamese networks, which separately treat the two branches, we propose deep Siamese cross-residual learning to entangle the two branches from the beginning to the end of the Siamese network. This strategy can make the two branches exchange instance-specific information at different nodes of the network and learn a more compact representation of the target. In addition, we propose a combined loss function, which consists of two complementary tasks. One task is to learn a matching function directly and the other one is to learn a classification function. Moreover, our model does not need to load any pretrained weights and is trained with limited sequences from scratch. Plenty of experiments show that our tracker performs favorably against many state-of-the-art tracking methods.

Find you if you drive: Inferring home locations for vehicles with surveillance camera data

Inferring home locations for users from spatiotemporal data has become increasingly important for real-world applications ranging from security, recommendation, advertisement targeting, to transportation scheduling. Existing home location inference studies are based either on geo-tagged social media data or continuous GPS data. Yet this inference problem in highly sparse vehicle trajectories in urban surveillance systems remains largely unexplored. In this paper, we propose an accurate home location inference framework for vehicles in urban traffic surveillance systems by considering both spatial and temporal characteristics. To the best of our knowledge, we are the first to predict exact home community for vehicles at such a fine granularity using the sparse and noisy surveillance camera data. First, we collect and preprocess multiple contextual datasets to obtain a context-rich road network with residential communities and surveillance cameras. Second, we detect the potential home location areas for each vehicle by clustering Origin–Destination (O-D) pairs extracted in vehicle’s camera-based trajectories. Then we further propose an in∕out time pattern to distinguish the home area candidate from the O-D clusters by leveraging time-aware constraints. Furthermore, to find the exact home community, we propose a Kernel Density Estimation (KDE) based inference method with a local camera selection strategy to effectively identify the home community from the residential communities near/in the home area candidate. Our comprehensive experiments on a large-scale real-world dataset demonstrate the effectiveness of our proposed method.

Unmanned Aerial Vehicle Video-Based Target Tracking Algorithm Using Sparse Representation

Target tracking based on unmanned aerial vehicle (UAV) video is a significant technique in intelligent urban surveillance systems for smart city applications, such as smart transportation, road traffic monitoring, inspection of stolen vehicle, etc. In this paper, a computer vision-based target tracking algorithm aiming at locating UAV-captured targets, like pedestrian and vehicle, is proposed using sparse representation theory. First of all, each target candidate is sparsely represented in the subspace spanned by a joint dictionary. Then, the sparse representation coefficient is further constrained by an $L_{2}$ regularization based on temporal consistency. To cope with the partial occlusion appearing in UAV videos, a Markov random field (MRF)-based binary support vector with contiguous occlusion constraint is introduced to our sparse representation model. For long-term tracking, the particle filter framework along with a dynamic template update scheme is designed. Both qualitative and quantitative experiments implemented on visible (Vis) and infrared (IR) UAV videos prove that the presented tracker can achieve better performances in terms of precision rate and success rate when compared with other state-of-the-art trackers.

Iot Driven Automated Object Detection Algorithm For Urban Surveillance System In Smart City

Automated object detection algorithm is an important research challenge in intelligent urban surveillance systems for Internet of Things (IoT) and smart cities applications. In particular, smart vehicle license plate recognition and vehicle detection are recognized as core research issues of these IoT-driven intelligent urban surveillance systems. They are key techniques in most of the traffic related IoT applications, such as road traffic real-time monitoring, security control of restricted areas, automatic parking access control, searching stolen vehicles, etc. In this paper, we propose a novel unified method of automated object detection for urban surveillance systems. We use this novel method to determine and pick out the highest energy frequency areas of the images from the digital camera imaging sensors, that is, either to pick the vehicle license plates or the vehicles out from the images. The other sensors like flame and ultrasonic sensor are used to monitor nearby objects. Our proposed method can not only help to detect object vehicles rapidly and accurately, but also can be used to reduce big data volume needed to be stored in urban surveillance systems

Choice of Application Layer Protocols for Next Generation Video Surveillance Using Internet of Video Things

Video surveillance has become ubiquitous due to the increasing security requirements in every sphere of life. The next generation video surveillance system (VSS) possesses great challenges in various applications, such as intelligent urban surveillance systems and smart cities. In these applications, we need to deal with the fast-growing number of surveillance nodes which introduce several constraints, e.g., high latency, high bandwidth, high energy consumption, and CPU and memory usage. To address these issues, the Internet of Video Things (IoVT), which is considered to be a part of the Internet of Things (IoT), can be a solution. The IoVT is composed of visual sensors (i.e., cameras) connected to the Internet. Unlike conventional systems, the VSS under an IoVT framework provides multiple layers (i.e., edge, fog, and cloud) of communication and decision making by capturing and analyzing rich contextual and behavioral information. Since an appropriate application layer protocol (ALP) can help in alleviating the challenges of future VSSs, the selection of ALPs is important for IoVT-based systems. Therefore, this paper presents a generic architecture of an IoVT-based VSS and a comparative analysis of several ALPs, such as MQTT, AMQP, HTTP, XMPP, CoAP, and DDS, with real-time experimentation. This analysis will assist the users to choose the appropriate ALPs in various surveillance applications and determine their suitability at different nodes of the IoVT framework.

Big data oriented novel background subtraction algorithm for urban surveillance systems

Due to the tremendous volume of data generated by urban surveillance systems, big data oriented low-complexity automatic background subtraction techniques are in great demand. In this paper, we propose a novel automatic background subtraction algorithm for urban surveillance systems in which the computer can automatically renew an image as the new background image when no object is detected. This method is both simple and robust with respect to changes in light conditions.

Infrared Image Enhancement Using Adaptive Histogram Partition and Brightness Correction

Infrared image enhancement is a crucial pre-processing technique in intelligent urban surveillance systems for Smart City applications. Existing grayscale mapping-based algorithms always suffer from over-enhancement of the background, noise amplification, and brightness distortion. To cope with these problems, an infrared image enhancement method based on adaptive histogram partition and brightness correction is proposed. First, the grayscale histogram is adaptively segmented into several sub-histograms by a locally weighted scatter plot smoothing algorithm and local minima examination. Then, the fore-and background sub-histograms are distinguished according to a proposed metric called grayscale density. The foreground sub-histograms are equalized using a local contrast weighted distribution for the purpose of enhancing the local details, while the background sub-histograms maintain the corresponding proportions of the whole dynamic range in order to avoid over-enhancement. Meanwhile, a visual correction factor considering the property of human vision is designed to reduce the effect of noise during the procedure of grayscale re-mapping. Lastly, particle swarm optimization is used to correct the mean brightness of the output by virtue of a reference image. Both qualitative and quantitative evaluations implemented on real infrared images demonstrate the superiority of our method when compared with other conventional methods.

Particle swarm optimization-based local entropy weighted histogram equalization for infrared image enhancement

Infrared image enhancement plays a significant role in intelligent urban surveillance systems for smart city applications. Unlike existing methods only exaggerating the global contrast, we propose a particle swam optimization-based local entropy weighted histogram equalization which involves the enhancement of both local details and fore-and background contrast. First of all, a novel local entropy weighted histogram depicting the distribution of detail information is calculated based on a modified hyperbolic tangent function. Then, the histogram is divided into two parts via a threshold maximizing the inter-class variance in order to improve the contrasts of foreground and background, respectively. To avoid over-enhancement and noise amplification, double plateau thresholds of the presented histogram are formulated by means of particle swarm optimization algorithm. Lastly, each sub-image is equalized independently according to the constrained sub-local entropy weighted histogram. Comparative experiments implemented on real infrared images prove that our algorithm outperforms other state-of-the-art methods in terms of both visual and quantized evaluations.

IoT-Driven Automated Object Detection Algorithm for Urban Surveillance Systems in Smart Cities

Automated object detection algorithm is an important research challenge in intelligent urban surveillance systems for Internet of Things (IoT) and smart cities applications. In particular, smart vehicle license plate recognition and vehicle detection are recognized as core research issues of these IoT-driven intelligent urban surveillance systems. They are key techniques in most of the traffic related IoT applications, such as road traffic real-time monitoring, security control of restricted areas, automatic parking access control, searching stolen vehicles, etc. In this paper, we propose a novel unified method of automated object detection for urban surveillance systems. We use this novel method to determine and pick out the highest energy frequency areas of the images from the digital camera imaging sensors, that is, either to pick the vehicle license plates or the vehicles out from the images. Our proposed method can not only help to detect object vehicles rapidly and accurately, but also can be used to reduce big data volume needed to be stored in urban surveillance systems.

Big Data-Driven Fast Reducing the Visual Block Artifacts of DCT Compressed Images for Urban Surveillance Systems

The Urban Surveillance Systems generate huge amount of video and image data and impose high pressure onto the recording disks. It is obvious that the research of video is a key point of big data research areas. Since videos are composed of images, the degree and efficiency of image compression are of great importance. Although the DCT based JPEG standard are widely used, it encounters insurmountable problems. For instance, image encoding deficiencies such as block artifacts have to be removed frequently. In this paper, we propose a new, simple but effective method to fast reduce the visual block artifacts of DCT compressed images for urban surveillance systems. The simulation results demonstrate that our proposed method achieves better quality than widely used filters while consuming much less computer CPU resources.

A Novel Vehicle Classification Model for Urban Traffic Surveillance Using the Deep Neural Network Model

The vehicle detection is the backbone of the urban surveillance systems, which is used to obtain and identify the various statistics of the urban vehicular mobility. Also the urban surveillance systems are used for the vehicle tracking or vehicular object classification. The proposed model has been designed for the purpose of the urban surveillance and vehicular modelling of the traffic. The proposed model has been designed for the vehicle position identification as well as the vehicle type classification using the deep neural network. The proposed model has been tested with a standard dataset image for the result evaluation. The experimental results has been shown the effectiveness of the proposed model, where the proposed model has been found successful in detection and classification of all of the vehicles in the given image.

Urban surveillance systems: from the laboratory to the commercial world

Research in the surveillance domain was confined for years in the military domain. Recently, as military spending for this kind of research was reduced and the technology matured, the attention of the research and development community turned to commercial applications of surveillance. In this paper we describe a state-of-the-art monitoring system developed by a corporate R&D lab in cooperation with the corresponding security business units. It represents a sizable effort to transfer some of the best results produced by computer vision research into a viable commercial product. Our description spans both practical and technical issues. From the practical point of view we analyze the state of the commercial security market, typical cultural differences between the research team and the business team and the perspective of the potential users of the technology. These are important issues that have to be dealt with or the surveillance technology will remain in the lab for a long time. From the technical point of view we analyze our algorithmic and implementation choices. We describe the improvements we introduced to the original algorithms reported in the literature in response to some problems that arose during field testing. We also provide extensive experimental results that highlight the strong points and some weaknesses of the prototype system.

Improvement of Congestion Detection on Expressways

This paper develops a method of improving congestion detection in urban motorway traffic surveillance and control systems in Japan. When developing motorway traffic surveillance and control systems, accurate congestion detection is often difficult or impossible because of the disturbances due to the oscillation of congested traffic flow. To improve the accuracy of congestion detection, it is necessary to distinguish noncongested flow from congested flow. Traffic flow data for the Tokyo Expressway and the Keiyo Expressway were collected. Computer experiments were carried out to develop a test to improve the accuracy of congestion detection on urban expressways. Applying the reference value proposed in this study greatly improved the accuracy of congestion detection.