Video Surveillance Data Research Articles

Enhancing real‐time traffic volume prediction: A two‐step approach of object detection and time series modelling

AbstractA two‐step framework that integrates real‐time data collection with time series forecasting models for predicting traffic volume is proposed. In the first step, the framework utilizes live highway surveillance video data and YOLO‐v7 object detector to construct accurate traffic volume data. In the second step, an ARIMA–LSTM time series model is applied to forecast future traffic volumes. Experimental results show that YOLO‐v7 achieved a vehicle detection accuracy of over 93.30%, ensuring high precision in traffic volume data construction. The ARIMA–LSTM model demonstrated superior performance in traffic volume prediction, with a mean squared error of 87.97, root mean squared error of 10,388.57, and mean absolute error of 101.39. YOLO‐v7's detection speed of 7.8 ms per frame further validates the feasibility of real‐time data construction. The findings indicate that the combination of YOLO‐v7 for vehicle detection and ARIMA–LSTM for traffic prediction is highly effective, offering a significant reduction in training time compared to more complex deep learning models while maintaining high prediction accuracy. This research presents a unified solution for traffic data collection and prediction, enhancing transportation infrastructure planning and optimizing traffic flow. Future work will focus on extending the prediction intervals and further refining the models to improve performance.

Robust Mixed-Rate Region-of-Interest-Aware Video Compressive Sensing for Transmission Line Surveillance Video

Classic video compression methods usually suffer from long encode time and requires large memories, making it hard to deploy on edge devices; thus, video compressive sensing, which requires less resources during encoding, is receiving more attention. We propose a robust mixed-rate ROI-aware video compressive sensing algorithm for transmission line surveillance video compression. The proposed method compresses foreground targets and background frames separately and uses reversible neural network to reconstruct original frames. The result on transmission line surveillance video data shows that the proposed compressive sensing method can achieve 26.47, 34.71 PSNR and 0.6839, 0.9320 SSIM higher than existing methods on 1.5% and 15% measurement rates, and the proposed ROI extraction net can precisely retrieve regions under high noise levels. This research not only demonstrates the potential for a more efficient video compression technique in resource-constrained environments, but also lays a foundation for future advancements in video compressive sensing techniques and their applications in various real-time surveillance systems.

Trajectory Recognition and Working Condition Analysis of Rod Pumping Systems Based on Pose Estimation Method with Heatmap-Free Joint Detection

Summary Rod pump systems are the primary production tools in oilfield development. Analyzing their working conditions provides a theoretical foundation for formulating production optimization plans and adjusting equipment parameters. Existing machine learning–based condition analysis methods rely on dynamometer cards and cannot capture the actual operational trajectory of the pumping unit. To address this issue, this paper proposes a keypoint detection method for pumping units based on pose estimation of heatmap-free joint detection from video data. A data annotation scheme suitable for the task of detecting pumping unit keypoints was developed, and the learning criteria for this task were optimized. An end-to-end heatmap-free pose estimation algorithm was used to process images of the pumping unit, yielding predicted keypoint positions, thereby enabling the identification of the keypoint motion trajectories of the pumping unit. Experiments validated the proposed method and compared it with general learning criteria. Results show that this method accurately captures the keypoint positions of the pumping unit, with the optimized learning criteria model improving by more than 5% compared with general methods and increasing the keypoint object keypoint similarity (OKS) by more than 30%. The model’s results can be used for the actual operational trajectory recognition of the pumping unit, automatically calculating the motion parameters of the polished rod, and intelligently assessing the balance and working condition analysis of the pumping unit. This realizes the intelligent application of video surveillance data, significantly contributing to the dynamic study of rod pump systems.

A bridge dynamic response analysis and load recognition method using traffic imaging

As the primary variable load of bridges, vehicle load is an important parameter for bridge health monitoring. However, traditional Weigh-in-Motion (WIM) systems and the commonly used method of placing sensors on the bridge are challenging to apply in load monitoring for many small and medium-sized bridges. Therefore, this paper proposes a bridge vehicle load identification method based on traffic surveillance video data. Leveraging the surveillance video data on the bridge, without introducing additional hardware devices, the displacement of target points is detected through sub-pixel level image detection algorithms, enabling non-contact measurement of bridge structural response through imaging. A spatiotemporal relationship model of structural displacement, vehicle load, and load distribution is established to solve for vehicle load. Finally, model bridge tests under various loading conditions and engineering practice experiments are conducted to validate the feasibility of the method. The results of the model bridge tests show that the structural displacement measured using traffic video measurement has a deviation of less than 10% compared to the measurements obtained using contact displacement sensors (LVDT), and it can accurately reflect the displacement characteristics of the structure. The results of the field tests demonstrate that the average estimation deviation for heavy vehicle loads ranging from 12 to 18 tons is approximately 18%, meeting the engineering requirements. The proposed method can provide load statistical information for the extensive health monitoring of small and medium-sized bridges and offer a new technical pathway for obtaining bridge load information.

Multi-Task Intelligent Monitoring of Construction Safety Based on Computer Vision

Effective safety management is vital for ensuring construction safety. Traditional safety inspections in construction heavily rely on manual labor, which is both time-consuming and labor-intensive. Extensive research has been conducted integrating computer-vision technologies to facilitate intelligent surveillance and improve safety measures. However, existing research predominantly focuses on singular tasks, while construction environments necessitate comprehensive analysis. This study introduces a multi-task computer vision technology approach for the enhanced monitoring of construction safety. The process begins with the collection and processing of multi-source video surveillance data. Subsequently, YOLOv8, a deep learning-based computer vision model, is adapted to meet specific task requirements by modifying the head component of the framework. This adaptation enables efficient detection and segmentation of construction elements, as well as the estimation of person and machine poses. Moreover, a tracking algorithm integrates these capabilities to continuously monitor detected elements, thereby facilitating the proactive identification of unsafe practices on construction sites. This paper also presents a novel Integrated Excavator Pose (IEP) dataset designed to address the common challenges associated with different single datasets, thereby ensuring accurate detection and robust application in practical scenarios.

Intelligent Surveillance of Airport Apron: Detection and Location of Abnormal Behavior in Typical Non-Cooperative Human Objects

Most airport surface surveillance systems focus on monitoring and commanding cooperative objects (vehicles) while neglecting the location and detection of non-cooperative objects (humans). Abnormal behavior by non-cooperative objects poses a potential threat to airport security. This study collects surveillance video data from civil aviation airports in several regions of China, and a non-cooperative abnormal behavior localization and detection framework (NC-ABLD) is established. As the focus of this paper, the proposed framework seamlessly integrates a multi-scale non-cooperative object localization module, a human keypoint detection module, and a behavioral classification module. The framework uses a serial structure, with multiple modules working in concert to achieve precise position, human keypoints, and behavioral classification of non-cooperative objects in the airport field. In addition, since there is no publicly available rich dataset of airport aprons, we propose a dataset called IIAR-30, which consists of 1736 images of airport surfaces and 506 video clips in six frequently occurring behavioral categories. The results of experiments conducted on the IIAR-30 dataset show that the framework performs well compared to mainstream behavior recognition methods and achieves fine-grained localization and refined class detection of typical non-cooperative human abnormal behavior on airport apron surfaces.

Generation of deposit-derived pyroclastic density currents by repeated crater rim failures at Stromboli Volcano (Italy)

The gravitational instability of hot material deposited during eruptive activity can lead to the formation of glowing avalanches, commonly known as deposit-derived pyroclastic density currents (PDCs). These currents can travel hundreds of metres to several kilometres from the source at exceptionally high temperatures, posing a catastrophic hazard to areas surrounding steep-slope volcanoes. The occurrence of deposit-derived PDCs is often associated with crater rim failure, which can be triggered by various factors such as magma thrust from dike injection, magma fingering, bulging or less commonly, powerful explosions. Here, the in-depth study of data from the multi-parametric monitoring network operating on Stromboli (Italy), including video surveillance, seismicity and ground deformation data, complemented by remote topographic sensing data, has facilitated the understanding of the events leading to the crater rim collapse on 9 October and 4 December 2022. The failures resulted in the remobilisation of 6.4 ± 1.0 × 103 m3 and 88.9 ± 26.7 × 103 m3 of material for the 9 October and the 4 December 2022, respectively, which propagated as PDCs along the NW side of the volcano and reached the sea in a few tens of seconds. These events were characterised by a preparatory phase marked by an increase in magmatic pressure in the preceding weeks, which correlated with an increase in the displacement rate of the volcano’s summit. There was also an escalation in explosive degassing, evidenced by spattering accompanied by seismic tremors in the hours before the collapse.These events have been interpreted as an initial increase in magma vesicularity, followed by the release of gas once percolation threshold was reached. The degassing process induced densification of the magma, resulting in increased thrust on the conduit walls due to increased magmastatic pressure. This phase coincided with crater rim collapse, often followed or accompanied by the onset of lava overflow phases. A mechanism similar to the one proposed may shed light on similar phenomena observed at other volcanoes. The analysis performed in this study highlights the need for a multi-parametric and multi-platform approach to fully understand such complex phenomena. By integrating different data sources, including seismic, deformation and remote sensing data, it is possible to identify the phenomena associated with the different phases leading to crater rim collapse and the subsequent development of deposit-derived PDCs.

Proximity tracking using ultra-wideband technology for equine social behaviour research

Sociopositive interactions with conspecifics are essential for equine welfare and quality of life. This study aimed to validate the use of wearable ultra-wideband (UWB) technology to quantify the spatial relationships and dynamics of social behaviour in horses by continuous (1/s) measurement of interindividual distances. After testing the UWB devices’ spatiotemporal accuracy in a static environment, the UWB measurement validity, feasibility and utility under dynamic field conditions was assessed in a group of 8 horses. Comparison of the proximity measurements with video surveillance data established the measurement accuracy and validity (r = 0.83, p < 0.0001) of the UWB technology. The utility for social behaviour research was demonstrated by the excellent accordance of affiliative relationships (preferred partners) identified using UWB with video observations. The horses remained a median of 5.82 m (95% CI 5.13–6.41 m) apart from each other and spent 20% (median, 95% CI 14–26%) of their time in a distance ≤ 3 m to their preferred partner. The proximity measurements and corresponding speed calculation allowed the identification of affiliative versus agonistic approaches based on differences in the approach speed and the distance and duration of the resulting proximity. Affiliative approaches were statistically significantly slower (median: 1.57 km/h, 95% CI 1.26–1.92 km/h, p = 0.0394) and resulted in greater proximity (median: 36.75 cm, 95% CI 19.5–62 cm, p = 0.0003) to the approached horse than agonistic approaches (median: 3.04 km/h, 95% CI 2.16–3.74 km/h, median proximity: 243 cm, 95% CI 130–319 cm), which caused an immediate retreat of the approached horse at a significantly greater speed (median: 3.77 km/h, 95% CI 3.52–5.85 km/h, p < 0.0001) than the approach.

Captionomaly: A Deep Learning Toolbox for Anomaly Captioning in Social Surveillance Systems

Real-time video stream monitoring is gaining huge attention lately with an effort to fully automate this process. On the other hand, reporting can be a tedious task, requiring manual inspection of several hours of daily clippings. Errors are likely to occur because of the repetitive nature of the task causing mental strain on operators. There is a need for an automated system that is capable of real-time video stream monitoring in social systems and reporting them. In this article, we provide a tool aiming to automate the process of anomaly detection and reporting. We combine anomaly detection and video captioning models to create a pipeline for anomaly reporting in descriptive form. A new set of labels by creating descriptive captions for the videos collected from the UCF-Crime (University of Central Florida-Crime) dataset has been formulated. The anomaly detection model is trained on the UCF-Crime, and the captioning model is trained with the newly created labeled set UCF-Crime video description (UCFC-VD). The tool will be used for performing the combined task of anomaly detection and captioning. Automated anomaly captioning would be useful in the efficient reporting of video surveillance data in different social scenarios. Several testing and evaluation techniques were performed. Source code and dataset: https://github.com/Adit31/Captionomaly-Deep-Learning-Toolbox-for-Anomaly-Captioning.

A novel approach for enhanced abnormal action recognition via coarse and precise detection stage

&lt;abstract&gt; &lt;p&gt;With the proliferation of urban video surveillance systems, the abundance of surveillance video data has emerged as a pivotal asset for enhancing public safety. Within these video archives, the identification of abnormal human actions carries profound implications for security incidents. Nevertheless, existing surveillance systems primarily rely on conventional algorithms, leading to both missed incidents and false alarms. To address the challenge of automating multi-object surveillance video analysis, this study introduces a comprehensive method for the detection and recognition of multi-object abnormal actions. This study comprises a two-stage framework: the coarse detection stage employs an enhanced YOWOv2E model for spatio-temporal action detection, while the precise detection stage utilizes a two-stream network for precise action classification. In parallel, this paper presents the PSA-Dataset to address the current limitations in the field of abnormal action detection. Experimental results, collected from both public datasets and a self-built dataset, illustrate the effectiveness of the proposed method in identifying a wide spectrum of abnormal actions. This work offers valuable insights for automating the analysis of human actions in videos pertaining to public security.&lt;/p&gt; &lt;/abstract&gt;

Research on Blockchain-Based Cereal and Oil Video Surveillance Abnormal Data Storage

Cereal and oil video surveillance data play a vital role in food traceability, which not only helps to ensure the quality and safety of food, but also helps to improve the efficiency and transparency of the supply chain. Traditional video surveillance systems mainly adopt a centralized storage mode, which is characterized by the deployment of multiple monitoring nodes and a large amount of data storage. It is difficult to guarantee the data security, and there is an urgent need for a solution that can achieve the safe and efficient storage of cereal and oil video surveillance data. This study proposes a blockchain-based abnormal data storage model for cereal and oil video surveillance. The model introduces a deep learning algorithm to process the cereal and oil video surveillance data, obtaining images with abnormal behavior from the monitoring data. The data are stored on a blockchain after hash operation, and InterPlanetary File System (IPFS) is used as a secondary database to store video data and alleviate the storage pressure on the blockchain. The experimental results show that the model achieves the safe and efficient storage of cereal and oil video surveillance data, providing strong support for the sustainable development of the cereal and oil industry.

A teacher–student deep learning strategy for extreme low resolution unsafe action recognition in construction projects

A large proportion of construction accidents are caused by workers’ unsafe actions. Due to the complexity of the work environment and excessive demands of safety supervision on construction sites, many modern information technologies, such as computer vision (CV) technologies, have been increasingly applied and gradually replaced the traditional manual supervision to automatically identify unsafe behaviors from surveillance video data. Current models focus on high resolution (HR) video containing high quality features. However, challenges remain for more specific real-world construction scenarios, including but not limited to lack of high-quality data, far-field video recognition, privacy protection, and limited computing resources. To address the challenges, in this paper, a simple but effective method relying on the Teacher–Student Learning Architecture is proposed, where the teacher network assists the student network in effectively capturing useful features for downstream action recognition from extreme low resolution (eLR) video which can be easily transferred to other tasks. In addition, a Knowledge Learning Model and a similarity loss function are proposed to better guide the training of student network. In numerical experiments and case studies, our proposed training strategy achieves an top-1 accuracy of 28.53% and 71.81% on eLR HMDB dataset and eLR CMA dataset, 5.96% and 3.89% higher than the baseline model. The proposed strategy can make the automatic safety monitoring of construction works more accurate and reliable, thereby effectively reducing the accident rate and management cost. In addition, the performance of proposed strategy depends on efficient knowledge distillation methods, which may inspire future applications of CV-based deep learning models in construction management to achieve higher recognition results while maintaining low computational cost.

A Lightweight Model for Real-Time Detection of Vehicle Black Smoke

This paper discusses the application of deep learning technology in recognizing vehicle black smoke in road traffic monitoring videos. The use of massive surveillance video data imposes higher demands on the real-time performance of vehicle black smoke detection models. The YOLOv5s model, known for its excellent single-stage object detection performance, has a complex network structure. Therefore, this study proposes a lightweight real-time detection model for vehicle black smoke, named MGSNet, based on the YOLOv5s framework. The research involved collecting road traffic monitoring video data and creating a custom dataset for vehicle black smoke detection by applying data augmentation techniques such as changing image brightness and contrast. The experiment explored three different lightweight networks, namely ShuffleNetv2, MobileNetv3 and GhostNetv1, to reconstruct the CSPDarknet53 backbone feature extraction network of YOLOv5s. Comparative experimental results indicate that reconstructing the backbone network with MobileNetv3 achieved a better balance between detection accuracy and speed. The introduction of the squeeze excitation attention mechanism and inverted residual structure from MobileNetv3 effectively reduced the complexity of black smoke feature fusion. Simultaneously, a novel convolution module, GSConv, was introduced to enhance the expression capability of black smoke features in the neck network. The combination of depthwise separable convolution and standard convolution in the module further reduced the model’s parameter count. After the improvement, the parameter count of the model is compressed to 1/6 of the YOLOv5s model. The lightweight vehicle black smoke real-time detection network, MGSNet, achieved a detection speed of 44.6 frames per second on the test set, an increase of 18.9 frames per second compared with the YOLOv5s model. The mAP@0.5 still exceeded 95%, meeting the application requirements for real-time and accurate detection of vehicle black smoke.

Tackling the tackle 2: Evaluation of referee and player behavioural change as measures of implementation of a law variation in community-level male amateur rugby union

ObjectivesTo evaluate player and referee behaviour during a lower tackle height law variation trial in community rugby union (‘rugby’). DesignProspective observational cohort study. MethodsIn a law variation trial in male amateur community rugby, coded match video surveillance data were analysed. Referee (sanctioning rate) and player (tackler body position) behaviour changes over one season (under the lowered, armpit-level maximum legal tackle height condition) were analysed in three approximately equal periods of the season. Secondarily, an independent professional referee reviewed illegal high tackle sanctioning data. ResultsOverall, 108 matches with 14,780 tackles were filmed and coded. Sanctioned illegal high tackle propensity was significantly higher in the mid-season (41 sanctioned high tackles/1000 tackle events; 95 % CI: 35–47), compared with first and last periods. Upright tacklers in tackles decreased significantly in the final vs. middle period of the season (rate ratio: 0.69; 95 % CI: 0.54–0.88; p < 0.01). Of all the coder-determined high tackles also assessed as high under the new law by the independent referee, 51 % were sanctioned by the on-field referee. ConclusionsPositive player and referee behavioural changes were observed during a lowered legal tackle height law variation in this community rugby setting. Increased mid-phase high tackle sanctioning by referees was followed by fewer tackles with upright tacklers in the subsequent (last) phase of the season. Encouraging positive behaviour changes of this nature, particularly if sustained (beyond trial study periods), may contribute to overall injury risk reduction, and hold considerable importance to inform future injury prevention strategies in rugby.

MACHINE LEARNING-BASED FACIAL RECOGNITION FOR VIDEO SURVEILLANCE SYSTEMS

Video surveillance systems play a crucial role in ensuring public safety and security. However, the traditional methods of surveillance often fall short in effectively identifying individuals, particularly in crowded or dynamic environments. This research addresses the limitations of conventional video surveillance by proposing a machine learning-based facial recognition system. The increasing demand for robust security measures necessitates the development of advanced technologies in video surveillance. Facial recognition has emerged as a promising solution, but existing systems struggle with accuracy and efficiency. This research aims to bridge these gaps by leveraging machine learning techniques for facial recognition in video surveillance. Conventional video surveillance struggles with accurate and rapid identification of individuals, leading to potential security lapses. This research addresses the challenge of enhancing facial recognition accuracy in real-time video feeds, especially in scenarios with varying lighting conditions and occlusions. While facial recognition has gained traction, there is a significant research gap in the implementation of machine learning algorithms tailored for video surveillance. This study aims to fill this void by proposing a novel methodology that combines deep learning and computer vision techniques for robust facial recognition in dynamic environments. The proposed methodology involves training a deep neural network on a diverse dataset of facial images to enable the model to learn intricate facial features. Additionally, computer vision algorithms will be employed to handle challenges such as occlusions and varying lighting conditions. The model''''s performance will be evaluated using real-world video surveillance data. Preliminary results demonstrate a significant improvement in facial recognition accuracy compared to traditional methods. The machine learning-based system exhibits enhanced performance in challenging scenarios, showcasing its potential for practical implementation in video surveillance systems.

New method for video stream brightness stabilization: algorithms and performance evaluation

Subject of study. In this paper, for the first time, an original method for estimating the change in the brightness of video data under the influence of changes in the lighting conditions of the scene and external noise is proposed. Algorithms for stabilizing the brightness of video data are also proposed. An objective assessment of the quality of video data pre-processed is given. The purpose of the research is to create a methodology for analyzing the variability of video data parameters under the influence of negative factors and to develop effective algorithms for stabilizing the parameters of the received video stream. The reliability of the method is tested using real video recordings pictured through various conditions. Objectives: To determine the most universal, resistant to external influences, and informative indicator necessary for an objective assessment of the quality of video data under various shooting conditions and scene lighting features; develop and programmatically implement algorithms for stabilizing video parameters based on modern programming tools. Research methods. Statistical analysis and pre-processing of video stream parameters as a random spatio-temporal process, algorithms for processing video data by digital filtering, and adaptive stabilization of video stream parameters. Research results. It has been proposed and experimentally proven that the optimal indicator of video stream quality is the average frame brightness (AFB). An algorithm for spatiotemporal processing of video data is proposed that generates a sequence of AFB values from the original video stream. The paper also proposes digital algorithms for filtering and stabilizing the brightness of a video stream and investigates the effectiveness of their application. Conclusions. The scientific novelty of the results obtained lies in a new method for analyzing and evaluating the parameters of video surveillance data and algorithms for filtering and stabilizing the brightness of the video stream. The performance of the proposed algorithms has been tested on real data. The algorithms are implemented in the Python software environment using the functions of the OpenCV library.

Video Blockchain: A Decentralized Approach for Secure and Sustainable Networks with Distributed Video Footage from Vehicle-Mounted Cameras in Smart Cities

In this paper, we explore video blockchain for establishing connectivity among vehicles in a smart city through utilizing blockchain technology. By leveraging intelligent vehicular systems that provide location-based visualization through multiple deployed cameras in vehicles, we expand the scope of collecting video surveillance data for observation, thereby enhancing overall situational awareness. We utilize the decentralized nature of blockchain to implement a vehicle-based surveillance system across a smart city. To ensure reliability, the integration of two cryptographic functions, hashing and signing, with the blockchain is employed. This integration ensures secure and tamper-proof solutions for the existing intelligent surveillance system. In this paper, our primary focus is on combining blockchain technology to achieve sustainable and robust smart solutions for intelligent vehicular distributed video networks while eliminating the need for third-party intermediaries. Through extensive experiments and analysis, we demonstrate the effectiveness and feasibility of our proposed video blockchain approach. The results indicate that this innovative framework provides enhanced security, privacy, and scalability for intelligent vehicular distributed networks in smart cities, paving the way for a connected and efficient urban environment.

AD-Graph: Weakly Supervised Anomaly Detection Graph Neural Network

The main challenge faced by video-based real-world anomaly detection systems is the accurate learning of unusual events that are irregular, complicated, diverse, and heterogeneous in nature. Several techniques utilizing deep learning have been created to detect anomalies, yet their effectiveness on real-world data is often limited due to the insufficient incorporation of motion patterns. To address these problems and enhance the traditional functionality of anomaly detection systems for surveillance video data, we propose a weakly supervised graph neural-network-assisted video anomaly detection framework called AD-Graph. To identify temporal information from a series of frames, we extract 3D visual and motion features and represent these in a language-based knowledge graph format. Next, a robust clustering strategy is applied to group together meaningful neighbourhoods of the graph with similar vertices. Furthermore, spectral filters are applied to these graphs, and spectral graph theory is used to generate graph signals and detect anomalous events. Extensive experimental results over two challenging datasets, UCF-Crime and ShanghaiTech, show improvements of 0.35% and 0.78% against a state-of-the-art model.

Data Exchange Mechanism for Real-Time Object Detection in Cloud-Edge IoT System

Real-time target detection based on massive surveillance video data plays a key role in the smart grid, especially in power vision applications. With the indepth construction of power Internet of Things, centralized cloud applications pose challenges to real-time applications due to high bandwidth costs, resulting in delays and loads. Edge computing attempts to perform all the tasks on the edge. However, limited resources on the edge generally cannot guarantee real-time tasks, while complex intelligence algorithms are difficult to fully deploy. Given that both cloud computing solution and edge-only computing solution cannot achieve a good effect, this paper proposes a data exchange mechanism for real-time object detection in a cloud-edge IoT system, which achieves a balance of object detection task executing between delay and accuracy by utilizing the designed intelligent task scheduler. Experimental results performed in real environments using real surveillance video datasets show that by using our cloud-edge collaboration mechanism, the task response time of the system is only about 12% of cloud computing solution while ensuring query accuracy within the practical scope. The system also improves the query accuracy by about 59% compared to edge computing solutions, while the task response time was reduced to its 72%.

Ontology-based detection and identification of complex event of illegal parking using SPARQL and description logic queries

Dynamic monitoring and management of traffic is a significant challenge which requires a mobile surveillance system which is becoming feasible with the evolution of 5G and related technologies. With the advent of 5G, huge amount of data can be transferred over long distance. Furthermore, the evolution of Multi-access Edge Computing (MEC) offers the processing near the edge, resulting in quick response which can cater to mobile surveillance. The mobile surveillance can be performed through a vehicle (Unmanned Aerial Vehicle (UAV)/mobile vehicle) which collects the multi-modal data and sends to nearest edge server over the 5G network. The server sends response quickly to the control center from where the appropriate action can be taken to mitigate its effect on the individuals in real-time. Furthermore, the analysis of multi-modal data captured by surveillance systems must be integrated and efficiently represented. Therefore, in this work, an ontology-based approach is proposed by integrating unstructured video data and structured sensor data for detecting and identifying the complex event of illegal parking. Identifying illegally parked vehicles needs dynamic and strict monitoring; which require the deployment of surveillance systems across the entire city. However, installing a video camera at each place is not feasible. Therefore, the surveillance thorough mobile vehicle is presented to identify illegal parked vehicle for traffic management over the 5G network. In addition, an ontology is developed to represent and reason over the roadside video data while integrating it with sensor data to form a knowledge graph. The knowledge graph retrieves semantic information about the events using SPARQL queries and Description Logics (DL) queries. Thus, a complete video is not required and the extracted information is stored in Resource Description Framework (RDF) format to reduce space requirement significantly. The approach is evaluated by recognizing the complex events of detecting illegally parked vehicles on a real-time dataset recorded using the experimental setup. Furthermore, the framework represents the surveillance video data of parked vehicles captured using the camera installed in a mobile vehicle to cover the road, including the vehicle information location, position, time, etc., as a knowledge graph that can be used later for further analysis.