Video Surveillance Data Research Articles

On the crack characterization of reinforced concrete structures: Experimental and data-driven numerical study

In today’s rapid infrastructure development, reinforced concrete (RC) construction accounts for majority of the total residential, commercial and industrial projects. These RC structures, once constructed, deteriorates significantly during their service life. The deterioration of RC structural elements increases mainly with the ageing and sustained loading on the structure and depends on various parameters. One of the primary parameter to access the serviceability of an RC structure is cracking. In this work, an experimental and a numerical study is carried out on the flexural crack behaviour of reinforced concrete beam members. The experimental investigation is focused on the effect of flexural crack by varying the percentage of tensile steel on beam sections. Later, a computer vision-based data-driven numerical tool for crack representation and quantification is developed and validated based on the real-time spatio-temporal video surveillance data of the flexural testing on beams. The developed numerical tool based on the experimental study can be used by the structural designers and practitioners for a robust serviceability-based design of reinforced concrete members and systems.

WON-OCDMA System Based on MW-ZCC Codes for Applications in Optical Wireless Sensor Networks

The growing demand for extensive and reliable structural health monitoring resulted in the development of advanced optical sensing systems (OSS) that in conjunction with wireless optical networks (WON) are capable of extending the reach of optical sensing to places where fibre provision is not feasible. To support this effort, the paper proposes a new type of a variable weight code called multiweight zero cross-correlation (MW-ZCC) code for its application in wireless optical networks based optical code division multiple access (WON-OCDMA). The code provides improved quality of service (QoS) and better support for simultaneous transmission of video surveillance, comms and sensor data by reducing the impact of multiple access interference (MAI). The MW-ZCC code’s power of two code-weight properties provide enhanced support for the needed service differentiation provisioning. The performance of this novel code has been studied by simulations. This investigation revealed that for a minimum allowable bit error rate of , and when supporting triple-play services (sensing, datacomms and video surveillance, respectively), the proposed WON-OCDMA using MW-ZCC codes could support up to 32 simultaneous services over transmission distances up to 32 km in the presence of moderate atmospheric turbulence.

Recognizing human behaviors from surveillance videos using the SSD algorithm

The aim is to better recognize human behaviors from surveillance videos. Human behavior recognition technology based on surveillance videos is researched, given the intellectual development of massive surveillance video data with full coverage. This technology builds a human behavior detection and recognition model using the new Single Shot MultiBox Detector (SSD) algorithm, which improves the recognition accuracy. The constructed model’s effectiveness is verified through comparisons with other traditional human behavior recognition algorithms via the TensorFlow framework. Results demonstrate the SSD model-based recognition algorithm’s accuracy is significantly higher than that of Direct Part Marking and Fast Convolutional Neural Network (CNN) algorithms. SSD’s average speed is 0.146 s/frame, and the average accuracy on different datasets is 82.8%. If the target is close or partially occluded, the SSD algorithm can also accurately detect the central target, and the detection efficiency is twice that of the R-CNN algorithm. The algorithm proposed has a simple structure and fast processing speed, which can solve the problems in target detection. The research results can provide a theoretical basis for the research on target detection related to human behavior recognition.

Intelligent Power Grid Video Surveillance Technology Based on Efficient Compression Algorithm Using Robust Particle Swarm Optimization

Companies that produce energy transmit it to any or all households via a power grid, which is a regulated power transmission hub that acts as a middleman. When a power grid fails, the whole area it serves is blacked out. To ensure smooth and effective functioning, a power grid monitoring system is required. Computer vision is among the most commonly utilized and active research applications in the world of video surveillance. Though a lot has been accomplished in the field of power grid surveillance, a more effective compression method is still required for large quantities of grid surveillance video data to be archived compactly and sent efficiently. Video compression has become increasingly essential with the advent of contemporary video processing algorithms. An algorithm’s efficacy in a power grid monitoring system depends on the rate at which video data is sent. A novel compression technique for video inputs from power grid monitoring equipment is described in this study. Due to a lack of redundancy in visual input, traditional techniques are unable to fulfill the current demand standards for modern technology. As a result, the volume of data that needs to be saved and handled in live time grows. Encoding frames and decreasing duplication in surveillance video using texture information similarity, the proposed technique overcomes the aforementioned problems by Robust Particle Swarm Optimization (RPSO) based run-length coding approach. Our solution surpasses other current and relevant existing algorithms based on experimental findings and assessments of different surveillance video sequences utilizing varied parameters. A massive collection of surveillance films was compressed at a 50% higher rate using the suggested approach than with existing methods.

Abnormal Detection in Big Data Video with an Improved Autoencoder.

With the rapid growth of video surveillance data, there is an increasing demand for big data automatic anomaly detection of large-scale video data. The detection methods using reconstruction errors based on deep autoencoders have been widely discussed. However, sometimes the autoencoder could reconstruct the anomaly well and lead to missing detections. In order to solve this problem, this paper uses a memory module to enhance the autoencoder, which is called the memory-augmented autoencoder (Memory AE) method. Given the input, Memory AE first obtains the code from the encoder and then uses it as a query to retrieve the most relevant memory items for reconstruction. In the training phase, the memory content is updated and encouraged to represent prototype elements of normal data. In the test phase, the learned memory elements are fixed, and reconstruction is obtained from several selected memory records of normal data. So, the reconstruction will tend to be close to normal samples. Therefore, the reconstruction of abnormal errors will be strengthened for abnormal detection. The experimental results on two public video anomaly detection datasets, i.e., Avenue dataset and ShanghaiTech dataset, prove the effectiveness of the proposed method.

A Fast Encryption Scheme Suitable for Video Surveillance Applications Using SHA-256 Hash Function and 1D Sine–Sine Chaotic Map

This paper proposes an image encryption technique which is fast and secure. The encryption scheme is designed for secure transmission of video surveillance data (keyframes) over insecure network. The image encryption technique employs 1D Sine–Sine system with better chaotic properties than its seed map and faster than higher-dimensional chaotic systems. Further, design of encryption scheme is based on two permutation rounds, which employs pixel swapping operation and diffusion operation which is simple and provides required security against plaintext, differential and various other attacks. Three separate chaotic sequences are generated using 1D Sine–Sine system which enhances the key space of the encryption scheme. Secret keys are updated dynamically with SHA-256 hash value obtained from plain image. Hash values of plain image are efficiently used without loss of any hash value information. This makes the encryption scheme plaintext sensitive and secure against plaintext attacks. Performance and security aspects of encryption scheme is analyzed both quantitatively using predefined security metrics and qualitatively by scrutinizing the internal working of encryption scheme. Computational complexity of encrypting a plain image of size [Formula: see text] is [Formula: see text] and is suitable for encrypting keyframes of video for secure surveillance applications.

Audio and video recordings as a type of electronic evidence in civil proceedings

The scientific article presents the results of the study of the procedural features of the collection, study and evaluation of audio and video recordings as court evidence in civil proceedings. The process of formation and perception of audio and video recordings as court evidence is studied, their role and significance in the process of court evidence are determined. The prevailing in the legal literature, the views of scholars on the affiliation of audio and video recordings to a particular type of evidence, in particular as a type of written evidence, as a type of physical evidence, as a separate independent type of evidence.The issue of using audio and video recordings as evidence in the process of judicial evidence in civil cases, in particular the question of the legality of its receipt by the party in the case leads to doubts about the admissibility of using these materials as court evidence. It is established that the objection of a party to the trial against the use of audio or video recordings made without their permission as evidence in a particular case, with reference to the violation of their right to respect for private life, raises the question of the admissibility of such evidence. The procedure for the use of video recordings obtained as a result of video surveillance at the workplace and sound recordings of business negotiations in the process of judicial evidence has been established. Some aspects of the legality of obtaining a video of an offense for submission to the court as evidence are explained.The legal positions of the European Court of Human Rights and the Constitutional Court of Ukraine on the possibility of using video surveillance data in the workplace in the process of judicial evidence are studied and it is established that business, professional activity of an individual can be considered as part of his private life. negotiations should be considered an interference with a person's private life. Four circumstances have been identified that the court must establish in determining the admissibility of a video made in the workplace as evidence in court. It is established that the systemic unity of these circumstances allows the court to conclude on the admissibility or inadmissibility of such a video as evidence in court

Artificial Intelligence-Enabled Traffic Monitoring System

Manual traffic surveillance can be a daunting task as Traffic Management Centers operate a myriad of cameras installed over a network. Injecting some level of automation could help lighten the workload of human operators performing manual surveillance and facilitate making proactive decisions which would reduce the impact of incidents and recurring congestion on roadways. This article presents a novel approach to automatically monitor real time traffic footage using deep convolutional neural networks and a stand-alone graphical user interface. The authors describe the results of research received in the process of developing models that serve as an integrated framework for an artificial intelligence enabled traffic monitoring system. The proposed system deploys several state-of-the-art deep learning algorithms to automate different traffic monitoring needs. Taking advantage of a large database of annotated video surveillance data, deep learning-based models are trained to detect queues, track stationary vehicles, and tabulate vehicle counts. A pixel-level segmentation approach is applied to detect traffic queues and predict severity. Real-time object detection algorithms coupled with different tracking systems are deployed to automatically detect stranded vehicles as well as perform vehicular counts. At each stage of development, interesting experimental results are presented to demonstrate the effectiveness of the proposed system. Overall, the results demonstrate that the proposed framework performs satisfactorily under varied conditions without being immensely impacted by environmental hazards such as blurry camera views, low illumination, rain, or snow.

CenterNet-Triplets application: surveillance camera illegal management detection

There are two types of illegal business in city management: out-of-store management and illegal mobile management, we simply refer to as outmanagement and flowmagement. The main way to deal with illegal buessiness is manual management. However, this requires a lot of costs, and it is impossible for city management personnel to conduct 24hour inspections. In order to improve management efficiency and save labor costs, this article uses city surveillance cameras, combined with deep learning object detection methods, to conduct real-time detection of illegal operations. Aiming at the situation where the amount of surveillance video data is small, a data enhancement method is proposed to improve the detection effect. CenterNet-Triplets is a object detection algorithm without anchor boxes. Its backbone network is Hourglass Network. The AP of CenterNet-Triplets on MS-COCO dataset is 47.0%, and it has a good detection speed. Experimental results show that CenterNet-Triplets is suitable as a method for detecting illegal operations in city management.

Machine Learning to Identify Metabolic Effects of Chlorine Gas Exposure

Background/Aim: Untargeted metabolomics enable the exploration of pathways that are affected by environmental respiratory exposures. However, traditional analyses of such data analyze each feature separately, ignoring interactions between features and reducing power. Independent component analysis (ICA) is a machine learning method that uncovers hidden patterns in data and has found widespread application in the analysis of medical imaging, video surveillance, and financial data. We propose that applying ICA to untargeted metabolomics data can alleviate the limitations of traditional feature-by-feature analyses. Methods: We applied ICA to approximately 25,000 untargeted metabolomic features measured in brachial lavage fluid (BALF) collected from 15 healthy participants who experienced separate exposures to Cl2 gas and clean air. We estimated a total of 7 components using ICA. We compared our results to traditional feature-by-feature analyses. Results: In the traditional analyses, no features remained significant after a false discovery rate (FDR) correction. Of the 7 ICA components, 3 showed statistically significant differences, assessed using paired t-tests, in expression between the two conditions (Cl vs. clean air). These three components (t-statistics: 2.4, 7.2, and 12.2) remain significant after FDR correction. Annotation of detected metabolites in these components suggests variations in oxidative stress-related metabolites, including methionine (an important anti-oxidant) and oxidized fatty acids in the Cl¬2 exposure compared with the clean air exposure. Other features in these components showed a negative mass defect, which is consistent with the presence of halogenated compounds that may form through chlorine nucleophilic substitution. These ICA components were not associated with the age or sex of the participants. Conclusions: Our results suggest Cl2 exposure influences the lung metabolome and demonstrate the value of using machine learning methods to uncover biological response to respiratory exposures. Disclaimer: The views in this abstract belong to the authors and are not necessarily those of the U.S. Environmental Protection Agency.

Spatiotemporal retrieval of dynamic video object trajectories in geographical scenes

AbstractCurrent studies on video trajectory retrieval focus on the retrieval and analysis of image content, neglecting the gap between the spatiotemporal continuity of retrieval conditions and the spatiotemporal discontinuity of multi‐camera video trajectories. In this study, we propose a method for the spatiotemporal retrieval of dynamic video object trajectories in geographic scenes. Based on the camera calibration, the proposed method organizes the scene, cameras, and trajectories, constructs the spatiotemporal constraints, and queries the trajectories using two measures: camera‐by‐camera retrieval and global trajectory retrieval. The proposed method was verified through experiments, and the results demonstrate that both measures can query trajectories effectively and reduce the spatiotemporal video review range under different spatiotemporal constraints. Furthermore, compared with camera‐by‐camera retrieval, global trajectory retrieval can reduce the spatiotemporal video review range further and return more accurate results. The proposed method may provide support for the spatial analysis and understanding of surveillance video data.

Preventing Animal-Vehicle Crashes using a Smart Detection Technology and Warning System

Animal-vehicle collisions (AVCs), and deer-vehicle collisions (DVCs) in particular, are a major safety problem on roads in Virginia, United States. Mitigation measures such as improved fencing and location-specific driver alerts are being implemented and evaluated in Virginia and elsewhere. One of the most promising mitigation methods uses a buried cable animal detection system (BCADS) to provide roadside or in-vehicle warnings to approaching drivers based on the active presence of an animal on or near the roadway. In this study, the Virginia Department of Transportation (VDOT) in collaboration with the Virginia Tech Transportation Institute (VTTI) implemented and monitored the performance of a BCADS on a public road to provide a real-world assessment of system capabilities and possible operation issues. Concurrently, a flashing light “Deer Crossing” warning sign was installed at the site and linked with the BCADS to alert approaching drivers when an animal crossing was detected. Continuous video surveillance data were collected for a year to monitor animal movement, vehicle traffic, and system performance. Study findings indicate that the BCADS is capable of detecting larger animals such as deer, and sometimes smaller animals such as coyotes, with almost 99% reliability. The system also performed well when covered by approximately 0.6 m (2 ft) of snow. Vehicle speed and brake light application data collected during warning sign activation showed that approximately 80% of drivers either braked or slowed in response, indicating that the sign was effective. A 75% reduction in deer-vehicle crashes was observed in the first year after system deployment.

An Accurate Matching Method for Projecting Vector Data into Surveillance Video to Monitor and Protect Cultivated Land

The integration of intelligent video surveillance and GIS (geograhical information system) data provides a new opportunity for monitoring and protecting cultivated land. For a GIS-based video monitoring system, the prerequisite is to align the GIS data with video image. However, existing methods or systems have their own shortcomings when implemented in monitoring cultivated land. To address this problem, this paper aims to propose an accurate matching method for projecting vector data into surveillance video, considering the topographic characteristics of cultivated land in plain area. Once an adequate number of control points are identified from 2D (two-dimensional) GIS data and the selected reference video image, the alignment of 2D GIS data and PTZ (pan-tilt-zoom) video frames can be realized by automatic feature matching method. Based on the alignment results, we can easily identify the occurrence of farmland destruction by visually inspecting the image content covering the 2D vector area. Furthermore, a prototype of intelligent surveillance video system for cultivated land is constructed and several experiments are conducted to validate the proposed approach. Experimental results show that the proposed alignment methods can achieve a high accuracy and satisfy the requirements of cultivated land monitoring.

Dynamic prioritization of surveillance video data in real-time automated detection systems

Automated object detection systems are a key component of modern surveillance applications. These systems rely on computationally expensive computer vision algorithms that perform object detection on visual data recorded by surveillance cameras. Due to the security and safety implications of these systems, this visual data st be processed accurately and in real-time. However, many of the frames that are created by the surveillance cameras may be of low importance, providing little or no useful information to the object detection system. Sub-sampling surveillance data by prioritizing important camera frames can greatly reduce unnecessary computation. Consequently, several works have explored dynamic visual data sub-sampling using various modalities of information (ie. spatial or temporal information) for prioritization. Few works, however, have combined and evaluated different modalities of information together for real-time prioritization of visual surveillance data. This work evaluates several individual and combined prioritization metrics derived from different modalities of information for use with a modern deep learning-based object detection algorithm. Both processing time and object detection rate are measured and used to rank the prioritization metrics. A novel approach that uses the historical detection confidences created by the object detection algorithm was demonstrated to be the best standalone prioritization metric. Additionally, a novel ensemble method that uses a KNN regressor to combine the best of the previously evaluated metrics to create a dynamic prioritization method is presented. This ensemble approach is shown to increase the object detection rate by up to 60% as compared to a static sub-sampling baseline as demonstrated using three publicly available datasets. The increased object detection rate was achieved while meeting the real-time constraints of the automated object detection system.

Networked Fault Detection of Field Equipment from Monitoring System Based on Fusing of Motion Sensing and Appearance Information

Recent development of Internet of Things (IoT) technologies has triggered a soaring application of intelligent monitoring systems. A sensitive online fault detection for industrial equipment is of utmost significance to improve industrial intelligence based on video sensor network. In this paper, we propose the Appearance and Motion SVM (AMSVM), an online fault detection method fusing multimodal information based on One-Class SVM (OCSVM), to monitor the working conditions of unsupervised equipment in the field. It utilizes multimodal features to detect faults in terms of the appearance and motion patterns of equipment. The motion pattern was generated using the OCSVM-encoded histogram of optical flow orientation (HOFO), and meanwhile we employed Local Binary Pattern Histogram (LBPH) to extract texture features to train OCSVM, depicting appearance patterns. Then, decision level information (i.e., appearance and motion patterns) are combined to produce a more precise characteristic for fault detection. The proposed method herein was validated on several industrial video surveillance data set.

Multimodal behavior analysis in computer-enabled laboratories using nonverbal cues

In the modern era, there is a growing need for surveillance to ensure the safety and security of the people. Real-time object detection is crucial for many applications such as traffic monitoring, security, search and rescue, vehicle counting, and classroom monitoring. Computer-enabled laboratories are generally equipped with video surveillance cameras in the smart campus. But, from the existing literature, it is observed that the use of video surveillance data obtained from smart campus for any unobtrusive behavioral analysis is seldom performed. Though there are several works on the students’ and teachers’ behavior recognition from devices such as Kinect and handy cameras, there exists no such work which extracts the video surveillance data and predicts the behavioral patterns of both the students and the teachers in real time. Hence, in this study, we unobtrusively analyze the students’ and teachers’ behavioral patterns inside a teaching laboratory (which is considered as an indoor scenario of a smart campus). Here, we propose a deep convolution network architecture to classify and recognize an object in the indoor scenario, i.e., the teaching laboratory environment of the smart campus with modified Single-Shot MultiBox Detector approach. We used six different class labels for predicting the behavioral patterns of both the students and the teachers. We created our dataset with six different class labels for training deep learning architecture. The performance evaluation demonstrates that the proposed method performs better with an accuracy of 0.765 for classification and localization.

HSAJAYA: An Improved Optimization Scheme for Consumer Surveillance Video Synopsis Generation

Video Surveillance is an active area of research and provides promising security measures for consumer applications. To ease consumer surveillance investigations, Video Synopsis (VS) serves as a powerful tool to assess hours of video in a shorter retro of time by projecting multiple objects concurrently. The optimization module in VS framework is considered to be a key module, yet to date, only traditional optimization techniques have been addressed for energy minimization. Amongst these, simulated annealing (SA) has been broadly employed to produce global optimal solution without getting trapped in local minima. However, the convergence time of SA is quite high as the next state is chosen randomly to achieve real-time performance. This article presents an improved energy minimization scheme using hybridization of SA and JAYA algorithm to achieve global optimal solution with faster convergence rate. The weights associated with the energy function are computed using analytic hierarchy process (AHP) instead of heuristic selection. From experimental evaluations and analysis, it is seen that the proposed scheme exhibits superior performance to minimize the overall energy cost with lesser computational time. The proposed scheme has a potential to quickly review consumer surveillance video data in a smart and efficient way.

A novel Gait-Appearance-based Multi-Scale Video Covariance Approach for pedestrian (re)-identification

In order to handle the complex databases of acquired images in the security area, a robust and adaptive framework for Video Surveillance Data Mining as well as for multi-shot pedestrian (re)-identification is required. The pedestrian’s signature must be invariant and robust against the noise and uncontrolled variation. In this paper a new fast Gait-Appearance-based Multi-Scale Video Covariance (GAMS-ViCov) unsupervised approach was proposed to efficiently describe any image-sequence, on streaming or stored in the database, of a pedestrian into a compact and fixed size signature while exploiting the whole relevant spatiotemporal information. The proposed model is based on multi-scale features extracted from a novel data structure called ‘Two-Half-Video-Tree’ (THVT) which represents the pedestrians and allows discarding the uncontrolled variations. THVT can efficiently model the gait and appearance of the upper and lower parts of the person’s silhouette into trees of multi-scale features. THVT can thus model the video data to new structured forms through a fast algorithm. Furthermore, GAMS-ViCov approach can also be competitive as a technique of dynamic video summarization using k-means clustering to model the signatures extracted from the image-sequences of each person into a cluster center. For each person’s cluster, the image-sequence that its signature is nearest to the centroid is selected and stored as the key image-sequence of this person. The proposed approach was evaluated for the person (re)-identification with i-LIDS and PRID databases. The experimental results show that GAMS-ViCov outperforms the most of unsupervised approaches.

Dynamic Spatio-Temporal Graph-Based CNNs for Traffic Flow Prediction

Forecasting future traffic flows from previous ones is a challenging problem because of their complex and dynamic nature of spatio-temporal structures. Most existing graph-based CNNs attempt to capture the static relations while largely neglecting the dynamics underlying sequential data. In this article, we present dynamic spatio-temporal graph-based CNNs (DST-GCNNs) by learning expressive features to represent spatio-temporal structures and predict future traffic flows from surveillance video data. In particular, DST-GCNN is a two stream network. In the flow prediction stream, we present a novel graph-based spatio-temporal convolutional layer to extract features from a graph representation of traffic flows. Then several such layers are stacked together to predict future flows over time. Meanwhile, the relations between traffic flows in the graph are often time variant as the traffic condition changes over time. To capture the graph dynamics, we use the graph prediction stream to predict the dynamic graph structures, and the predicted structures are fed into the flow prediction stream. Experiments on real datasets demonstrate that the proposed model achieves competitive performances compared with the other state-of-the-art methods.

КОМПЛЕКСНЫЕ ИССЛЕДОВАНИЯ АКТИВНОСТИ АВАЧИНСКОГО ВУЛКАНА В 2013–НАЧАЛЕ 2020 гг.: НАЗЕМНЫЕ, ВИДЕО И СЕЙСМИЧЕСКИЕ НАБЛЮДЕНИЯ

The results of monitoring at high-temperature fumarolic fields (Eastern and Western) of Avachinsky volcano in 2013–2020 are presented. Those were visual observations, temperature measurement and gas sampling, supplemented with video surveillance data. In order to identify the relationship between fumarolic activity and deep processes, the analysis of the volcano area seismicity for the specified period was carried out. Two periods of increased seismic activity have been identified. The first period, November 2014 – January 2015, is associated with changes in the character of gas emission, morphology and temperature of Western Fumarole observed in the field, reflected in the ratio of gas fluxes of the two high-temperature fumarolic fields. The second seismic increase, in October – December 2019, was accompanied in the dark by glowing, sometimes by bright flashes in the area of Western Fumarole, and as a result of ground survey in 2020 further changes in its morphology were revealed.