Surveillance Video Quality Research Articles

Stochastic joint rate control and resource allocation for wireless video surveillance

Wireless video surveillance has become increasingly prevalent due to the rapid development of cellular communication technology and the low deployment cost of cellular surveillance cameras. However, it is still a challenging task to deliver high-quality low-latency surveillance videos in resource-limited cellular networks due to the fluctuating characteristics of wireless channels. In this paper, we consider a wireless video surveillance system consisting of one base station and multiple cellular surveillance cameras, which upload their videos to the base station. To optimize the long-term system performance in terms of surveillance video quality and latency, we formulate a stochastic optimization problem that jointly considers video rate control and uplink resource allocation. The formulation captures the tradeoff between video quality and delivery latency while considering the different time scales of video rate control and channel resource allocation. With the application of Lyapunov optimization, we transform the problem into a sequence of per-slot deterministic problems, and propose an online joint rate control and resource allocation algorithm to solve these per-slot problems. Specifically, the proposed algorithm decomposes each of the per-slot problems into two independent subproblems, namely, the rate control subproblem at large time scale and the uplink resource allocation subproblem at small time scale. We derive the closed-form solution for the rate control subproblem, and design a fast-convergent iterative algorithm to solve the uplink resource allocation subproblem optimally. We prove the asymptotical optimality of our proposed algorithm. Extensive simulation results validate our theoretical analysis and demonstrate that the proposed algorithm can well balance the surveillance video quality and delivery latency.

Self-Controllable Super-Resolution Deep Learning Framework for Surveillance Drones in Security Applications

This paper proposes a self-controllable super-resolution adaptation algorithm in drone platforms. The drone platforms are generally used for surveillance in target network areas. Thus, super-resolution algorithms which are for enhancing surveillance video quality are essential. In surveillance drone platforms, generating video streams obtained by CCTV cameras is not static, because the cameras record the video when abnormal objects are detected. The generation of streams is not predictable, therefore, this unpredictable situation can be harmful to reliable surveillance monitoring. To handle this problem, the proposed algorithm designs superresolution adaptation. With the proposed algorithm, the shallow model which is fast and low-performance will be used if the stream queue is near overflow. On the other hand, the deep model which is highperformance and slow will be used if the queue is idle to improve the performance of super-resolution.

Training configuration analysis of a convolutional neural network object tracker for night surveillance application

<span lang="EN-US">Automated surveillance during the night is important as it is the period when crimes usually happened. By providing continuous monitoring, coupled with a real-time alert system, appropriate action can be taken immediately if a crime is detected. However, low lighting conditions during the night can degrade the quality of surveillance videos, where the captured images will have low contrast and less discriminative features. Consequently, these factors contribute to the problem of bad appearance representation of the object of interest in the tracking algorithm. Thus, a convolutional neural network-based object tracker for night surveillance is proposed by exploiting the deep feature strength in representing object features spatially and semantically. The proposed convolutional network consists of six layers that consist of three convolutional neural networks (CNN) and three fully connected (FC) layers. The network will be trained by using a binary classifier approach of objects and its background classes, which is updated on a fixed interval so that it fully encapsulates the changes in object appearance as it moves in the scene. The algorithm has been tested with different sets of training data configurations to find the best optimum ones with regards to VOT2015 evaluation protocols, tested on 14-night surveillance videos. The results show that the configuration of a total of 250 training samples with a sample ratio of 4:1 between positive and negative data delivers the best performance for the sequence length of [1,550]. It can be inferred that more information on the object is required compared to the background, where the background might be homogeneous due to low lighting conditions. In conclusion, this algorithm is suitable to be implemented for night surveillance application.</span>

How to Assess the Quality of Compressed Surveillance Videos Using Face Recognition

Video surveillance plays an important role in public security. To store the growing volume of surveillance videos, video compression is beneficial for reducing video volume; however, it is simultaneously harmful to the video quality. Video quality assessment (VQA) methods help to achieve a tradeoff between the data volume and perceptual quality of compressed surveillance videos. Generally speaking, surveillance video quality assessment (SVQA) is different from conventional VQA, because surveillance videos are usually used for specific tasks, e.g., pedestrian recognition, rather than for entertainment purposes. Therefore, in this paper, we propose two full-reference SVQA methods based on the concept of quality of recognition. We first design two new tasks, distorted face verification (DFV) and distorted face identification (DFI), based on which we further propose two SVQA methods, DFV-SVQA and DFI-SVQA, and corresponding quality metrics. The core components of the DFV-SVQA and DFI-SVQA methods are feature extractors (a DFV model and a DFI model), which we construct using convolutional-neural-network-based face recognition models. In addition, we construct a real-world surveillance video data set, based on which we analyze how various factors, including the video codec, compression level, face resolution, and light intensity, affect the quality of compressed surveillance videos. We find that, compared with conventional VQA methods, our methods are more effective in measuring the quality of surveillance videos while maintaining an acceptable time efficiency.

Image Enhancement for Surveillance Video of Coal Mining Face Based on Single-Scale Retinex Algorithm Combined with Bilateral Filtering

Surveillance videos of coal mining faces have close relation to the safety of coal miners and mining efficiency. However, surveillance videos are always disturbed by some severe conditions such as atomization, low illumination, glare, and so on. Therefore, this paper proposed a hybrid algorithm (SSR-BF) based on the integration of single-scale Retinex (SSR) and bilateral filtering (BF) to enhance the image quality of surveillance videos. BF was coupled with SSR to reduce the noises and perfect the edge information in the image. The schematic diagram and pseudocode of SSR-BF was designed, and the parameters were set rationally to ensure the enhancement effects through some simulations. Finally, some comparisons with other methods were carried out, and the simulation results demonstrated that the proposed algorithm was superior to others and could be applied to image enhancement for poormonochrome images, especially the surveillance video of a coal mining face.

A Super-resolution Reconstruction Algorithm for Surveillance Video

Recent technological developments have resulted in surveillance video becoming a primary method of preserving public security. Many city crimes are observed in surveillance video. The most abundant evidence collected by the police is also acquired through surveillance video sources. Surveillance video footage offers very strong support for solving criminal cases, therefore, creating an effective policy, and applying useful methods to the retrieval of additional evidence is becoming increasingly important. However, surveillance video has had its failings, namely, video footage being captured in low resolution (LR) and bad visual quality. In this paper, we discuss the characteristics of surveillance video and describe the manual feature registration – maximum a posteriori – projection onto convex sets to develop a super-resolution reconstruction method, which improves the quality of surveillance video. From this method, we can make optimal use of information contained in the LR video image, but we can also control the image edge clearly as well as the convergence of the algorithm. Finally, we make a suggestion on how to adjust the algorithm adaptability by analyzing the prior information of target image.

Research of Useful Algorithms to Improve Surveillance Video Quality

Research of Useful Algorithms to Improve Surveillance Video Quality

Frame Interpolation Based on Visual Correspondence and Coherency Sensitive Hashing

The technology of frame interpolation can be applied in intelligent monitoring systems to improve the quality of surveillance video. In this paper, a region-guided frame interpolation algorithm is proposed by introducing two innovative improvements. On the one hand, a detection approach is presented based on visual correspondence for detecting the motion regions that correspond to attracted objects in video sequences, which can narrow the prediction range of interpolated frames. On the other hand, spatial and temporal mapping rules are proposed using coherency sensitive hashing, which can obtain more accurate predicted values of interpolated pixels. Experiments show that the proposed method can achieve encouraging performance in terms of visual quality and quantitative measures.

A two-step approach to see-through bad weather for surveillance video quality enhancement

Adverse weather conditions such as snow, fog or heavy rain greatly reduce the visual quality of outdoor surveillance videos. Video quality enhancement can improve the visual quality of surveillance videos providing clearer images with more details to better meet human perception needs and also improve video analytics performance. Existing work in this area mainly focuses on the quality enhancement for high-resolution videos or still images, but few algorithms are developed for enhancing surveillance videos, which normally have low resolution, high noises and compression artifacts. In addition, for snow or rain conditions, the image quality of near-field view is degraded by the obscuration of apparent snowflakes or raindrops, while the quality of far-field view is degraded by the obscuration of fog-like snowflakes or raindrops. Very few video quality enhancement algorithms have been developed to handle both problems. In this paper, we propose a novel video quality enhancement algorithm for see-through snow, fog or heavy rain. Our algorithm not only improves human visual perception experiences for video surveillance, but also reveal more video contents for better video content analyses. The proposed algorithm handles both near-field and far-field snow/rain effects by proposed a two-step approach: (1) the near-field enhancement algorithm identifies obscuration pixels by snow or rain in the near-field view and removes these pixels as snowflakes or raindrops; different from state-of-the-art methods, our proposed algorithm in this step can detect snowflakes on foreground objects or background, and apply different methods to fill in the removed regions. (2) The far-field enhancement algorithm restores the image’s contrast information not only to reveal more details in the far-field view, but also to enhance the overall image’s quality; in this step, the proposed algorithm adaptively enhances the global and local contrast, which is inspired on the human visual system, and accounts for the perceptual sensitivity to noises, compression artifacts, and the texture of image content. From our extensive testing, the proposed approach significantly improves the visual quality of surveillance videos by removing snow/fog/rain effects.

Video Surveillance Over Wireless Sensor and Actuator Networks Using Active Cameras

Although there has been much work focused on the camera control issue on keeping tracking a target of interest, few has been done on jointly considering the video coding, video transmission, and camera control for effective and efficient video surveillance over wireless sensor and actuator networks (WSAN). In this work, we propose a framework for real-time video surveillance with pan-tilt cameras where the video coding and transmission as well as the automated camera control are jointly optimized by taking into account the surveillance video quality requirement and the resource constraint of WSANs. The main contributions of this work are: i) an automated camera control method is developed for moving target tracking based on the received surveillance video clip in consideration of the impact of video transmission delay on camera control decision making; ii) a content-aware video coding and transmission scheme is investigated to save network node resource and maximize the received video quality under the delay constraint of moving target monitoring. Both theoretical and experimental results demonstrate the superior performance of the proposed optimization framework over existing systems.