Surveillance Scenarios Research Articles

Realistic frontal face reconstruction using coupled complementarity of far-near-sighted face images

There is still a huge gap in the accuracy of face recognition in public video surveillance scenarios. The far-sighted low-resolution (LR) frontal faces have holistic facial profiles but lack sufficient clearness, while the near-sighted high-resolution (HR) tilted faces show rich facial details yet incomplete facial structure suffering from the overhead self-occlusion of the head blocking the face. Following this observation, this paper proposes a dual-branch HR frontal face reconstruction network to explicitly exploit such coupled complementarity hidden in the far-near face images of the same subject, where one branch performs super-resolution (SR) of the LR frontal face and the other branch performs detail fusion and holistic compensation between multiple HR tilted faces as well as the super-resolved frontal result. In particular, we propose a secondary relevance attention mechanism to enhance the embedding of key features, which sequentially performs rough and precise feature matching and embedding, thus enabling coarse-to-fine progressive compensation. Further, scale-entangled densely connected blocks (SEDCB) are used to gradually integrate the relevance information at different scales (due to the different sighting distances) to promote the information interaction between the features of tilted faces. Besides, we also propose a ternary coupled sample pair (LR far-sighted frontal face, HR near-sighted tilted face, normal ground truth clear face) training scheme to supervise the network optimization. Extensive experimental results on two real-world tilt-view face datasets show that our method can not only reconstruct more realistic HR frontal faces but also facilitate the down-stream face identification task compared with the competing counterparts.

Vehicle Re-Identification with Spatio-Temporal Model Leveraging by Pose View Embedding

Vehicle re-identification (Re-ID) research has intensified as numerous advancements have been made along with the rapid development of person Re-ID. In this paper, we tackle the vehicle Re-ID problem in open scenarios. This research differs from the early-stage studies that focused on a certain view, and it faces more challenges due to view variations, illumination changes, occlusions, etc. Inspired by the research of person Re-ID, we propose leveraging pose view to enhance the discrimination performance of visual features and utilizing keypoints to improve the accuracy of pose recognition. However, the visual appearance information is still limited by the changing surroundings and extremely similar appearances of vehicles. To the best of our knowledge, few methods have been aware of the spatio-temporal information to supplement visual appearance information, but they neglect the influence of the driving direction. Considering the peculiar characteristic of vehicle movements, we observe that vehicles’ poses on camera views indicating their directions are closely related to spatio-temporal cues. Consequently, we design a two-branch framework for vehicle Re-ID, including a Keypoint-based Pose Embedding Visual (KPEV) model and a Keypoint-based Pose-Guided Spatio-Temporal (KPGST) model. These models are integrated into the framework, and the results of KPEV and KPGST are fused based on a Bayesian network. Extensive experiments performed on the VeRi-776 and VehicleID datasets related to functional urban surveillance scenarios demonstrate the competitive performance of our proposed approach.

An Efficient Graph Based Approach for Reducing Coverage Loss From Failed Cameras of a Surveillance Network

The optimal placement of surveillance cameras to maximize the total coverage of camera networks is of significant research interest due to the proliferation of camera networks in diverse application scenarios. Existing camera placement algorithms are mostly designed to optimize coverage. In the event of failure of one or more cameras, these optimization algorithms reorient all the remaining cameras in the surveillance network to regain the lost coverage; and hence are not cost-effective. A simple and straightforward solution to regain the lost coverage is by reorienting the field of view of active cameras to overlap with those of the damaged cameras. In this paper, we propose the Visibility Graph Reduction (VGR) algorithm, a novel graph-based approach to select potential cameras amongst the active cameras, that can optimally alleviate the loss in coverage resulting from the failure of one or more cameras. We validate our algorithm on map images across diverse indoor and outdoor surveillance scenarios with existing camera networks. Experimental results show that our algorithm minimizes the number of cameras that need to be reoriented, and produces similar coverage results as that of reorientating all the available active cameras in the surveillance network. Our approach is both cost-effective and computationally efficient, and thus minimizes the human effort involved in reorienting cameras across practical scenarios.

Intelligent Reflecting Surface-Assisted Proactive Eavesdropping Over Suspicious Broadcasting Communication With Statistical CSI

This paper studies a novel wireless surveillance scenario, where a half-duplex legitimate monitor (E), aided by an intelligent reflecting surface (IRS), aims to eavesdrop the frequency division multiple access based suspicious broadcasting communication from a suspicious transmitter (ST) to multiple suspicious receivers (SRs). Under this setup, instead of eavesdropping the ST over all frequency bands (FBs), E proactively jams over certain <i>undesirable</i> FBs, i.e., jams certain SRs via E-SRs and E-IRS-SRs links, to deliberately force the ST reallocating more transmit power over the unjammed (eavesdroppable) <i>desirable</i> FBs. Then as a reward, E itself can achieve better eavesdropping over those FBs accordingly. Our objective is to maximize the <i>sum ergodic eavesdropping rate</i> of E, by jointly optimizing the set of jammed FBs, the jamming power allocation of E and the reflection coefficients at the IRS. A two-stage optimization procedure is proposed: i) given each possible jammed set, the jamming power allocation and the reflection coefficients are optimized via successive convex approximation and alternative iterations; ii) the optimal and sub-optimal jammed sets are further determined using the exhaustive search and the designed heuristic scheme. Results show the effectiveness of the proposed scheme compared to competitive benchmarks.

Robust Planning for Persistent Surveillance With Energy-Constrained UAVs and Mobile Charging Stations

We consider persistent (long-horizon) surveillance over an environment by using energy-constrained unmanned aerial vehicles (UAVs), which are supported by unmanned ground vehicles (UGVs) serving as mobile charging stations. The goal is to periodically visit a set of monitoring points by the UAVs while minimizing the maximum time between the consecutive visits to any of those points. In general, the optimal planning of UAVs and UGVs in such a persistent surveillance scenario is an NP-hard combinatorial optimization problem. Furthermore, the problem also demands a solution strategy that can successfully handle obstacles, especially on the ground, that are unknown a priori in many real-life scenarios. We present a scalable and robust approximate algorithm that is based on 1) forming uniform UAV-UGV teams, 2) decomposing the environment into maximal partitions that can be covered by the UAVs in a single fuel cycle as long as the UAVs are released sufficiently close to the centers of the partitions, 3) maintaining the teams uniformly distributed over a cyclic path traversing those partitions, and 4) having the UAVs in each team cover their current partition and be transported to the next partition while being recharged by the UGV. We support our proposed algorithm with some theoretical results and simulations.

Discriminative information restoration and extraction for weakly supervised low-resolution fine-grained image recognition

The existing methods of fine-grained image recognition mainly devote to learning subtle yet discriminative features from the high-resolution input. However, their performance deteriorates significantly when they are used for low quality images because a lot of discriminative details of images are missing. We propose a discriminative information restoration and extraction network, termed as DRE-Net, to address the problem of low-resolution fine-grained image recognition, which has widespread application potential, such as shelf auditing and surveillance scenarios. DRE-Net is the first framework for weakly supervised low-resolution fine-grained image recognition and consists of two sub-networks: (1) fine-grained discriminative information restoration sub-network (FDR) and (2) recognition sub-network with the semantic relation distillation loss (SRD-loss). The first module utilizes the structural characteristic of minimum spanning tree (MST) to establish context information for each pixel by employing the spatial structures between each pixel and other pixels, which can help FDR focus on and restore the critical texture details. The second module employs the SRD-loss to calibrate recognition sub-network by transferring the correct relationships between every two pixels on the feature map. Meanwhile the SRD-loss can further prompt the FDR to recover reliable and accurate fine-grained details and guide the recognition sub-network to perceive the discriminative features from the correct relationships. Extensive experiments on three benchmark datasets and one retail product dataset demonstrate the effectiveness of our proposed framework.

Temporal Logic Control Synthesis for Distributed Multi-Agent Cooperative Tasking

In this work, we present a novel control synthesis method for cooperative multi-agent systems to fulfill a global mission given as linear temporal logic on finite traces (LTLf). The proposed method first synthesizes satisfiable global controllers for uncontrolled system, such that the global specification is met; and then, searches for a decomposable global controller among them over a maximum synchronization scheme; finally, further refining the synchronization scheme to obtain decomposed distributed controllers. To search for a decomposable global controller among the satisfiable ones, we present an informed searching algorithm based on the decomposability analysis of global controllers. A multi-UAV cooperative fire surveillance scenario is developed to illustrate the method.

Multiplicatively weighted Voronoi-based sensor collaborative redeployment in software-defined wireless sensor networks

Large-scale deployment of mobile wireless sensor networks has been widely used in some dangerous and hostile urban security surveillance scenarios. As a new network architecture, software-defined networks was introduced into wireless sensor networks to form a new software-defined wireless sensor networks to solve the problem of balanced large-scale deployment of sensor networks and simplify the complexity of network management. In this article, we first develop an original confident information coverage–based multiplicatively weighted Voronoi diagram through sensor clustering and sensor collaborative sensing. And then, we propose two sensor collaborative redeployment algorithms based on the novel confident information coverage–based multiplicatively weighted Voronoi diagram and software-defined wireless sensor networks architecture to provide high-confidence coverage and improve the coverage ratio. Finally, we demonstrate the superiority of the confident information coverage–based multiplicatively weighted Voronoi diagram and the effectiveness and efficiency of our proposed algorithms via a series of experiments.

Nuclear radiation detection based on the convolutional neural network under public surveillance scenarios

Abstract Nuclear energy is a clean and popular form of energy, but leakage and loss of nuclear material pose a threat to public safety. Radiation detection in public spaces is a key part of nuclear security. Common security cameras equipped with complementary metal oxide semiconductor (CMOS) sensors can help with radiation detection. Previous work with these cameras, however, required slow, complex frame-by-frame processing. Building on the previous work, we propose a nuclear radiation detection method using convolution neural networks (CNNs). This method detects nuclear radiation in changing images with much less computational complexity. Using actual video images captured in the presence of a common Tc-99m radioactive source, we construct training and testing sets. After training the CNN and processing our test set, the experimental results show the high performance and effectiveness of our method.

Hybrid Modality Metric Learning for Visible-Infrared Person Re-Identification

Visible-infrared person re-identification (Re-ID) has received increasing research attention for its great practical value in night-time surveillance scenarios. Due to the large variations in person pose, viewpoint, and occlusion in the same modality, as well as the domain gap brought by heterogeneous modality, this hybrid modality person matching task is quite challenging. Different from the metric learning methods for visible person re-ID, which only pose similarity constraints on class level, an efficient metric learning approach for visible-infrared person Re-ID should take both the class-level and modality-level similarity constraints into full consideration to learn sufficiently discriminative and robust features. In this article, the hybrid modality is divided into two types, within modality and cross modality. We first fully explore the variations that hinder the ranking results of visible-infrared person re-ID and roughly summarize them into three types: within-modality variation, cross-modality modality-related variation, and cross-modality modality-unrelated variation. Then, we propose a comprehensive metric learning framework based on four kinds of paired-based similarity constraints to address all the variations within and cross modality. This framework focuses on both class-level and modality-level similarity relationships between person images. Furthermore, we demonstrate the compatibility of our framework with any paired-based loss functions by giving detailed implementation of combing it with triplet loss and contrastive loss separately. Finally, extensive experiments of our approach on SYSU-MM01 and RegDB demonstrate the effectiveness and superiority of our proposed metric learning framework for visible-infrared person Re-ID.

State-of-the-Art Violence Detection Techniques: A review

Surveillance systems are playing a significant role in law enforcement and city safety. It is important to detect violent and suspicious behaviors automatically in video surveillance scenarios, for instance, railway stations, schools, hospitals to avoid any casualties which could cause social, economic, and ecological damage. Automatic detection of violence for quick actions is very significant and can efficiently help law enforcement departments. So, researchers are doing a lot of research on different techniques for detecting violence. This research study reviews various techniques and methods for detecting violent or anomalous activities from surveillance video that have been proposed by many researchers in recent years. The method of detection is divided into three categories. These categories are based on the classification techniques used. These categories are: traditional violence detection using machine learning, Support Vector Machine (SVM) &amp; Deep Learning. Feature extraction &amp; Object detection techniques are also described for each category. Moreover, dataset &amp; video features that help in the recognition process are also discussed. The overall research finding has been discussed which will help the researcher in their future work in this field.

A Framework for Pedestrian Attribute Recognition Using Deep Learning

The pedestrian attribute recognition task is becoming more popular daily because of its significant role in surveillance scenarios. As the technological advances are significantly more than before, deep learning came to the surface of computer vision. Previous works applied deep learning in different ways to recognize pedestrian attributes. The results are satisfactory, but still, there is some scope for improvement. The transfer learning technique is becoming more popular for its extraordinary performance in reducing computation cost and scarcity of data in any task. This paper proposes a framework that can work in surveillance scenarios to recognize pedestrian attributes. The mask R-CNN object detector extracts the pedestrians. Additionally, we applied transfer learning techniques on different CNN architectures, i.e., Inception ResNet v2, Xception, ResNet 101 v2, ResNet 152 v2. The main contribution of this paper is fine-tuning the ResNet 152 v2 architecture, which is performed by freezing layers, last 4, 8, 12, 14, 20, none, and all. Moreover, data balancing techniques are applied, i.e., oversampling, to resolve the class imbalance problem of the dataset and analysis of the usefulness of this technique is discussed in this paper. Our proposed framework outperforms state-of-the-art methods, and it provides 93.41% mA and 89.24% mA on the RAP v2 and PARSE100K datasets, respectively.

Pedestrian Attribute Recognition in Video Surveillance Scenarios Based on View-attribute Attention Localization

Pedestrian attribute recognition in surveillance scenarios is still a challenging task due to the inaccurate localization of specific attributes. In this paper, we propose a novel view-attribute localization method based on attention (VALA), which utilizes view information to guide the recognition process to focus on specific attributes and attention mechanism to localize specific attribute-corresponding areas. Concretely, view information is leveraged by the view prediction branch to generate four view weights that represent the confidences for attributes from different views. View weights are then delivered back to compose specific view-attributes, which will participate and supervise deep feature extraction. In order to explore the spatial location of a view-attribute, regional attention is introduced to aggregate spatial information and encode inter-channel dependencies of the view feature. Subsequently, a fine attentive attribute-specific region is localized, and regional weights for the view-attribute from different spatial locations are gained by the regional attention. The final view-attribute recognition outcome is obtained by combining the view weights with the regional weights. Experiments on three wide datasets (richly annotated pedestrian (RAP), annotated pedestrian v2 (RAPv2), and PA-100K) demonstrate the effectiveness of our approach compared with state-of-the-art methods.

YinYang-Net: Complementing Face and Body Information for Wild Gender Recognition

Soft biometrics inference in surveillance scenarios is a topic of interest for various applications, particularly in security-related areas. However, soft biometric analysis is not extensively reported in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">wild conditions</i> . In particular, previous works on gender recognition report their results in face datasets, with relatively good image quality and frontal poses. Given the uncertainty of the availability of the facial region in wild conditions, we consider that these methods are not adequate for surveillance settings. To overcome these limitations, we: 1) present frontal and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">wild</i> face versions of three well-known surveillance datasets; and 2) propose YinYang-Net (YY-Net), a model that effectively and dynamically complements facial and body information, which makes it suitable for gender recognition in wild conditions. The frontal and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">wild</i> face datasets derive from widely used Pedestrian Attribute Recognition (PAR) sets (PETA, PA-100K, and RAP), using a pose-based approach to filter the frontal samples and facial regions. This approach retrieves the facial region of images with varying image/subject conditions, where the state-of-the-art face detectors often fail. YY-Net combines facial and body information through a learnable fusion matrix and a channel-attention sub-network, focusing on the most influential body parts according to the specific image/subject features. We compare it with five PAR methods, consistently obtaining state-of-the-art results on gender recognition, and reducing the prediction errors by up to 24% in frontal samples. The announced PAR datasets versions and YY-Net serve as the basis for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">wild</i> soft biometrics classification and are available in here.

Improved Anomaly Detection in Surveillance Videos with Multiple Probabilistic Models Inference

Anomaly detection in surveillance videos is an extremely challenging task due to the ambiguous definitions for abnormality. In a complex surveillance scenario, the kinds of abnormal events are numerous and might co-exist, including such as appearance and motion anomaly of objects, long-term abnormal activities, etc. Traditional video anomaly detection methods cannot detect all these kinds of abnormal events. Hence, we utilize multiple probabilistic models inference to detect as many different kinds of abnormal events as possible. To depict realistic events in a scene, the parameters of our methods are tailored to the characteristics of video sequences of practical surveillance scenarios. However, there is a lack of video anomaly detection methods suitable for real-time processing, and the trade-off between detection accuracy and computational complexity has not been given much attention. To reduce high computational complexity and shorten frame processing times, we employ a variable-sized cell structure and extract a compact feature set from a limited number of video volumes during the feature extraction stage. In conclusion, we propose a real-time video anomaly detection algorithm called MPI-VAD that combines the advantages of multiple probabilistic models inference. Experiment results on three publicly available datasets show that the proposed method attains competitive detection accuracies and superior frame processing speed.

Crowd Counting Via Perspective-Guided Fractional-Dilation Convolution

Crowd counting is critical for numerous video surveillance scenarios. One of the main issues in this task is how to handle the dramatic scale variations of pedestrians caused by the perspective effect. To address this issue, this paper proposes a novel convolution neural network-based crowd counting method, termed Perspective-guided Fractional-Dilation Network (PFDNet). By modeling the continuous scale variations, the proposed PFDNet is able to select the proper fractional dilation kernels for adapting to different spatial locations. It significantly improves the flexibility of the state-of-the-arts that only consider the discrete representative scales. In addition, by avoiding the multi-scale or multi-column architecture that used in other methods, it is computationally more efficient. In practice, the proposed PFDNet is constructed by stacking multiple Perspective-guided Fractional-Dilation Convolutions (PFC) on a VGG16-BN backbone. By introducing a novel generalized dilation convolution operation, the PFC can handle fractional dilation ratios in the spatial domain under the guidance of perspective annotations, achieving continuous scales modeling of pedestrians. To deal with the problem of unavailable perspective information in some cases, we further introduce an effective perspective estimation branch to the proposed PFDNet, which can be trained in either supervised or weakly-supervised setting once the branch has been pre-trained. Extensive experiments show that the proposed PFDNet outperforms state-of-the-art methods on ShanghaiTech A, ShanghaiTech B, WorldExpo'10, UCF-QNRF, UCF_CC_50 and TRANCOS dataset, achieving MAE 53.8, 6.5, 6.8, 84.3, 205.8, and 3.06 respectively.

Borrowing Arrows With Thatched Boats: Exploiting The Reactive Primary Communications for Boosting Jamming-Assisted Proactive Eavesdropping

This paper investigates a cognitive information surveillance scenario, where a half-duplex legitimate monitor (E) aims to eavesdrop the frequency division multiple access based suspicious downlink broadcasting communication, which shares the same spectrum with a primary downlink broadcasting system. Specifically, the suspicious transmitter (ST) employs water-filling power allocation over all orthogonal frequency bands (FBs) to maximize the sum suspicious communication rate of all suspicious receivers (SRs), while the primary transmitter (PT) purposely maximizes the minimum communication rate among the primary receivers (PRs). Under this setup, E picks up certain FBs to jam and eavesdrops over the remaining FBs, and intends to control its transmit jamming beamforming over the jammed FBs to deliberately interfere with the corresponding PRs and SRs, such that the PT and the ST will reactively reallocate their power to facilitate E's eavesdropping. Our objective is to maximize the sum instantaneous (ergodic) eavesdropping rate at E, by jointly optimizing its jamming set and the corresponding jamming beamformers. The optimization problem is non-convex and its approximate and sub-optimal solutions are provided by some convex optimization procedures, along with the optimal and greedy jamming set selections. Results show the effectiveness of our proposed jamming-assisted eavesdropping compared to competitive benchmarks.

Study on attitude heading reference system for an Unmanned Aerial Vehicle in an aerial surveillance scenario

Study on attitude heading reference system for an Unmanned Aerial Vehicle in an aerial surveillance scenario

Aggregative feedback optimization for distributed cooperative robotics

In this paper we propose Aggregative Tracking Feedback, i.e., a novel distributed feedback optimization law that steers network systems to a steady state, which is optimal according to an aggregative optimization problem. Aggregative optimization is a recently emerged distributed optimization framework in which the agents of a network minimize the sum of local objective functions. These functions depend both on local and aggregate decision variables (e.g., the barycenter). Motivated by this problem setup, we design a distributed feedback optimization law in which each agent reconstructs information not locally available while concurrently steering the network to an optimal steady state. We perform a system theoretical analysis based on a singular perturbation approach to show that Aggregative Tracking Feedback, in case of strongly convex objective functions, steers the network with a linear convergence rate to the problem minimum. Finally, we show some numerical simulations on a multi-robot surveillance scenario to validate the effectiveness of the proposed method.

CHANGE DETECTION FOR AREA SURVEILLANCE USING A MOVING CAMERA

This paper tackles area surveillance with a moving camera by change detection. None of the existing datasets for change detection meets a surveillance scenario where a camera is mounted on a moving platform and pointed in the direction of moving. Thus, this paper creates a new dataset including several challenging points. For this dataset, this paper employs a composable method and proposes some components. To evaluate the proposed components, some corresponding classic methods were also tested on the dataset. As a result, the proposals outperformed them. Moreover, this paper investigated the relationship between the parameters of the components and their performance.