Visual Surveillance System Research Articles

Optimizing Visual Sensor Coverage Overlaps for Multiview Surveillance Systems

Modern surveillance systems rely on coverage overlapping to attain multi-viewing capabilities and to support coverage redundancy. This paper examines the coverage overlapping configurations in visual surveillance systems. This paper proposes a robust dynamic programming framework to optimize visual surveillance sensor coverage overlaps. Based on visual sensor parameters information, and the features of the area to be monitored, this paper uses a deterministic modeling approach to model the sensor coverage in a 2-D space. Then, the minimization and the maximization arrangements of the coverage overlapping are formulated as discrete optimization problems. The obtained solutions from the dynamic programming technique are evaluated with respect to local and global greedy search algorithms. The results reveal the feasibility of the proposed technique compared with the benchmarked optimization methods in terms of the amount of coverage redundancy.

A scene image classification technique for a ubiquitous visual surveillance system

The concept of smart cities has quickly evolved to improve the quality of life and provide public safety. Smart cities mitigate harmful environmental impacts and offences and bring energy-efficiency, cost saving and mechanisms for better use of resources based on ubiquitous monitoring systems. However, existing visual ubiquitous monitoring systems have only been developed for a specific purpose. As a result, they cannot be used for different scenarios. To overcome this challenge, this paper presents a new ubiquitous visual surveillance mechanism based on classification of scene images. The proposed mechanism supports different applications including Soil, Flood, Air, Plant growth and Garbage monitoring. To classify the scene images of the monitoring systems, we introduce a new technique, which combines edge strength and sharpness to detect focused edge components for Canny and Sobel edges of the input images. For each focused edge component, a patch that merges nearest neighbor components in Canny and Sobel edge images is defined. For each patch, the contribution of the pixels in a cluster given by k-means clustering on edge strength and sharpness is estimated in terms of the percentage of pixels. The same percentage values are considered as a feature vector for classification with the help of a Support Vector Machine (SVM) classifier. Experimental results show that the proposed technique outperforms the state-of-the-art scene categorization methods. Our experimental results demonstrate that the SVM classifier performs better than rule and template-based methods.

Discriminative Semi-Coupled Projective Dictionary Learning for Low-Resolution Person Re-Identification

Person re-identification (re-ID) is a fundamental task in automated video surveillance. In real-world visual surveillance systems, a person is often captured in quite low resolutions. So we often need to perform low-resolution person re-ID, where images captured by different cameras have great resolution divergences. Existing methods cope problem via some complicated and time-consuming strategies, making them less favorable in practice, and their performances are far from satisfactory. In this paper, we design a novel Discriminative Semi-coupled Projective Dictionary Learning (DSPDL) model to effectively and efficiently solve this problem. Specifically, we propose to jointly learn a pair of dictionaries and a mapping to bridge the gap across low(er) and high(er) resolution person images. Besides, we develop a novel graph regularizer to incorporate positive and negative image pair information in a parameterless fashion. Meanwhile, we adopt the efficient and powerful projective dictionary learning technique to boost the our efficiency. Experiments on three public datasets show the superiority of the proposed method to the state-of-the-art ones.

Vanishing point detection for visual surveillance systems in railway platform environments

Visual surveillance is of paramount importance in public spaces and especially in train and metro platforms which are particularly susceptible to many types of crime from petty theft to terrorist activity. Image resolution of visual surveillance systems is limited by a trade-off between several requirements such as sensor and lens cost, transmission bandwidth and storage space. When image quality cannot be improved using high-resolution sensors, high-end lenses or IR illumination, the visual surveillance system may need to increase the resolving power of the images by software to provide accurate outputs such as, in our case, vanishing points (VPs). Despite having numerous applications in camera calibration, 3D reconstruction and threat detection, a general method for VP detection has remained elusive. Rather than attempting the infeasible task of VP detection in general scenes, this paper presents a novel method that is fine-tuned to work for railway station environments and is shown to outperform the state-of-the-art for that particular case. In this paper, we propose a three-stage approach to accurately detect the main lines and vanishing points in low-resolution images acquired by visual surveillance systems in indoor and outdoor railway platform environments. First, several frames are used to increase the resolving power through a multi-frame image enhancer. Second, an adaptive edge detection is performed and a novel line clustering algorithm is then applied to determine the parameters of the lines that converge at VPs; this is based on statistics of the detected lines and heuristics about the type of scene. Finally, vanishing points are computed via a voting system to optimize detection in an attempt to omit spurious lines. The proposed approach is very robust since it is not affected by ever-changing illumination and weather conditions of the scene, and it is immune to vibrations. Accurate and reliable vanishing point detection provides very valuable information, which can be used to aid camera calibration, automatic scene understanding, scene segmentation, semantic classification or augmented reality in platform environments.

Features-Based Moving Objects Tracking for Smart Video Surveillances: A Review

Video surveillance is one of the most active research topics in the computer vision due to the increasing need for security. Although surveillance systems are getting cheaper, the cost of having human operators to monitor the video feed can be very expensive and inefficient. To overcome this problem, the automated visual surveillance system can be used to detect any suspicious activities that require immediate action. The framework of a video surveillance system encompasses a large scope in machine vision, they are background modelling, object detection, moving objects classification, tracking, motion analysis, and require fusion of information from the camera networks. This paper reviews recent techniques used by researchers for detection of moving object detection and tracking in order to solve many surveillance problems. The features and algorithms used for modelling the object appearance and tracking multiple objects in outdoor and indoor environment are also reviewed in this paper. This paper summarizes the recent works done by previous researchers in moving objects tracking for single camera view and multiple cameras views. Nevertheless, despite of the recent progress in surveillance technologies, there still are challenges that need to be solved before the system can come out with a reliable automated video surveillance.

How genomes could transform surveillance and facial recognition

Despite decades of research, automatic visual surveillance systems have not delivered the results initially promised to the public. A key issue behind this, which remains unsolved, is the variability, complexity and inconsistency of the images being produced. The science of genomics offers insights that could solve this problem, and also prompt advances in facial recognition.

РАЗРАБОТКА СРЕДСТВ АГЕНТНОГО ЛОГИЧЕСКОГО ПРОГРАММИРОВАНИЯ ДЛЯ МНОГОКАНАЛЬНОГО ИНТЕЛЛЕКТУАЛЬНОГО ВИДЕОНАБЛЮДЕНИЯ

Experimental means developed in the Actor Prolog parallel object-oriented logic language for implementation of heterogeneous multichannel intelligent visual surveillance systems are considered. These means are examined by the instance of a logic program for permanent monitoring of people's body parts temperature in the area of visual surveillance. The logic program implements a fusion of heterogeneous data acquired by two devices: (1) 3D coordinates of human body are measured using a time-of-flight (ToF) camera; (2) 3D coordinates of human body skeleton are computed on the base of 3D coordinates of the body; (3) a thermal video is acquired using a thermal imaging camera. Special built-in classes are developed in the Actor Prolog language for the acquisition and analysis of 3D and 2D video data: the Kinect and KinectBuffer classes implement interaction between the logic program and the ToF camera of the Kinect-2 device; the BufferedScene class implements storing and transferring 3D images; the Canvas3D class implements 3D graphics based on the Java3D open source library. In the considered example, the thermal video is projected to the 3D surface of the human body; then the temperature of the human body is projected to the vertices and edges of the skeleton. A special logical agent (i.e., the logic program written in Actor Prolog) implements these operations in real-time and transfers the data to another logical agent. The latter agent implements a time average of the temperature of the human skeletons and displays colored 3D images of the skeletons; the average temperature of the vertices and edges of the skeletons is depicted by colors. A method of remote predicate calls is used for the interaction of the logical agents; this method was developed and implemented in Actor Prolog to support the agent logic programming paradigm. The logic programming means under consideration are developed for the purpose of the implementation of logical analysis of video scene semantics in the intelligent visual surveillance systems. © 2018 Radioelektronika, Nanosistemy, Informacionnye Tehnologii. All rights reserved.

Person Re-Identification Using Hybrid Representation Reinforced by Metric Learning

Person Re-Identification (Re-Id) is among the main constituents of an automated visual surveillance system. It aims at finding out true matches of a given query person from a large repository of non-overlapping camera images/videos. In this paper, we have proposed an efficient Re-Id approach that is based on a highly discriminative hybrid person representation which combines the low-level hand-crafted appearance based features together with the mid-level attributes and semantic based deep features. The low-level hand crafted features are extracted by using hierarchical Gaussian and local histogram distributions in different color spaces. These features incorporate discriminative texture, shape and color information which is invariant to distractors, e.g., variations in pose, viewpoint and illumination, and so on. The mid-level attribute based deep features are extracted to incorporate contextual- and semantic-based information. The feature space is optimized and self-learned using cross-view quadratic discriminant analysis and multiple metric learning, with the aim to reduce the intra-class differences and increase the inter-class variations for robust person matching. The proposed framework is evaluated on publicly available small scale (VIPeR, PRID450s, and GRID) and large scale (CUHK01, Market1501, and DukeMTMC-ReID) person Re-Id datasets. The experimental results show that the hybrid hand-crafted and deep features outperformed the existing state-of-the-art in approaches in the unsupervised paradigm.

CVSS

A large amount of surveillance videos and images need sufficient storage. In this article, an architecture of cloud-based surveillance systems and its modules will be designed, the Cloud-based Visual Surveillance System (CVSS) will be implemented on a private cloud using a Virtual Machine (VM). The users are able to link their cameras to the CVSS system so that the goal of this design can be achieved. The authors' CVSS system is able to push notification messages of captured videos to receivers, and their users could receive a surveillance video along with its events. The CVSS system fully makes use of the merits of cloud computing, which make it more advanced as stated in the evaluation section of this article. The contributions of this article are to be implemented in the CVSS system with: (1) video stream input, (2) intelligent visual surveillance, (3) real-time video transcoding and storage, (4) message pushing and media streaming output.

QUIS‐CAMPI: an annotated multi‐biometrics data feed from surveillance scenarios

The accuracy of biometric recognition in unconstrained scenarios has been a major concern for a large number of researchers. Despite such efforts, no system can recognise in a fully automated manner human beings in totally wild conditions such as in surveillance environments. Several sets of degraded data have been made available to the research community, where the reported performance by state-of-the-art algorithms is already saturated, suggesting that these sets do not reflect faithfully the conditions in such hard settings. To this end, the authors introduce the QUIS-CAMPI data feed, comprising samples automatically acquired by an outdoor visual surveillance system, with subjects on-the-move and at-a-distance (up to 50 m). Also, they supply a high-quality set of enrolment data. When compared to similar data sources, the major novelties of QUIS-CAMPI are: (i) biometric samples are acquired in a fully automatic way; (ii) it is an open dataset, i.e. the number of probe images and enroled subjects grow on a daily basis; and (iii) it contains multi-biometric traits. The ensemble properties of QUIS-CAMPI ensure that the data span a representative set of covariate factors of real-world scenarios, making it a valuable tool for developing and benchmarking biometric recognition algorithms capable of working in unconstrained scenarios.

Global homography calibration for monocular vision-based pose measurement of mobile robots

Compared with the size of a calibration board, the inspected area within the camera field of view is much bigger in many applications such as visual surveillance systems. In this paper, we present a global image-to-ground homography calibration method to obtain the mapping between the image and the planar scene lying in the whole camera field of view. The calibration approach is proposed by fusing multiple local homography matrices, with each of them only reflecting the relationship between a small calibration board and its corresponding small image part. By further calibrating a height-related homography, visual measurement can be extended for arbitrary planar surfaces with known heights. The proposed method presents such merits as no requirement for camera internal parameters, high calibration accuracy, and ease of implementation. Experimental results show that (1) the proposed global homography method achieves more accurate measurement results than the local homography method; (2) the height-related homography is calibrated with high accuracy; (3) the proposed approach can be used for mobile robot localization with accuracy close to the performance limit of a monocular camera.

Video Enhancement for Medical and Surveillance Applications

Live video laparoscopic surgery and visual surveillance has been an important research in the past few years. Laparoscopic surgery is a modern video tracking surgical technique in which operations are performed distantly from their location through small incisions (usually 0.5–1.5 cm) in a different place in the body. The visual surveillance system starts with motion detection or tracking. This motion object detection method, attempts to locate connected regions that define, relate the moving objects within the scene; like frame-to-frame difference, background subtraction and motion analysis, the object is detected and tracked from an enhanced video. For these we have defined two different approaches. First one is the Laparoscope Surgery video and Surveillance video Enhancement (LSSVE) algorithm and another algorithm used for only surveillance application background subtraction algorithm. The main goal of the video enhancement algorithm is many objects or activities of interest occur in a blood vessel and dark environment which cannot be seen easily without enhancement. For this development, we follow three different methods, In the second algorithm of background subtraction using dynamic threshold and a mixture of Gaussian three different methods are used effectively for object detection and compared their basis of performance on the accessible tacking the accurate location and detection. In laparoscope surgery, a camera and light provide feedback to the surgeon, who sees the enlarged and video enhanced surgical elements on a TV monitor. The video surveillance system does frame differences, after the object foreground detection, the parameters like speed, velocity and motion are determined. Keywords: Background modelling, background subtraction, live video laparoscopic surgery, LSSVE algorithm, object tracking, visual surveillance.

Pedestrian Counting With Back-Propagated Information and Target Drift Remedy

Pedestrian density is one of the important factors in designing visual surveillance and intelligent transportation systems, but it is challenging to obtain accurate and robust estimates because of both inconsistent crowd patterns in the scenes and target drift caused by imbalanced data distribution. Most of existing global regression frameworks focus on the former challenge to improve the robustness of regression learning, but very few work concerns on mitigating the suffering from the latter one. This paper proposes a novel counting-by-regression framework to utilize the importance of training samples to improve the robustness against inconsistent feature-target relationship based on a recently-proposed learning paradigm—learning with privileged information. To this end, the concept of back-propagation is for the first time considered to select more informative samples contributed to robust fitting performance. Moreover, the direction of target drift along the continuously-changing target dimension is discovered by learning local classifiers under different situation of pedestrian density, which can thus be exploited in our algorithm to further boost the performance. Experimental evaluation on the public UCSD and shopping Mall benchmarks verifies that our approach significantly beats the state-of-the-art counting-by-regression frameworks.

Trajectory-based vehicle tracking at low frame rates

In smart cities, an intelligent traffic surveillance system plays a crucial role in reducing traffic jams and air pollution, thus improving the quality of life. An intelligent traffic surveillance should be able to detect and track multiple vehicles in real-time using only limited resources. Conventional tracking methods usually run at a high video-sampling rate, assuming that the same vehicles in successive frames are similar and move only slightly. However, in cost effective embedded surveillance systems (e.g., a distributed wireless network of smart cameras), video frame rates are typically low because of limited system resources. Therefore, conventional tracking methods perform poorly in embedded surveillance systems because of discontinuity of the moving vehicles in the captured recordings. In this study, we present a fast and light algorithm that is suitable for an embedded real-time visual surveillance system to detect effectively and track multiple moving vehicles whose appearance and/or position changes abruptly at a low frame rate. For effective tracking at low frame rates, we propose a new matching criterion based on greedy data association using appearance and position similarities between detections and trackers. To manage abrupt appearance changes, manifold learning is used to calculate appearance similarity. To manage abrupt changes in motion, the next probable centroid area of the tracker is predicted using trajectory information. The position similarity is then calculated based on the predicted next position and progress direction of the tracker. The proposed method demonstrates efficient tracking performance during rapid feature changes and is tested on an embedded platform (ARM with DSP-based system).

Robust Video Stabilization Using Particle Keypoint Update and l₁-Optimized Camera Path.

Acquisition of stabilized video is an important issue for various type of digital cameras. This paper presents an adaptive camera path estimation method using robust feature detection to remove shaky artifacts in a video. The proposed algorithm consists of three steps: (i) robust feature detection using particle keypoints between adjacent frames; (ii) camera path estimation and smoothing; and (iii) rendering to reconstruct a stabilized video. As a result, the proposed algorithm can estimate the optimal homography by redefining important feature points in the flat region using particle keypoints. In addition, stabilized frames with less holes can be generated from the optimal, adaptive camera path that minimizes a temporal total variation (TV). The proposed video stabilization method is suitable for enhancing the visual quality for various portable cameras and can be applied to robot vision, driving assistant systems, and visual surveillance systems.

Review on Vision-Based Gait Recognition: Representations, Classification Schemes and Datasets

Gait has unique advantage at a distance when other biometrics cannot be used since they are at too low resolution or obscured, as commonly observed in visual surveillance systems. This paper provides a survey of the technical advancements in vision-based gait recognition. A wide range of publications are discussed in this survey embracing different perspectives of the research in this area, including gait feature extraction, classification schemes and standard gait databases. There are two major groups of the state-of-the-art techniques in characterizing gait: Model-based and motion-free. The model-based approach obtains a set of body or motion parameters via human body or motion modeling. The model-free approach, on the other hand, derives a description of the motion without assuming any model. Each major category is further organized into several subcategories based on the nature of gait representation. In addition, some widely used classification schemes and benchmark databases for evaluating performance are also discussed.

Distributed collaborative surveillance system based on leader election protocols

A variety of visual surveillance systems has been developed to satisfy different purposes. However, traditional visual surveillance systems are inflexible and passive, and thus incur significantly high overhead upon recording video information. Therefore, to improve such systems, researchers have been incorporating some special technologies into traditional visual surveillance systems. Doing so makes the advantages of these adopted technologies able to be attached to these systems such that the systems can benefit much from these technologies. In this study, the authors devise a novel visual surveillance system based on a distributed collaborative mechanism among all monitoring cameras. More specifically, the authors utilise the concept of a well-known distributed application, i.e. leader election, to implement this mechanism. Such a mechanism helps a visual surveillance system with more than one monitoring camera automatically and efficiently assign the proper number of cameras to track a particular object. Hence, upon a moderate number of objects entering the monitoring area, the monitoring cameras will be appointed to each object almost evenly.

Motion Estimation-based Human Fall Detection for Visual Surveillance

Currently, the world’s elderly population continues to grow at a dramatic rate. As the number of senior citizens increases, detection of someone falling has attracted increasing attention for visual surveillance systems. This paper presents a novel fall-detection algorithm using motion estimation and an integrated spatiotemporal energy map of the object region. The proposed method first extracts a human region using a background subtraction method. Next, we applied an optical flow algorithm to estimate motion vectors, and an energy map is generated by accumulating the detected human region for a certain period of time. We can then detect a fall using k-nearest neighbor (kNN) classification with the previously estimated motion information and energy map. The experimental results show that the proposed algorithm can effectively detect someone falling in any direction, including at an angle parallel to the camera’s optical axis.

Unobtrusive silhouette extraction using multivariate analysis and shadow removal in RGB color model

Object extraction is the initial step for any image processing task, such as visual surveillance, object tracking and intelligent transport system. Apart from other techniques, such as biometric system, these methods do not require any physical contact with an individual and can be performed from a distance and, therefore, used severely. Although several object extraction methods are available, this paper proposes a simple method to extract an object from video sequence based on the correlation between color model components. As the use of a single threshold is not suitable to correctly segment the object from the background, we are using two thresholds. Experimental results prove that this method can work well in static as well as dynamic background having different illumination conditions. Moreover, segmentation has side effects like moving cast shadows and self shadow, both of which reduce accuracy. In this paper, we present an automatic shadow removal method to extract objects. The analytical comparison with other foreground extraction, shadow detection and removal algorithms and their results are also presented for better understanding. The proposed method extract silhouette without any pre or post processing step of shadow removal.

Moving Object Detection Using Scanning Camera on a High-Precision Intelligent Holder.

During the process of moving object detection in an intelligent visual surveillance system, a scenario with complex background is sure to appear. The traditional methods, such as “frame difference” and “optical flow”, may not able to deal with the problem very well. In such scenarios, we use a modified algorithm to do the background modeling work. In this paper, we use edge detection to get an edge difference image just to enhance the ability of resistance illumination variation. Then we use a “multi-block temporal-analyzing LBP (Local Binary Pattern)” algorithm to do the segmentation. In the end, a connected component is used to locate the object. We also produce a hardware platform, the core of which consists of the DSP (Digital Signal Processor) and FPGA (Field Programmable Gate Array) platforms and the high-precision intelligent holder.