Surveillance Scenes Research Articles

Estimation of crowd density in surveillance scenes based on deep convolutional neural network

As an effective way for crowd monitoring, control and behavior understanding, crowd density estimation is an important research topic in artificial intelligence applications. In this paper, we propose a new crowd density estimation method by deep convolutional neural network (ConvNet). The contributions are two-folds: first, typical deep networks are imported for crowd density estimation. Second, a new dataset including 31 crowd Subway-carriage scenes with over 160K density annotated images is introduced to better evaluate the accuracy of cross-scene crowd density estimation methods. Experiment results confirm the good performance of our proposed method for real-world application.

夜间交通监控中的反光干扰消除和车辆检测方法研究

在交通监控场景中，基于前车灯特征的车辆检测是夜间道路交通监控中的车辆智能检测方法。然而，前车灯在路面形成的反射光会对基于车灯的夜间车辆检测算法造成很大干扰。本文提出了消除路面反射光干扰的夜间车辆检测方法，利用反射光与车灯在视频图像中的亮度方差特征差异，构造基于决策树的反射光和车灯的分类算法，识别视频图像中的车灯和反射光，效消除图像中的反射光，然后提取候选车灯ROI；最后通过对车灯的几何约束匹配实现夜间车辆检测；该方法有效提高了夜间交通车辆检测的性能，检测性能优于现有算法。 In traffic surveillance scene, headlight based vehicle detection is an important intelligent vehicle detection approach in nighttime road traffic monitoring. However, the road reflection generated by the vehicles’ headlights will cause great interference to the night time vehicle detection algo-rithm. In this paper, we propose a reflection elimination approach by use of the difference of in-tensity variance between reflected light and vehicle’s headlight in the nighttime traffic images. We construct a decision tree based classification algorithm to classify the reflected light and headlight; then eliminate the reflected lights from the traffic images and extract the candidate vehicle light ROI effectively. Finally, we use the headlights geometric constraints to realize night vehicle detection. Our algorithm can effectively improve the performance of night vehicle detection, and the detection rate is better than the existing algorithm.

Adaptive weighted moving shadow detection based on multiple features

Moving object detection is a fundamental task for automatic systems in video surveillance scenes, whose performance is downgraded by moving shadows unfortunately since moving objects and their shadows tend to present similar motion patterns and most moving object detection methods confused them frequently. To deal with this problem, we propose an adaptive weighted moving shadow detection method based on multiple features. In the proposed method, intensity, color and texture properties with neighboring information are exploited to generate feature maps, which are utilized to detect moving shadows respectively in small random selected videos to determine the weight for each feature map. Subsequently, the adaptive weighted fusion strategy is applied to fusion these feature maps for shadow detection according to the empirical threshold. At last, a series of spatial adjustment operations are implemented to correct misclassified pixels for obtaining refined detection results. By analyzing extensive experimental and comparison results, it demonstrates that the effectiveness and robustness of the proposed method for different video scenes and the superiority than some state-of-the-art methods.

Online growing neural gas for anomaly detection in changing surveillance scenes

Anomaly detection is still a challenging task for video surveillance due to complex environments and unpredictable human behaviors. Most existing approaches train offline detectors using manually labeled data and predefined parameters, and are hard to model changing scenes. This paper introduces a neural network based model called online Growing Neural Gas (online GNG) to perform an unsupervised learning. Unlike a parameter-fixed GNG, our model updates learning parameters continuously, for which we propose several online neighbor-related strategies. Specific operations, namely neuron insertion, deletion, learning rate adaptation and stopping criteria selection, get upgraded to online modes. In the anomaly detection stage, the behavior patterns far away from our model are labeled as anomalous, for which far away is measured by a time-varying threshold. Experiments are implemented on three surveillance datasets, namely UMN, UCSD Ped1/Ped2 and Avenue dataset. All datasets have changing scenes due to mutable crowd density and behavior types. Anomaly detection results show that our model can adapt to the current scene rapidly and reduce false alarms while still detecting most anomalies. Quantitative comparisons with 12 recent approaches further confirm our superiority.

Localization of region of interest in surveillance scene

In this paper, we present a method for autonomously detecting and extracting region(s)-of-interest (ROI) from surveillance videos using trajectory-based analysis. Our approach, localizes ROI in a stochastic manner using correlated probability density functions that model motion dynamics of multiple moving targets. The motion dynamics model is built by analyzing trajectories of multiple moving targets and associating importance to regions in the scene. The importance of each region is estimated as a function of the total time spent by multiple targets, their instantaneous velocity and direction of movement whilst passing through that region. We systematically validate our model and benchmark our technique against competing baselines through extensive experimentation using public datasets such as CAVIAR, ViSOR, and CUHK as well as a scenario-specific in-house surveillance dataset. Results obtained have demonstrated the superiority of the proposed technique against a few popular existing state-of-the-art techniques.

Random walks colour histogram modification for human tracking

Accurate human tracking in surveillance scenes is one of the preliminary requirements for other tasks. However, when the human target is small, the extracted features may not be prominent and thus the tracking performance is unsatisfactory. The colour feature is relatively robust to the change of target size and shape, but it is prone to be affected by the background information. For the above reasons, the authors introduce random walker segmentation into human tracking and determine the background region according to the distribution characteristics of segmentation results. Even if the colour of the target is very similar to that of the background, this algorithm can segment the target. Furthermore, the principal component analysis method is used to distinguish human targets from the background as well. During tracking, the authors prevent the degradation of the target model by adding new target information. In order to overcome the mean‐shift local optimisation problem, the authors search for the candidate target region with the largest weight according to the sum of all probability in each region. Experimental results further show that the authors’ tracking algorithm demonstrates better performance on tracking small human target under some challenging scenes compared with several existing tracking methods.

EEG and Eye Tracking Demonstrate Vigilance Enhancement with Challenge Integration.

Maintaining vigilance is possibly the first requirement for surveillance tasks where personnel are faced with monotonous yet intensive monitoring tasks. Decrement in vigilance in such situations could result in dangerous consequences such as accidents, loss of life and system failure. In this paper, we investigate the possibility to enhance vigilance or sustained attention using “challenge integration,” a strategy that integrates a primary task with challenging stimuli. A primary surveillance task (identifying an intruder in a simulated factory environment) and a challenge stimulus (periods of rain obscuring the surveillance scene) were employed to test the changes in vigilance levels. The effect of integrating challenging events (resulting from artificially simulated rain) into the task were compared to the initial monotonous phase. EEG and eye tracking data is collected and analyzed for n = 12 subjects. Frontal midline theta power and frontal theta to parietal alpha power ratio which are used as measures of engagement and attention allocation show an increase due to challenge integration (p < 0.05 in each case). Relative delta band power of EEG also shows statistically significant suppression on the frontoparietal and occipital cortices due to challenge integration (p < 0.05). Saccade amplitude, saccade velocity and blink rate obtained from eye tracking data exhibit statistically significant changes during the challenge phase of the experiment (p < 0.05 in each case). From the correlation analysis between the statistically significant measures of eye tracking and EEG, we infer that saccade amplitude and saccade velocity decrease with vigilance decrement along with frontal midline theta and frontal theta to parietal alpha ratio. Conversely, blink rate and relative delta power increase with vigilance decrement. However, these measures exhibit a reverse trend when challenge stimulus appears in the task suggesting vigilance enhancement. Moreover, the mean reaction time is lower for the challenge integrated phase (RTmean = 3.65 ± 1.4s) compared to initial monotonous phase without challenge (RTmean = 4.6 ± 2.7s). Our work shows that vigilance level, as assessed by response of these vital signs, is enhanced by challenge integration.

Pixel-to-Model Distance for Robust Background Reconstruction

Background information is crucial for many video surveillance applications such as object detection and scene understanding. In this paper, we present a novel pixel-to-model (P2M) paradigm for background modeling and restoration in surveillance scenes. In particular, the proposed approach models the background with a set of context features for each pixel, which are compressively sensed from local patches. We determine whether a pixel belongs to the background according to the minimum P2M distance, which measures the similarity between the pixel and its background model in the space of compressive local descriptors. The pixel feature descriptors of the background model are properly updated with respect to the minimum P2M distance. Meanwhile, the neighboring background model will be renewed according to the maximum P2M distance to handle ghost holes. The P2M distance plays an important role of background reliability in the 3-D spatial–temporal domain of surveillance videos, leading to the robust background model and recovered background videos. We applied the proposed P2M distance for foreground detection and background restoration on synthetic and real-world surveillance videos. Experimental results show that the proposed P2M approach outperforms the state-of-the-art approaches both in indoor and outdoor surveillance scenes.

Robust moving object detection based on spatio‐temporal confidence relationship

It is still an open problem in the context of complex scenarios like dynamic background and illumination variations, although numerous moving object detection schemes have been demonstrated. Great efforts have been made to develop some probability distribution pixels obey, for which samples are collected from spatial domain and/or temporal domain. However, significant attention has not been paid to the confidence relationship between pixels and its spatio-temporal neighbours in previous works. In this Letter, a confidence relationship model is proposed to complete the moving object detection task in complex environments. Experiments on typical surveillance scenes verify that the proposed algorithm has attractive robustness and high accuracy for illumination variations and dynamic background against state-of-the-art methods.

Background light ray modeling for change detection

This paper is an extension of the work that was originally reported in Shimada et al. (2013). This paper proposes a change detection method based on spatio-temporal light ray consistency. The proposed method introduces light field sensing, which is used to generate an arbitrary in-focus plane. Change detection is performed in a surveillance scene, where the background region can be filtered out by an out-focusing process. This approach resolves a longstanding issue in background modeling-based object detection, which often suffers from false positives in the background regions. To realize this new change detection method, a new feature representation, called the local ray pattern (LRP), is introduced. The LRP evaluates the spatial consistency of the light rays, and this plays an important role in distinguishing whether the light rays come from the in-focus plane or elsewhere. A combination of the LRP and Gaussian mixture model (GMM)-based background modeling realizes change detection in the in-focus plane. Experimental results demonstrate the proposed method’s effectiveness and its applicability to video surveillance.

People-flow counting in complex environments by combining depth and color information

People-flow counting is one of the key techniques of intelligence video surveillance systems and the information of people-flow obtained from this technique is an very important evidence for many applications, such as business analysis, staff planning, security, etc. Traditionally, the color image information based methods encounter kinds of challenges, such as shadows, illumination changing, cloth color, etc., while the depth information based methods suffer from lack of texture. In this paper, we propose an effective approach of people-flow counting by combining color and depth information. First, we adopt a background subtraction technique to fast obtain the moving regions on depth images. Second, the water filling algorithm is used to effectively detect head candidates on the moving regions. Then we use the SVM to recognize the real heads from the candidates. Finally, we adopt a weighted K Nearest Neighbor based multi-target tracking method to track each confirmed head and count the people through the surveillance region. Four datasets constructed from two surveillance scenes are used to evaluate the proposed method. Experimental results show that our method outperform the state-of-the-art methods. Our method can work stably on condition of kinds of interruptions and can not only obtain high precisions, but also high recalls on four datasets.

New Technique for Extraction Moving Object Based on Active Contours for Intelligent Visual Surveillance Applications

With the increasing concerns about public and road safety, it has become crucial to have in place sophisticated visual surveillance systems that can detect and locate potentially dangerous situations. In fact, being able to accurately define an object’s pose or location in an image is one of the current challenging research topics and knows many applications. It can be used to monitor traffic, analyse crowd flux statistics and congestion, identify persons, and detect abnormal behaviours. Within this work, we develop an automatic approach that can localise moving objects robustly and accurately in surveillance scenes. The basic approach consists of two major parts: the motion detection and the object localisation based on the active contours technique. The proposed method has been tested on different real urban traffic videos, and the experiment results demonstrate that our algorithm can locate effectively and accurately the moving objects; optimise the results of the localized objects and also decrease the computations load.

A new method for violence detection in surveillance scenes

Violence detection is a hot topic for surveillance systems. However, it has not been studied as much as for action recognition. Existing vision-based methods mainly concentrate on violence detectio...

Family Secrets: Shame and Privacy in Modern Britain. By Deborah Cohen (Oxford, Oxford University Press, 2013. xi plus 372 pp.)

Not many people find anything of interest in Croydon, a proverbially dull swathe of suburbia to the south of London. However, Deborah Cohen has, and uses some fascinating material from the Croydon local studies archive for at least one of her chapters on adoption in this wide-ranging study of secrecy and privacy in the British family over the last two centuries. The book sets out to show “what families attempted to hide in the past and why” (1), aiming to illustrate the part that families played “in the transformation of social mores” (2) over this period. This is a fairly daunting task to set yourself, and although the book is original and many of the stories in it are powerful, heartbreaking and very well told, it does not always work in the manner intended. It could be said that there are two main approaches to the history of the modern self and its secrets: the first suggests that the decline of communal ties and the emergence of privacy, together with the rise of the idea of universal rights, has expanded the sphere of the personal, thereby licensing a kind of obsession with identity, selfhood and confession. In that reading, we have become more and more open—more free according to many writers—and more willing to disclose the most intimate details of our lives, while both secrecy and privacy have declined to the point of nothingness in the age of mass communication and, latterly, digital technology. The other, more critical version of this story derives from Foucault, who argued that the modern world has seen the development of countless mechanisms for extracting the “truth of the self.” Confessing secrets, in that interpretation, indicates the scene of discipline and surveillance, denoting (and concealing) the ever-more intense process of “individualisation and totalisation.”

Region-based Mixture of Gaussians modelling for foreground detection in dynamic scenes

One of the most widely used techniques in computer vision for foreground detection is to model each background pixel as a Mixture of Gaussians (MoG). While this is effective for a static camera with a fixed or a slowly varying background, it fails to handle any fast, dynamic movement in the background. In this paper, we propose a generalised framework, called region-based MoG (RMoG), that takes into consideration neighbouring pixels while generating the model of the observed scene. The model equations are derived from expectation maximisation theory for batch mode, and stochastic approximation is used for online mode updates. We evaluate our region-based approach against ten sequences containing dynamic backgrounds, and show that the region-based approach provides a performance improvement over the traditional single pixel MoG. For feature and region sizes that are equal, the effect of increasing the learning rate is to reduce both true and false positives. Comparison with four state-of-the art approaches shows that RMoG outperforms the others in reducing false positives whilst still maintaining reasonable foreground definition. Lastly, using the ChangeDetection (CDNet 2014) benchmark, we evaluated RMoG against numerous surveillance scenes and found it to be amongst the leading performers for dynamic background scenes, whilst providing comparable performance for other commonly occurring surveillance scenes.

Cooperative object tracking using dual‐pan–tilt–zoom cameras based on planar ground assumption

Pan–tilt–zoom (PTZ) cameras play an important role in visual surveillance system. Dual-PTZ camera system is the simplest and most typical one. The superiority of this system lies in that it can obtain both large-view information and high-resolution local-view information of the tracked object at the same time. One method to achieve such task is to use master–slave configuration. One camera (master) tracks moving objects at low resolution and provides the positional information to another camera (slave). Then the slave camera can point towards the object at high resolution and track it dynamically. In this paper, we propose a novel framework exploiting planar ground assumption to achieve cooperative tracking. The approach differs from conventional methods in that we exploit planar geometric constraint to solve the camera collaboration problem. Compared with the existing approach, the proposed framework can be used in the case of wide baseline, and allows the depth change of the tracked object. The proposed method can also adapt to the dynamic change of the surveillance scene. Besides, we also describe a self-calibration method of homography matrix which is induced by the ground plane between two cameras. We demonstrate the effectiveness of the proposed method by testing it with a tracking system for surveillance applications.

Robust multiple cameras pedestrian detection with multi-view Bayesian network

Multi-camera pedestrian detection is the challenging problem in the field of surveillance video analysis. However, existing approaches may produce “phantoms” (i.e., fake pedestrians) due to the heavy occlusions in real surveillance scenario, while calibration errors and the diverse heights of pedestrians may also heavily decrease the detection performance. To address these problems, this paper proposes a robust multiple cameras pedestrian detection approach with multi-view Bayesian network model (MvBN). Given the preliminary results obtained by any multi-view pedestrian detection method, which are actually comprised of both real pedestrians and phantoms, the MvBN is used to model both the occlusion relationship and the homography correspondence between them in all camera views. As such, the removal of phantoms can be formulated as an MvBN inference problem. Moreover, to reduce the influence of the calibration errors and keep robust to the diverse heights of pedestrians, a height-adaptive projection (HAP) method is proposed to further improve the detection performance by utilizing a local search process in a small neighborhood of heights and locations of the detected pedestrians. Experimental results on four public benchmarks show that our method outperforms several state-of-the-art algorithms remarkably and demonstrates high robustness in different surveillance scenes.

Scene Dynamics Estimation for Parameter Adjustment of Gaussian Mixture Models

The scene dynamics can provide useful statistical information for adjusting parameters of Gaussian mixture models (GMMs) in video surveillance. The contributions of this paper are twofold. First, an adaptive scene dynamics estimation approach is proposed. Second, we propose a scene-dynamics based method to adjust two types of GMMs’ parameters, i.e., the learning rates and number of Gaussian components. For the learning rates, the scene dynamics are integrated into different kinds of pixel-type feedback schemes to control different kinds of learning rates. Experimental results demonstrate that the proposed method can effectively improve the performance of GMMs in surveillance scenes with complex dynamic backgrounds.

Multiple View Oriented Matching Algorithm for People Reidentification

People reidentification is one of the most challenging tasks in computer vision, and considerable efforts have been directed toward providing solutions to this problem. The existence of extensive camera networks and surveillance systems increases the amount of people images obtained, but, on the other hand, implies the need for new algorithms to enable reidentification of people captured by the cameras. There is no one optimal model that solves the entire problem, but a set of distinctive features can be used to help in the matching process. Our proposal consists of using the orientation of each person captured in the surveillance scene to considerably improve the reidentification process. An iterative algorithm maximizes the number of successful matches and speeds up the process. A comparison with other earlier relevant studies is presented using available datasets.

The Effects of the Relationships Between Object Features on Multiple-Identity Tracking

Simultaneous tracking of multiple moving objects is essential in tasks such as traffic control, automobile driving, and scene surveillance. Recently, an increasing number of studies have focused on the roles of object identity and location binding in unique target tracking tasks, but contradictory results have been reported. In the present study, we introduced for the first time the Stroop stimuli of Chinese characters to the multiple-identity tracking paradigm, taking advantage of the ease to control the overall size, familiarity, and visual complexity of the Chinese characters. The results showed that when the observers were asked to track unique objects that bear two distinct features, the feature conflict disrupted the tracking performance, even when the given object has a distinctive identity. Our data revealed that the internal disaccord of semantic and physical features on an object can disturb the identity-location binding process in a multiple-identity tracking task, but does not affect the location information significantly.