Real-time Background Subtraction Research Articles

WFH on Duty Employee Attendance System

Abstract: Automatic face recognition (AFR) technologies have seen dramatic improvements in performance over the past years, and such systems are now widely used for security and commercial applications. So Smart Attendance using Real Time Face Recognition is a real world solution which comes with day to day activities of handling students. The task is very difficult as the real time background subtraction in an image is still a challenge. To detect real time human face are used and a simple fast Principal Component Analysis has used to recognize the faces de-tested with a high accuracy rate. The matched faces are used to mark attendance of the student. Our system maintains the attendance records of students automatically. Manual entering of in logbooks becomes a difficult task and it also wastes the time. So we designed an efficient module that comprises of face recognition to manage the attendance records of students. Our module enrols the student’s face. Keywords: Automatic Face Recognition(AFR), Employee Attendance System, Attendance Management System(AMS)

Asynchronous Semantic Background Subtraction.

The method of Semantic Background Subtraction (SBS), which combines semantic segmentation and background subtraction, has recently emerged for the task of segmenting moving objects in video sequences. While SBS has been shown to improve background subtraction, a major difficulty is that it combines two streams generated at different frame rates. This results in SBS operating at the slowest frame rate of the two streams, usually being the one of the semantic segmentation algorithm. We present a method, referred to as “Asynchronous Semantic Background Subtraction” (ASBS), able to combine a semantic segmentation algorithm with any background subtraction algorithm asynchronously. It achieves performances close to that of SBS while operating at the fastest possible frame rate, being the one of the background subtraction algorithm. Our method consists in analyzing the temporal evolution of pixel features to possibly replicate the decisions previously enforced by semantics when no semantic information is computed. We showcase ASBS with several background subtraction algorithms and also add a feedback mechanism that feeds the background model of the background subtraction algorithm to upgrade its updating strategy and, consequently, enhance the decision. Experiments show that we systematically improve the performance, even when the semantic stream has a much slower frame rate than the frame rate of the background subtraction algorithm. In addition, we establish that, with the help of ASBS, a real-time background subtraction algorithm, such as ViBe, stays real time and competes with some of the best non-real-time unsupervised background subtraction algorithms such as SuBSENSE.

Fusing Self-Organized Neural Network and Keypoint Clustering for Localized Real-Time Background Subtraction.

Moving object detection in video streams plays a key role in many computer vision applications. In particular, separation between background and foreground items represents a main prerequisite to carry out more complex tasks, such as object classification, vehicle tracking, and person re-identification. Despite the progress made in recent years, a main challenge of moving object detection still regards the management of dynamic aspects, including bootstrapping and illumination changes. In addition, the recent widespread of Pan-Tilt-Zoom (PTZ) cameras has made the management of these aspects even more complex in terms of performance due to their mixed movements (i.e. pan, tilt, and zoom). In this paper, a combined keypoint clustering and neural background subtraction method, based on Self-Organized Neural Network (SONN), for real-time moving object detection in video sequences acquired by PTZ cameras is proposed. Initially, the method performs a spatio-temporal tracking of the sets of moving keypoints to recognize the foreground areas and to establish the background. Then, it adopts a neural background subtraction, localized in these areas, to accomplish a foreground detection able to manage bootstrapping and gradual illumination changes. Experimental results on three well-known public datasets, and comparisons with different key works of the current literature, show the efficiency of the proposed method in terms of modeling and background subtraction.

Online Robust Principal Component Analysis With Change Point Detection

Robust principal component analysis (PCA) is a key technique for dynamical high-dimensional data analysis, including background subtraction for surveillance video. Typically, robust PCA requires all observations to be stored in memory before processing. The batch manner makes robust PCA inefficient for big data. In this paper, we develop an efficient online robust PCA method, namely, online moving window robust principal component analysis (OMWRPCA). Unlike the existing algorithms, OMWRPCA can successfully track not only slowly changing subspaces but also abruptly changing subspaces. Embedding hypothesis testing into the algorithm enables OMWRPCA to detect change points of the underlying subspaces. Extensive numerical experiments, including real-time background subtraction, demonstrate the superior performance of OMWRPCA compared with other state-of-the-art approaches.

A Fully Convolutional Encoder–Decoder Spatial–Temporal Network for Real-Time Background Subtraction

Background subtraction is described as the task of distinguishing pixels into moving objects and the background in a frame. In this paper, we propose a fully convolutional encoder–decoder spatial–temporal network (FCESNet) to achieve real-time background subtraction. In the proposed many-to-many architecture method encoded features of consecutive frames are fed into a spatial–temporal information transmission (STIT) module to capture the spatial–temporal correlation in the frame sequence, and then a decoder is designed to output the subtraction results of all frames. A “patch-based” training method is designed to increase the practicability and flexibility of the proposed method. The experiments over CDNet2014 have shown that the proposed method could achieve state-of-the-art performance. The proposed method is proved to be able to achieve real-time background subtraction.

Multi-color background-free coherent anti-Stokes Raman scattering microscopy using a time-lens source.

We demonstrate a multi-color background-free coherent anti-Stokes Raman scattering (CARS) imaging system, using a robust, all-fiber, low-cost, multi-wavelength time-lens source. The time-lens source generates picosecond pulse trains at three different wavelengths. The first is 1064.3 nm, the second is tunable between 1052 nm and 1055 nm, and the third is tunable between 1040 nm and 1050 nm. When the time-lens source is synchronized with a mode-locked Ti:Sa laser, two of the three wavelengths are used to detect different Raman frequencies for two-color on-resonance imaging, whereas the third wavelength is used to obtain the off-resonance image for nonresonant background subtraction. Mixed poly(methyl methacrylate) (PMMA) and polystyrene (PS) beads are used to demonstrate two-color background-free CARS imaging. The synchronized multi-wavelength time-lens source enables pixel-to-pixel wavelength-switching. We demonstrate simultaneous two-color CARS imaging of CH2 and CH3 stretching vibration modes with real-time background subtraction in ex vivo mouse tissue.

Online Schatten quasi-norm minimization for robust principal component analysis

The Schatten-p quasi-norm (0 < p < 1) is widely and successfully used as a superior rank surrogate to the well-known nuclear norm. It has been proved that the Schatten-p quasi-norm is admirable for the robust principal component analysis problem both in theory and in practice. However, existing algorithms for solving the Schatten-p quasi-norm minimization problem suffer from large memory requirement and high computational complexity, which is due to (1) they work in batch manner, which requires to load all the data in the memory during process; (2) they relay on singular value decomposition on a dense matrix in every iteration. Hence they are only able to handle problems of certain scale and highly inefficient in large scale cases. In this paper, by reformulating the Schatten-p quasi-norm in variational formulation, we design an online algorithm to solve the Schatten-p quasi-norm minimization problem, in which the memory cost is independent of the data sample size. Moreover, the new algorithm is totally free of singular value decomposition. Numerical study on subspace recovery demonstrates encouraging results of our algorithm compared with the widely used rank surrogate functions for online optimization. Besides, experiments on real video background subtraction indicate that the newly proposed online algorithm is extremely fast, and meets the real-time background subtraction for practical video processing tasks.

Real-Time Adaptive Histogram Min-Max Bucket (HMMB) Model for Background Subtraction

This paper proposes an efficient real-time background subtraction algorithm, which is essential in many computer vision applications. Initially, histograms of the intensity values for each channel of a pixel position in a set of training framesare constructed. A background model, histogram min-max bucket, is constructed from the minimum and maximum values of contiguous non-zero frequencies of the temporal intensity histogram. A novel feature of this algorithm is the use of a single sliding window to update the system adaptively, capturing the proper background even under sudden and/or gradual illumination changes in the scene. We incorporate both local and global information with the help of the current and the previously visited pixel values for binary classification of a pixel into foreground and background. This algorithm is compared with several state-of-the-art techniques and experimental studies show that the proposed method outperforms all these methods in terms of accurate binary classification.

RISE

This paper proposes RISE, an automated Reconfigurable framework for real-time background subtraction applied to Intelligent video SurveillancE. RISE is devised with a new streaming-based methodology that adaptively learns/updates a corresponding dictionary matrix from background pixels as new video frames are captured over time. This dictionary is used to highlight the foreground information in each video frame. A key characteristic of RISE is that it adaptively adjusts its dictionary for diverse lighting conditions and varying camera distances by continuously updating the corresponding dictionary. We evaluate RISE on natural-scene vehicle images of different backgrounds and ambient illuminations. To facilitate automation, we provide an accompanying API that can be used to deploy RISE on FPGA-based system-on-chip platforms. We prototype RISE for end-to-end deployment of three widely-adopted image processing tasks used in intelligent transportation systems: License Plate Recognition (LPR), image denoising/reconstruction, and principal component analysis. Our evaluations demonstrate up to 87-fold higher throughput per energy unit compared to the prior-art software solution executed on ARM Cortex-A15 embedded platform.

Robust Online Matrix Factorization for Dynamic Background Subtraction.

We propose an effective online background subtraction method, which can be robustly applied to practical videos that have variations in both foreground and background. Different from previous methods which often model the foreground as Gaussian or Laplacian distributions, we model the foreground for each frame with a specific mixture of Gaussians (MoG) distribution, which is updated online frame by frame. Particularly, our MoG model in each frame is regularized by the learned foreground/background knowledge in previous frames. This makes our online MoG model highly robust, stable and adaptive to practical foreground and background variations. The proposed model can be formulated as a concise probabilistic MAP model, which can be readily solved by EM algorithm. We further embed an affine transformation operator into the proposed model, which can be automatically adjusted to fit a wide range of video background transformations and make the method more robust to camera movements. With using the sub-sampling technique, the proposed method can be accelerated to execute more than 250 frames per second on average, meeting the requirement of real-time background subtraction for practical video processing tasks. The superiority of the proposed method is substantiated by extensive experiments implemented on synthetic and real videos, as compared with state-of-the-art online and offline background subtraction methods.

Multispectral open-air intraoperative fluorescence imaging.

Intraoperative fluorescence imaging informs decisions regarding surgical margins by detecting and localizing signals from fluorescent reporters, labeling targets such as malignant tissues. This guidance reduces the likelihood of undetected malignant tissue remaining after resection, eliminating the need for additional treatment or surgery. The primary challenges in performing open-air intraoperative fluorescence imaging come from the weak intensity of the fluorescence signal in the presence of strong surgical and ambient illumination, and the auto-fluorescence of non-target components, such as tissue, especially in the visible spectral window (400-650 nm). In this work, a multispectral open-air fluorescence imaging system is presented for translational image-guided intraoperative applications, which overcomes these challenges. The system is capable of imaging weak fluorescence signals with nanomolar sensitivity in the presence of surgical illumination. This is done using synchronized fluorescence excitation and image acquisition with real-time background subtraction. Additionally, the system uses a liquid crystal tunable filter for acquisition of multispectral images that are used to spectrally unmix target fluorescence from non-target auto-fluorescence. Results are validated by preclinical studies on murine models and translational canine oncology models.

Non-convex online robust PCA: Enhance sparsity via ℓp-norm minimization

Compressive sensing, matrix rank optimization and Robust PCA-based matrix decomposition have an increasing number of non-convex approaches for optimizing the ideal ℓ0-norm sparsity. This paper presents a novel online non-convex ℓp-norm based Robust PCA (OLP-RPCA) approach, where 0 < p < 1. OLP-RPCA is developed from the offline version LP-RPCA. Our LP-RPCA method uses a new objective function in the Alternating Direction Method of Multipliers (ADMM) framework to efficiently solve the Robust PCA problem. More importantly, our OLP-RPCA method can achieve real-time performance on large-scale data without parallelizing or implementing on a graphics processing unit. We mathematically and empirically show that the computational complexity of our OLP-RPCA algorithm is linear in both the sample dimension and the number of samples. The proposed approaches are evaluated in various applications including Gaussian/non-Gaussian image denoising, face modeling, real-time background subtraction and video inpainting and compared against numerous state-of-the-art low-rank minimization methods to demonstrate the robustness of our algorithms.

Fast Compressive Target Detection for Wireless Video Sensor Nodes

In order to prolong the lifetime of visual target detection and tracking system based on wireless video sensor networks, many efficient methods have been proposed to reduce the energy consumption of the battery-powered video sensor nodes. Focused on reducing the amount of image data for computing, this paper presents a fast compressive method of target detection for video sensor nodes using structured compressive sensing. The major contributions are as follows: Firstly, we construct a novel structured measurement matrix for sampling the image. Secondly, we use an efficient adaptive Gaussian mixture model for real-time background subtraction. Experimental results show that our method can achieve good performance and over two times faster than traditional Gaussian mixture model.

Hardware Design of Moving Object Detection on Reconfigurable System

Moving object detection including background subtraction and morphological processing is a critical research topic for video surveillance because of its high computational loading and power consumption. This paper proposes a hardware design to accelerate the computation of background subtraction with low power consumption. A real-time background subtraction method is designed with a frame-buffer scheme and function partition to improve throughput, and implemented using Verilog HDL on FPGA. The design parallelizes the computations of background update and subtraction with a seven-stage pipeline. A stripe-based morphological processing and accounting for the completion of detected objects is devised. Simulation results for videos of VGA resolutions on a low-end FPGA device show 368 fps throughput for only the real-time background subtraction module, and 51 fps for the whole system, including off-chip memory access. Real-time efficiency with low power consumption and low resource utilization is thus demonstrated.

A Self-adaptive CodeBook (SACB) model for real-time background subtraction

Effective and efficient background subtraction is important to a number of computer vision tasks. In this paper, we introduce a new background model that integrates several new techniques to address key challenges for background modeling for moving object detection in videos. The novel features of our proposed Self-adaptive CodeBook (SACB) background model are: a more effective color model using YCbCr color space, a statistical parameter estimation method, and a new algorithm for adding new background codewords into the permanent model and deleting noisy codewords from the models. Also, a new block-based approach is introduced to exploit the local spatial information. The proposed model is rigorously tested and has shown significant performance improvements over several previous models.

3D measures exploitation for a monocular semi-supervised fall detection system

Falls have been reported as the leading cause of injury-related visits to emergency departments and the primary etiology of accidental deaths in elderly. Thus, the development of robust home surveillance systems is of great importance. In this article, such a system is presented, which tries to address the fall detection problem through visual cues. The proposed methodology utilizes a fast, real-time background subtraction algorithm, based on motion information in the scene and pixels intensity, capable to operate properly in dynamically changing visual conditions, in order to detect the foreground object. At the same time, it exploits 3D space's measures, through automatic camera calibration, to increase the robustness of fall detection algorithm which is based on semi-supervised learning approach. The above system uses a single monocular camera and is characterized by minimal computational cost and memory requirements that make it suitable for real-time large scale implementations.

Energy-Efficient Distributed Target Tracking in Wireless Video Sensor Networks

In order to prolong the lifetime of wireless video sensor networks (WVSNs), many methods for visual target tracking are proposed to reduce the energy cost of battery-powered video sensor nodes. Focus on reducing the amounts of image data for computation and communication, this paper presents an energy-efficient method for distributed target tracking in WVSNs. The major contributions of the proposed approach are as follows. First, each activated video node uses real-time background subtraction based on balanced compressive sensing adaptive gaussian mixture Model for target detection. Second, targets in different views are associated by homograph projection of the position and hue histogram. Third, a Sign of Innovation based consensus filter is proposed for distributed target tracking. Experimental results show the method can effectively reduce the energy cost and achieve satisfying performance in real scenes.

Real-time Background Subtraction for Video Avatar

Background subtraction is widely used method of detecting moving objects in static background in computer vision fields. In this paper, we present real-time background subtraction method for video avatar system or video conferencing without blue screen. We observed the noise of digital camera and made a range matrix for background masking. Noise reductions were applied to both background and mask using median filter blurring and open operation. Our experiment was performed using notebook PC and built-in webcam. The results showed our approach is feasible to real-time indoor environments.

Real Time Face Recognition Using AdaBoost Improved Fast PCA Algorithm

This paper presents an automated system for human face recognition in a real time background world for a large homemade dataset of persons face. The task is very difficult as the real time background subtraction in an image is still a challenge. Addition to this there is a huge variation in human face image in terms of size, pose and expression. The system proposed collapses most of this variance. To detect real time human face AdaBoost with Haar cascade is used and a simple fast PCA and LDA is used to recognize the faces detected. The matched face is then used to mark attendance in the laboratory, in our case. This biometric system is a real time attendance system based on the human face recognition with a simple and fast algorithms and gaining a high accuracy rate..

Robust Real-Time Background Subtraction Based on Local Neighborhood Patterns

This paper describes an efficient background subtraction technique for detecting moving objects. The proposed approach is able to overcome difficulties like illumination changes and moving shadows. Our method introduces two discriminative features based on angular and modular patterns, which are formed by similarity measurement between two sets of RGB color vectors: one belonging to the background image and the other to the current image. We show how these patterns are used to improve foreground detection in the presence of moving shadows and in the case when there are strong similarities in color between background and foreground pixels. Experimental results over a collection of public and own datasets of real image sequences demonstrate that the proposed technique achieves a superior performance compared with state-of-the-art methods. Furthermore, both the low computational and space complexities make the presented algorithm feasible for real-time applications.