Foreground Object Detection Research Articles

Detection and Removal of Foreground Objects in Spherical Images for the Synthesis of Photorealistic Intermediate Images

A method is proposed for the removal of foreground objects from spherical images as applied to the synthesis of intermediate images. The synthesis procedure is based on a 3D model of the captured scene. As a rule, the objects that appear in the frame but are not fixed in the model are deformed during the synthesis of an intermediate frame, thus making the result unrealistic. The method proposed allows one to remove such objects from the foreground. The method is based on the comparison of two images of the scene taken from close viewpoints and on the redundancy of available information. The method was tested on panoramas and models available in the Google Street View service.

Cauchy Mixture Model-based Foreground Object Detection with New Dynamic Learning Rate Using Spatial and Statistical information for Video Surveillance Applications

This paper presents a background modeling technique based on Cauchy mixture model (CMM) for moving object detection. The proposed approach detects the objects under different scenarios such as cluttered background, sudden and varied illumination, in the presence of shadow, slow- and fast-moving objects and under different weather conditions. The qualitative and quantitative performance evaluation of the proposed method on background model challenge (BMC) datasets, containing both real and synthetic videos, exhibits a superior performance over the state-of-the-art methods. The average accuracy for real and synthetic videos is 94.92% and 98.01%, respectively. The average F-score for real videos is 96.49%, and that for synthetic videos is 98.86%. The area under the curve (AUC) reveals an improved performance of 4.6% and 3.6% for real videos and synthetic videos of BMC dataset, respectively.

Rapid and Robust Background Modeling Technique for Low-Cost Road Traffic Surveillance Systems

Fast and accurate detection of vehicles on road traffic scenes captured by traffic surveillance cameras, is essential for large-scale deployment of automated traffic surveillance systems. The state-of-the-art techniques typically employ background modeling for low-complexity foreground detection. However, this is a challenging problem as these methods need to be robust to varying road scene conditions (such as illumination changes, camera jitter, stationary vehicles, and heavy traffic) leading to huge computation cost. In this paper, we propose a highly accurate yet low-complexity foreground (i.e., vehicle) detection technique, which can effectively deal with the varying road scene conditions, and generate accurate pixel-level foreground masks in real-time. We propose a novel robust block-based feature suitable for modeling road background and detecting vehicles as foreground, and employ Bayesian probabilistic modeling on these features. The experimental evaluations on widely used traffic datasets demonstrate that the proposed method can achieve comparable accuracy to the existing state-of-the-art techniques but at a much higher processing frame rate (40x speedup over PAWCS). The real-time performance of the proposed system has also been demonstrated by implementing it on a low-cost embedded platform, Odroid XU-4, that still achieves a frame rate of over 80 frames/s, thereby enabling the real-time detection of foreground objects in road scenes.

Incremental Tensor-Based Completion Method for Detection of Stationary Foreground Objects

In tasks such as abandoned luggage detection and stopped car detection, stationary foreground objects (SFOs) need to be detected and properly classified in real time. Different methods have been proposed to detect SFOs, but they are mainly focused on certain types of objects. In this paper, an incremental singular value decomposition-based method is presented to detect all types of SFOs such as abandoned objects and removed objects. The proposed method decomposes the video tensor spatiotemporally and divides it into background and foreground components. An appropriate analysis is applied to the foreground tensor to define a pixel time series of each stationary foreground category. Such analysis leads to the fact that SFOs can be detected easily owing to their continuous persistence in the decomposed foreground tensor. Furthermore, the unique structure of the pixel time series of each category allows identifying the category of the detected objects, whether they are abandoned or removed, and detecting the exact time of the start and end of each event. The results demonstrate that the proposed method achieves a superior performance in detecting SFOs at both object and pixel levels. In addition, the proposed method is computationally simple, and its complexity is lower compared to other approaches; hence, it can adequately satisfy real-time requirements.

Combining background subtraction algorithms with convolutional neural network

Accurate and fast extraction of foreground objects is a key prerequisite for a wide range of computer vision applications, such as object tracking and recognition. Thus, many background subtraction (BGS) methods for foreground object detection have been proposed in recent decades. However, this is still regarded as a tough problem due to a variety of challenges, such as illumination variations, camera jitter, dynamic backgrounds, and shadows. Currently, there is no single method that can handle all the challenges in a robust way. We try to solve this problem from a perspective of combining different state-of-the-art BGS algorithms to create a more robust and more advanced foreground detection algorithm. More specifically, an encoder–decoder fully convolutional neural network architecture is adapted and trained to automatically learn how to leverage the characteristics of different algorithms to fuse the results produced by different BGS algorithms and produce a more precise result. Comprehensive experiments evaluated on the CDnet 2014 dataset demonstrate that the proposed method outperforms all the considered single BGS algorithms. We show that our solution is more efficient than other BGS combination strategies.

A Content-Aware Image Retargeting Quality Assessment Method Using Foreground and Global Measurement

Image retargeting methods aim to minimize the perceptual loss while changing sizes and aspect ratios of images. Since optimal retargeting methods for different images are generally not the same, the image retargeting quality assessment (IRQA) becomes a meaningful task. This paper proposes a content-aware image retargeting quality assessment method using foreground and global measurement to achieve better performance. In our proposed method, images are first divided into two categories according to the foreground object detection result, and then different corresponding measurements are designed for them. For those with obvious foreground object, both foreground and global measurement are applied. For others, only global measurement is conducted. Foreground measurement includes two complementary features: the high-level semantic similarity feature and the low-level size ratio feature. Global measurement includes another two features: an improved aspect ratio similarity (ARS) feature and edge group similarity (EGS) feature. Two public databases, i.e., the RetargetMe and CUHK, have been evaluated, and experimental results demonstrate that our method is quite effective, and it also provides state-of-the-art performance in the IRQA. aa Our code are available at https://github.com/SCUT-ML-GUO/IRQA.

Background modeling method based on multi-feature fusion

In order to build a robust background model and improve the accuracy of detection of foreground objects, the temporal correlation of pixels at the same position of the video image and the spatial correlation of neighboring pixels are considered comprehensively. This paper proposed a background modeling method based on multi-feature fusion. By using the domain correlation of pixels in a single frame image to quickly establish an initial background model whichis updated using pixel values, frequency, update time and sensitivity of the video image sequence, the ghost phenomenon is effectively improved and the holes and false prospects for moving targets are reduced. Through multiple sets of data tests, it shows that the algorithm improves the adaptability and robustness of dynamic background and complex background.

Adaptive GMM and BP Neural Network Hybrid Method for Moving Objects Detection in Complex Scenes

Moving foreground objects detection in complex scenes is a tough job because it requires high recognition accuracy. Adaptive Gaussian mixture model (AGMM) can be used to extract the foreground objects and it shows good performance, however, the detection quality of the foreground objects under complex scenes is not excellent. In this paper, an AGMM and BP neural network hybrid method is proposed, which is used to extract the foreground objects in complex scenes such as, dynamic backgrounds, illumination changes and moving shadows. In this method, an improved BP neural network is used to post-process the images of the foreground objects that are extracted from the AGMM. The neural network has strong robustness by learning the statistical features of the images. Momentum term and adaptive learning rate are added in the BP neural network algorithm to improve the training speed and robustness of the network. The experimental results show that the proposed AGMM and BP neural network hybrid method can extract the complete foreground objects effectively when compared with some other moving objects detection algorithms.

Foreground object detection for video surveillance by fuzzy logic based estimation of pixel illumination states

Foreground object detection for video surveillance by fuzzy logic based estimation of pixel illumination states

Erratum to “Multiscale Fully Convolutional Network for Foreground Object Detection in Infrared Videos” [Apr 18 617-621

The architecture of the proposed multiscale fully convolutional network (MFCN) in our paper [1] is mainly derived from a salient object method [2] and a semantic segmentation method [3] . We missed these two reference papers in the original paper. The MFCN is an encoder-decoder architecture; the output has the same size with the input. For the encoder part, we use the pretrained VGG16 network, and for the decoder part, we upsample the features with deconvolution operations. The contrast layer for feature extraction is derived from the salient detection method [2] .

Object detection from dynamic scene using joint background modeling and fast deep learning classification

In this paper, we couple effective dynamic background modeling with fast deep learning classification to develop an accurate scheme for human-animal detection from camera-trap images with cluttered moving objects. We introduce a new block-wise background model, named as Minimum Feature Difference (MFD), to model the variation of the background of the camera-trap sequences and generate the foreground object proposals. We then develop a region proposals verification to reduce the number of false alarms. Finally, we perform complexity-accuracy analysis of DCNN to construct a fast deep learning classification scheme to classify these region proposals into three categories: human, animals, and background patches. The optimized DCNN is able to maintain high level of accuracy while reducing the computational complexity by 14 times, which allows near real-time implementation of the proposed method on CPU machines. Our experimental results demonstrate that the proposed method outperforms existing methods on our and Alexander von Humboldt Institute camera-trap datasets in both foreground segmentation and object detection.

Multiscale Fully Convolutional Network for Foreground Object Detection in Infrared Videos

Accurate and fast infrared (IR) foreground object detection is one of the most significant issues to be solved due to its important meaning for IR target recognition, IR precise guidance, IR video surveillance, and so on. A common approach for such tasks is “background subtraction,” which aims to detect foreground object through background modeling. Thus far, many background subtraction methods have been proposed and have achieved good performance. However, due to the special characteristics of IR images, a few algorithms are suitable for IR foreground object detection. Recently, features learned from convolutional neural networks (CNNs) have demonstrated great success in many vision tasks, such as classification and recognition. In this letter, we propose a novel multiscale fully convolutional network architecture for IR foreground object detection. Given a CNN model pretrained on a large-scale image data set, our method takes output features from different layers of the network. With features from multiple scales, our feature representation contains both category-level semantics and fine-grain details. The experimental results on IR image sequences show that the proposed method achieves the state-of-the-art performance while operating in real time.

Foreground Object Detection Based On Chrominance And Texture Features With Enhancement By Canny Filter

Foreground Object Detection Based On Chrominance And Texture Features With Enhancement By Canny Filter

Road Scene Content Analysis for Driver Assistance and Autonomous Driving

This paper aims to develop a vision-based driver assistance system for scene awareness using video frames obtained from a dashboard camera. A saliency image map is devised with features pertinent to the driving scene. This saliency map mimics the human contour and motion sensitive visual perception by extracting spatial, spectral, and temporal information from the input frames and applying entropy driven image-context-feature data fusion. The resultant fusion output comprises high-level descriptors for still segment boundaries and non-stationary object appearance. Following the segmentation and foreground object detection stage, an adaptive maximum likelihood classifier selects road surface regions. The proposed scene driven vision system improves the driver’s situational awareness by enabling adaptive road surface classification. As experimental results demonstrate, context-aware low-level to high-level information fusion based on human vision model produces superior segmentation, tracking, and classification results that lead to high- level abstraction of driving scene.

Foreground Object Detection and Separation Based on Region Contrast

Foreground object detection is one of the major important tasks in the field of computer vision which attempt to discover important objects in still image or image sequences or locate related targets from the scene. Foreground objects detection is very important for several approaches like object recognition, surveillance, image annotation, and image retrieval, etc. In this work, a proposed method has been presented for detection and separation foreground object from image or video in both of moving and stable targets. Comparisons with general foreground detectors such as background subtraction techniques our approach are able to detect important target for case the target is moving or not and can separate foreground object with high details.

Moving object detection algorithm based on pixel spatial sample difference consensus

Moving object detection is an essential component for security video surveillance system and other computer vision applications. Although the latest object detection methods get promising detection expectations, however, accurate detection is still a tricky problem due to various challenges such as aperture effects, illumination variations, camouflage issues and retention problems in unconstrained video environments. In this paper, we propose a brand-new theoretical framework for foreground object detection based on the stable spatial relationship between current pixel and the randomly selected pixels in current frame. Different from the existing methods which determine the moving object area by comparing each pixel value with its surrounding pixels or by comparing two pixel values occupying the same positions in adjacent frames, the proposed algorithm sets up a spatial sample set for each individual pixel and defines Spatial Sample Difference Consensus (SSDC), which denotes changes of stable spatial relationship rather than direct changes in pixel values. Thus, the proposed algorithm computes the SSDC between two adjacent frames to subtract the moving objects. The experiments on recent data-set in both indoor and outdoor surveillance video sequences show that the proposed method achieved promising performance after compared with several state-of-the art methods.

An optimised background modelling for efficient foreground extraction

Nowadays, analysing videos from a surveillance system in real-time is very important for resolving the security related social issues. Foreground extraction and object detection is a vital task in video analysis. In the proposed methods background, modelling is treated as an optimisation problem and solved using particle swarm optimisation. The background is modelled at regular intervals of time for adapting the changes in the environment. Then the background subtraction is applied to the current frame with the corresponding background modelled frame to extract the foreground. Added to it the optical flow applied image is compared with the foreground extracted image to avoid the false positives (FP) and false negatives (FN). This proposed foreground extraction technique for real-time videos gives results better than the previous algorithms with respect to the quality of extraction and space complexity.

An optimised background modelling for efficient foreground extraction

Nowadays, analysing videos from a surveillance system in real-time is very important for resolving the security related social issues. Foreground extraction and object detection is a vital task in video analysis. In the proposed methods background, modelling is treated as an optimisation problem and solved using particle swarm optimisation. The background is modelled at regular intervals of time for adapting the changes in the environment. Then the background subtraction is applied to the current frame with the corresponding background modelled frame to extract the foreground. Added to it the optical flow applied image is compared with the foreground extracted image to avoid the false positives (FP) and false negatives (FN). This proposed foreground extraction technique for real-time videos gives results better than the previous algorithms with respect to the quality of extraction and space complexity.

Recovering low-rank and sparse matrix based on the truncated nuclear norm

Recovering the low-rank, sparse components of a given matrix is a challenging problem that arises in many real applications. Existing traditional approaches aimed at solving this problem are usually recast as a general approximation problem of a low-rank matrix. These approaches are based on the nuclear norm of the matrix, and thus in practice the rank may not be well approximated. This paper presents a new approach to solve this problem that is based on a new norm of a matrix, called the truncated nuclear norm (TNN). An efficient iterative scheme developed under the linearized alternating direction method multiple framework is proposed, where two novel iterative algorithms are designed to recover the sparse and low-rank components of matrix. More importantly, the convergence of the linearized alternating direction method multiple on our matrix recovering model is discussed and proved mathematically. To validate the effectiveness of the proposed methods, a series of comparative trials are performed on a variety of synthetic data sets. More specifically, the new methods are used to deal with problems associated with background subtraction (foreground object detection), and removing shadows and peculiarities from images of faces. Our experimental results illustrate that our new frameworks are more effective and accurate when compared with other methods.

Fast illumination-robust foreground detection using hierarchical distribution map for real-time video surveillance system

Foreground detection is one of the most important and fundamental tasks in many computer vision applications such as real-time video surveillance. Although there have been many efforts to find solutions to this problem, many obstacles such as illumination changes, noises, dynamic backgrounds, and computational complexities have prevented them from being used in real surveillance systems. In this paper, to alleviate these inherent limitations of conventional methods, we propose a fast illumination-robust foreground detection (FIFD) system that provides robustness against illumination variations and noises from various real circumstances with an efficient computational scheme. In contrast to the conventional approaches, our method focuses on efficiently formulating the foreground object detection system by leveraging a foreground candidate region detection and hierarchical distribution map. Specifically, our approach consists of three parts. First, for a query image, foreground candidates are detected by fusing multiple methods. The existence and the block size of the foreground object are determined through the use of the foreground continuity. Second, the foreground block is found from the estimated distribution map and then detected from the extracted valid blocks. Finally, with a labeling scheme, the foreground is detected. To intensively evaluate our approach compared to the conventional methods, we use the publicly available I2R and traffic datasets, and we build a novel electron multiplying charge-coupled device foreground detection benchmark taken in an environment with light lower than 10lux. Experimental results show that our approach provides satisfactory performance compared to the state-of-the-art methods even under very challenging circumstances. Furthermore, our approach is very efficient in that it takes only approximately 31 ms per frame.