Foreground Frame Research Articles

Fully Automatic Approach for Smoke Tracking Based on Deep Image Quality Enhancement and Adaptive Level Set Model

In recent decades, the need for advanced systems with good precision, low cost, and high-time response for wildfires and smoke detection and monitoring has become an absolute necessity. In this paper, we propose a novel, fast, and autonomous approach for denoising and tracking smoke in video sequences captured from a camera in motion. The proposed method is based mainly on two stages: the first one is a reconstruction and denoising path with a novel lightweight convolutional autoencoder architecture. The second stage is a specific scheme designated for smoke tracking, and it consists of the following: first, the foreground frames are extracted with the HSV color model and textural features of smoke; second, possible false detections of smoke regions are eliminated with image processing technique and last smoke contours detection is performed with an adaptive nonlinear level set. The obtained experimental results exposed in this paper show the potential of the proposed approach and prove its efficiency in smoke video denoising and tracking with a minimized number of false negative regions and good detection rates.

HubPUG: proper motions for local group dwarfs observed with HST utilizing Gaia as a reference frame

ABSTRACT We present the method behind HubPUG, a software tool built for recovering systemic proper motions (PMs) of targets (e.g. clusters or resolved galaxies) in Hubble Space Telescope (HST) fields with two epochs of observations by utilizing stars observed by Gaia as a foreground frame of reference.HST PM experiments have typically relied on the use of distant background galaxies or quasi-stellar objects (QSOs) as stationary sources against which to measure PMs. Without consistent profiles, background galaxies are more difficult to centroid, but benefit on-aggregate from their large numbers. QSOs, though they can be fit with stellar point-spread functions, are sparse, with most fields containing none. Historically, the use of stars as references against which to measure PMs would have been difficult because they have individual PMs of their own. However, Gaia has now provided positions and PMs for over 1.4 billion stars, which are much more likely to be well-imaged in the fields around targets versus background sources, have predictable stellar profiles, and require less observing time per-image for good signal-to-noise. This technique allows us to utilize the power of Gaia to measure the PM of targets too faint for Gaia to observe itself. We have recovered PMs for the Milky Way satellites Sculptor and Draco with comparable uncertainties over HST-only and Gaia-only measurements, limited primarily by the current capabilities of the Gaia data. We also show the promise of this method for satellites of M31 with a new PM measurement for Andromeda VII.

Low-Cost Approach for Improving Video Transmission Efficiency in WVSN

The wireless visual sensor network (WVSN) as a new emerged intelligent visual system, has been applied in many video monitoring sites. However, there is still great challenge because of the limited wireless network bandwidth. To resolve the problem, we propose a real-time dynamic texture approach which can detect and reduce the temporal redundancy during many successive image frames. Firstly, an adaptively learning background model is improved to discover successive similar image frames from the inputting video sequence. Then, the dynamic texture model based on the singular value decomposition is adopted to distinguish foreground and background element dynamics. Furthermore, a background discarding strategy based on visual motion coherence is proposed to determine whether each image frame is streamed or not. To evaluate the trade-off performance of the proposed method, it is tested on the CDW-2014 dataset, which can accurately detect the first foreground frame when the moving objects of interest appear in the field of view in the most tested dynamic scenes, and the misdetection rate of the undetected foreground frames is near to zero. Compared to the original stream, it can reduce the occupied bandwidth a lot and its computational cost is relatively lower than the state-of-the-art methods.

Rethinking the Bottom-Up Framework for Query-Based Video Localization

In this paper, we focus on the task query-based video localization, i.e., localizing a query in a long and untrimmed video. The prevailing solutions for this problem can be grouped into two categories: i) Top-down approach: It pre-cuts the video into a set of moment candidates, then it does classification and regression for each candidate; ii) Bottom-up approach: It injects the whole query content into each video frame, then it predicts the probabilities of each frame as a ground truth segment boundary (i.e., start or end). Both two frameworks have respective shortcomings: the top-down models suffer from heavy computations and they are sensitive to the heuristic rules, while the performance of bottom-up models is behind the performance of top-down counterpart thus far. However, we argue that the performance of bottom-up framework is severely underestimated by current unreasonable designs, including both the backbone and head network. To this end, we design a novel bottom-up model: Graph-FPN with Dense Predictions (GDP). For the backbone, GDP firstly generates a frame feature pyramid to capture multi-level semantics, then it utilizes graph convolution to encode the plentiful scene relationships, which incidentally mitigates the semantic gaps in the multi-scale feature pyramid. For the head network, GDP regards all frames falling in the ground truth segment as the foreground, and each foreground frame regresses the unique distances from its location to bi-directional boundaries. Extensive experiments on two challenging query-based video localization tasks (natural language video localization and video relocalization), involving four challenging benchmarks (TACoS, Charades-STA, ActivityNet Captions, and Activity-VRL), have shown that GDP surpasses the state-of-the-art top-down models.

Efficient Sparse Representation based Action Recognition in video

Human Action Recognition (HAR) is an interesting and helpful topic in various real-life applications such as surveillance based security system, computer vision and robotics. The selected features and feature representation methods, classification algorithms decides the accuracy of the HAR systems. A new feature called, Skeletonized STIP (Spatio Temporal Interest Points) is identified and used in this work. The skeletonization on the action video’s foreground frames are performed and the new feature is generated as STIP values of the skeleton frame sequence. Then the feature set is used for initial dictionary construction in sparse coding. The data for action recognition is huge, since the feature set is represented using the sparse representation. To refine the sparse representation the max pooling method is used and the action recognition is performed using SVM classifier. The proposed approach outperforms on the benchmark datasets.

MOVING OBJECT DETECTION USING SPATIAL CORRELATION IN LAB COLOUR SPACE

Abstract. Background subtraction-based techniques of moving object detection are very common in computer vision programs. Each technique of background subtraction employs image thresholding algorithms. Different thresholding methods generate varying threshold values that provide dissimilar moving object detection results. A majority of background subtraction techniques use grey images which reduce the computational cost but statistics-based image thresholding methods do not consider the spatial distribution of pixels. In this study, authors have developed a background subtraction technique using Lab colour space and used spatial correlations for image thresholding. Four thresholding methods using spatial correlation are developed by computing the difference between opposite colour pairs of background and foreground frames. Out of 9 indoor and outdoor scenes, the object is detected successfully in 7 scenes whereas existing background subtraction technique using grey images with commonly used thresholding methods detected moving objects in 1–5 scenes. Shape and boundaries of detected objects are also better defined using the developed technique.

A Competent Frame Work for Efficient Object Detection, Tracking and Classification

Observation is the rising idea in the present innovation, as it assumes a key part in checking sharp exercises at the niches and corner of the world. Among which moving Object distinguishing and following by methods for PC vision systems is the significant part in reconnaissance. On the off chance that we consider moving object recognition in video investigation is the underlying stride among the different PC applications In this paper, we proposed robust video object detection and tracking technique. The proposed technique is divided into three phases namely detection phase, tracking phase and evaluation phase in which detection phase contains Foreground segmentation and Noise reduction. Mixture of Adaptive Gaussian model is proposed to achieve the efficient foreground segmentation. In addition to it the fuzzy morphological filter model is implemented for removing the noise present in the foreground segmented frames. Moving object tracking is achieved by the blob detection which comes under tracking phase. Finally, the evaluation phase has feature extraction and classification. Texture based and quality based features are extracted from the processed frames which is given for classification in weka. There are three classifiers such as J48, k-nearest neighbor and Multilayer perceptron are used. The performance of the proposed technique is measured through evaluation phase and is tabulated.

Human Action Recognition Using Median Background and Max Pool Convolution with Nearest Neighbor

Discovering human activities on mobile devices is a challenging task for human action recognition. The ability of a device to recognize its user's activity is important because it enables context-aware applications and behavior. Recently, machine learning algorithms have been increasingly used for human action recognition. During the past few years, principal component analysis and support vector machines is widely used for robust human activity recognition. However, with global dynamic tendency and complex tasks involved, this robust human activity recognition (HAR) results in error and complexity. To deal with this problem, a machine learning algorithm is proposed and explores its application on HAR. In this article, a Max Pool Convolution Neural Network based on Nearest Neighbor (MPCNN-NN) is proposed to perform efficient and effective HAR using smartphone sensors by exploiting the inherent characteristics. The MPCNN-NN framework for HAR consists of three steps. In the first step, for each activity, the features of interest or foreground frame are detected using Median Background Subtraction. The second step consists of organizing the features (i.e. postures) that represent the strongest generic discriminating features (i.e. postures) based on Max Pool. The third and the final step is the HAR based on Nearest Neighbor that postures which maximizes the probability. Experiments have been conducted to demonstrate the superiority of the proposed MPCNN-NN framework on human action dataset, KARD (Kinect Activity Recognition Dataset).

Compressive Online Video Background–Foreground Separation Using Multiple Prior Information and Optical Flow

In the context of video background–foreground separation, we propose a compressive online Robust Principal Component Analysis (RPCA) with optical flow that separates recursively a sequence of video frames into foreground (sparse) and background (low-rank) components. This separation method operates on a small set of measurements taken per frame, in contrast to conventional batch-based RPCA, which processes the full data. The proposed method also leverages multiple prior information by incorporating previously separated background and foreground frames in an n-ℓ1 minimization problem. Moreover, optical flow is utilized to estimate motions between the previous foreground frames and then compensate the motions to achieve higher quality prior foregrounds for improving the separation. Our method is tested on several video sequences in different scenarios for online background–foreground separation given compressive measurements. The visual and quantitative results show that the proposed method outperforms other existing methods.

Approach for moving object detection using visible spectrum and thermal infrared imaging

Moving object detection is one of the most promising research areas, which is required in different applications, such as video monitoring and surveillance systems, human activity recognition systems, vehicle counting, and anomaly detection. Various methods for object detection using single sensor and a few using multimodal techniques have been reported in the literature. However, such systems fail to handle adverse or challenging atmospheric conditions such as illumination variations, scale and appearance change of objects or targets, occlusions, and camouflaged conditions. We have presented an approach for the detection of moving objects using structural similarity metric (SSIM) and Gaussian mixture model (GMM). SSIM is used to compute similarity between reference mean background frame and foreground frame of visible spectrum (VIS) and thermal infrared (IR) independently. The computation of similarity measure is performed in an image spatial domain. The threshold results of SSIM are fused together using different pixel-level fusion methods such logical “OR,” discrete wavelet transform, and principal components analysis. Temporal analysis is performed to eliminate noise and false positives (unwanted background regions) using GMM on fused results. We have compared the results with recent methods for different complex scenarios and found out that approximately F -measure increases up to 80%. Hence, the proposed method proves to be a robust moving object detection technique in multimodality domain.

Feature Tracking Velocimetry Applied to Airborne Measurement Data from Murg Creek

A new image feature tracking velocimetry is presented and tested on airborne video data available from a previous study at Murg Creek (Canton Thurgau, Switzerland). Here, the seeded flow scenery had been recorded by an off-the-shelf action camera mounted to a low-cost quadcopter, and video frames were ortho-rectified to sizes of 4482×2240 px2 at a scale of 64 px/m. The new velocimetry approach is as follows: An adaptive Gaussian mixture model is used for video background subtraction. Then, scale-invariant keypoints on each remaining binary foreground image frame are determined by a feature detection algorithm, and corresponding feature points in subsequent frame pairs are matched using the iterative random sample consensus method. The related feature shifts in metric space divided by the video frame rate finally give the velocity vectors. The obtained velocimetry fields are compared with findings from both a particle image velocimetry and particle tracking velocimetry analysis in terms of accuracy and needed computational power. Indication is given that the feature tracking algorithm presents slightly less precise results, but clearly outperforms the other two in relation to computational power. Therefore, the new simplified method provides a high potential tool that may enable a future way to real time surface velocity measurements obtained from unmanned airborne vehicles.

Shadow removal for pedestrian detection and tracking in indoor environments

This paper presents a method of shadow removal to improve the accuracy of pedestrian detection and tracking in indoor environments. The proposed method can be divided into four steps: building a background model which can be automatically updated, extract moving objects region, eliminating moving objects shadows, classifying and track pedestrians. The background model is built with pixel value and the updating of Gussian. The approach for real time background-foreground extraction is used to extract pedestrian region that may contains multiple shadows. In the gray histogram space, based on the depth value of the gray images, a reasonable threshold is set to remove shadows from various pedestrians. In this work, we propose a methodology using the foreground frames without shadows to detect and track the pedestrians across training datasets. Comparative experimental results show that our method is capable of dealing with shadows and detecting moving pedestrians in cluttered environments.

A time-slice optimization based weak feature association algorithm for video condensation

There are a lot of complex environments in real scene, such as illumination variation, shadow variation, object occlusion, which will directly affect the performance of video synopsis. In this paper, we adopt Grid Background Model as object detection algorithm, proposing algorithm based on weak feature to solve the object occlusion problem, at last we propose to use time-slice optimization algorithm to solve the visualization problem of video condensation. Specifically, Grid Background Model is adopted to segment the foreground from the background, then we use current frame to update background frame, and then binarize the foreground frame to perform Neighborhood illumination invariant shadow elimination. A clear foreground can be obtained by doing the procedure above as well as Gaussian noise elimination and morphological operation such as inflation and corrosion to remove cavities. Meanwhile, the outline of the object is extracted by using the canny edge detector. In the object tracking section, we will introduce how to use the weak features, such as color, speed and direction on the basis of location prediction based on tracking algorithm to perform object association, and the extraction of accurate information of abstract and outline of the object at the same time. Finally, in the video condensation section, we will describe how to use optical time-slice based minimum energy model to perform video condensation according to frame sequence. The experimental result shows that, the method mentioned above can provide a new approach for solving the occlusion problems of video condensation, and have better visualization of abstract video, and achieve up to 6 times concentration to the original video.

Action-Based Pedestrian Identification via Hierarchical Matching Pursuit and Order Preserving Sparse Coding

Pedestrian identification is a very important topic in the area of intelligent surveillance and public safety, where the near front face images of pedestrian can hardly be obtained due to high installation angle of camera, long-distance location and extreme light variations. This paper presents a new action-based pedestrian identification algorithm, which adopts hierarchical matching pursuit (HMP) to extract features and order preserving sparse coding (OPSC) to do classification. Two-layer HMP features are extracted from foreground frame image patches by sparse coding, max pooling and normalization, which preserve both local and global information. OPSC is taken as classifier to take full advantage of the spatial structure information, which is different from traditional temporal OPSC algorithm. The spatiotemporal order preserving sparse coding-based classification is also investigated. The effectiveness of the proposed method is verified on public data sets, and the experimental results show the superiority of our method.

Galleries of the Stelvio-Lake Como: Sanford Robinson Gifford.

Galleries of the Stelvio-Lake Como: Sanford Robinson Gifford.

Detection of abnormal events via optical flow feature analysis.

In this paper, a novel algorithm is proposed to detect abnormal events in video streams. The algorithm is based on the histogram of the optical flow orientation descriptor and the classification method. The details of the histogram of the optical flow orientation descriptor are illustrated for describing movement information of the global video frame or foreground frame. By combining one-class support vector machine and kernel principal component analysis methods, the abnormal events in the current frame can be detected after a learning period characterizing normal behaviors. The difference abnormal detection results are analyzed and explained. The proposed detection method is tested on benchmark datasets, then the experimental results show the effectiveness of the algorithm.

A framework for suspicious object detection from surveillance video

Detection of suspicious objects from a surveillance video is an important and challenging task. This paper presents a method to detect a suspicious object from surveillance video. It consists of three main steps. In the first step, background subtraction is performed through background modelling using a running average method which makes the system efficient to detect foreground objects. In the second step, static objects are detected from foreground frames through contour features and geometric histogram of an object. In the third step, static objects are classified into human and non-human objects using skin colour region detection and edge-based object recognition method. Edge-based object recognition method is efficient to recognise full and partially visible object. If static object is a non-human, an alarm is raised after a specified time. Experimental results have been performed on the IEEE dataset for Performance Evaluation of Tracking and Surveillance (PETS) and own dataset. The results demonstrate that proposed system is suitable for real-time surveillance video with detection accuracy of 90.9%.

Anomaly Detection Based on Motion Direction

A novel algorithm is proposed in this paper to detect anomalous human behaviors based on motion directions. According to different motion direction rules for different events, we introduce block-based motion directions to model those events, and use support vector machine (SVM) to detect the abnormalous actions from real-time monitoring video sequences. To increase the robustness against noise and to capture the slight movement of the object, we select the foreground frames (the frames having human object) with a background edge model before the action feature extraction. Then, action features are extracted using normalized histogram analysis from the motion directions of all the foreground frames. Our experiments on public areas such as hallway show that our algorithm is able to track complex actions of single and multiple people accurately and is robust against the variation of object size, lighting, and noise during their movements. Our algorithm is of low computation complexity thus it can be used for real time monitoring.