Motion Detection Scheme Research Articles

Smoke detection in video using convolutional neural networks and efficient spatio-temporal features

Fire detection in its early stages is of a great importance in different environmental related applications. Among the visual signs of fire, smoke appears earlier than the flames in many cases, and quickly reaches the environment. Thus, it can be used for early detection of fire using machine vision techniques. Existing approaches have tried to do it either by traditional machine learning methods applying various combinations of color, texture, and motion features, or by deep learning-based methods that can automatically capture the smoke features from images. However, the former approaches face number of challenges due to the transparent nature of smoke and its variant visual appearance in different environments, and the later ones are not able to capture motion characteristics of smoke, so, their efficiency in various environmental conditions is still problematic and often cause false alarms. In this study, we aim to introduce a hybrid approach that is based on deep learning, spatio, and spatio-temporal characteristics of the smoke. Doing so, we use all the strengths of these techniques to detect the smoke as accurately as possible. The proposed method consists of four stages: 1) moving pixels are extracted from input images by an efficient motion detection scheme; 2) the extracted moving areas are given individually to a tailored convolutional neural network to identify candidate smoke regions; 3) an efficient combination of spatial and spatio-temporal features is extracted from each candidate region based on the distinct characteristics of smoke; 4) a support vector machine classifier is used to further classify real smoke from non-smoke regions using the extracted features. The proposed method is implemented using Python programming language, and extensive experiments conducted on “Visor”, “Bilkent”, and “Yuan” benchmark datasets which show it has high performance and accuracy. The results also indicate that its reliability is far better than the competitors in terms of false alarm rate. The average accuracy and false positive rates obtained on 10 testing videos are 97.63% and 3.8%, respectively. Also, the proposed method is able to detect both white and black smokes which, to our best knowledge, has not been addressed in any of the related researches.

Event-based imaging velocimetry applied to a cylinder wake flow in air

Event-based vision is a new upcoming field within the field of computer vision. Unlike conventional frame-based imaging, event-based vision (or dynamic vision sensing) asynchronously records binary signals of contrast changes at the pixel level with microsecond resolution. Event-based imaging of small particles illuminated by a laser or other continuous light source generates streaks of events in a space-time domain. From this data the particles’ position in both space and time can be inferred. The present work explores the possibilities of harnessing the potentials of event- based vision for fluid flow measurement with focus on two-dimensional, two-component velocity field estimation at particle image densities corresponding to that of conventional PIV. Three different motion detection schemes are implemented. Measurements of a cylinder wake flow demonstrate that EBIV provides time resolved velocity data that is suitable for the retrieval of flow statistics as well as spectral and modal analysis. The present work is an extension of the recently published Exp. Fluids article by the authors on event-based imaging velocimetry and provides further details on current limitations related to event-based imaging.

Event-based imaging velocimetry: an assessment of event-based cameras for the measurement of fluid flows

Contrary to conventional frame-based imaging, event-based vision or dynamic vision sensing asynchronously records binary signals of intensity changes for given pixels with microsecond resolution. The present work explores the possibilities of harnessing the potential of event-based vision for fluid flow measurement. The described implementations of event-based imaging velocimetry rely on imaging small particles that are illuminated by a laser light sheet which is similar to classical two-dimensional, two-component (2d–2c) PIV with the difference that a continuously operating laser light sheet is used without modulation of the laser or camera. The moving particles generate continuous time-stamped events on the detector that are later used to infer their velocity using patch-wise processing schemes. Two flow estimation algorithms are proposed; one uses a “motion compensation” that maximizes the local contrast and the other is based on a sum-of-correlations approach. The underlying motion detection schemes in combination with the complete absence of background signal allow straightforward retrieval of the events associated with individual particles thereby allowing the reconstruction of individual particle tracks. Alternatively, the event data can be processed with conventional PIV algorithms using images reconstructed from the event data stream. The concepts are demonstrated on simple flows in water and air.Graphic abstract

Robust Privacy-Preserving Motion Detection and Object Tracking in Encrypted Streaming Video

Video privacy leakage is becoming an increasingly severe public problem, especially in cloud-based video surveillance systems. It leads to the new need for secure cloud-based video applications, where the video is encrypted for privacy protection. Despite some methods that have been proposed for encrypted video moving object detection and tracking, none has robust performance against complex and dynamic scenes. In this paper, we propose an efficient and robust privacy-preserving motion detection and multiple object tracking scheme for encrypted surveillance video bitstreams. By analyzing the properties of the video codec and format-compliant encryption schemes, we propose a new compressed-domain feature to capture motion information in complex surveillance scenarios. Based on this feature, we design an adaptive clustering algorithm for moving object segmentation with an accuracy of 4×4 pixels. We then propose a multiple object tracking scheme that uses Kalman filter estimation and adaptive measurement refinement. The proposed scheme does not require video decryption or full decompression and has a very low computation load. The experimental results demonstrate that our scheme achieves the best detection and tracking performance compared with existing works in the encrypted and compressed domain. Our scheme can be effectively used in complex surveillance scenarios with different challenges, such as camera movement/jitter, dynamic background, and shadows.

Simple optical knife-edge effect based motion detection approach for a microcantilever

Here, we describe a simple optical motion detection method for a microcantilever by positioning a microcantilever in front of a photodetector and focusing a light to the side of the cantilever in the in-plane direction. The out-of-plane vibration of the microcantilever, acting as an knife-edge itself, modulates the amount of light reaching the photodetector, allowing the alternate motion detection scheme. The method presented offers the optical responsivity around 0.3 mW/μm, which is high enough to observe the thermomechanical fluctuations of a microcantilever in order of a few pm/Hz. The demonstrated detection scheme can reduce the complexity and the relative size of the detection configuration, often required in the conventional methods, and would be useful as a complementary technique to existing ones.

High-accuracy scheme based on a look-up table for motion detection in an optical camera communication system.

In this paper, a high-accuracy motion detection (MD) scheme based on a look-up table (LUT) is proposed and experimentally demonstrated in an optical camera communication (OCC) system. The LUT consists of predefined motions and strings that represent the predefined motions. The predefined motions include straight lines, polylines, circles, and number shapes. At the transmitter, the data with on-off keying (OOK) format is modulated on an 8×8 light-emitting diode (LED) array. The motion is generated by the user's finger in the free space link. At the receiver, the motion and data are captured by the mobile phone front camera. The captured motion is expressed as a string indicating directions of motion, then it is matched as a predefined motion in LUT by calculating the Levenshtein distance (LD) and modified Jaccard coefficient (MJC). Using the proposed scheme, four types of motions are recognized accurately and data transmission is achieved simultaneously. Also, 1760 motion samples from 4 users are investigated over the free space transmission. The experimental results show that the accuracy of the proposed MD scheme can reach 98% at the distance without the loss of finger centroids. In addition, as the transmitter is not blocked, the bit error rate (BER) is below 1e-6 at the distance of 80cm.

Performance evaluation of neural network assisted motion detection schemes implemented within indoor optical camera based communications.

This paper investigates the performance of the neural network (NN) assisted motion detection (MD) over an indoor optical camera communication (OCC) link. The proposed study is based on the performance evaluation of various NN training algorithms, which provide efficient and reliable MD functionality along with vision, illumination, data communications and sensing in indoor OCC. To evaluate the proposed scheme, we have carried out an experimental investigation of a static indoor downlink OCC link employing a mobile phone front camera as the receiver and an 8 × 8 red, green and blue light-emitting diodes array as the transmitter. In addition to data transmission, MD is achieved using a camera to observe user's finger movement in the form of centroids via the OCC link. The captured motion is applied to the NN and is evaluated for a number of MD schemes. The results show that, resilient backpropagation based NN offers the fastest convergence with a minimum error of 10-5 within the processing time window of 0.67 s and a success probability of 100 % for MD compared to other algorithms. We demonstrate that, the proposed system with motion offers a bit error rate which is below the forward error correction limit of 3.8 × 10-3, over a transmission distance of 1.17 m.

Multi-Target Intense Human Motion Analysis and Detection Using Channel State Information

Intense human motion, such as hitting, kicking, and falling, in some particular scenes indicates the occurrence of abnormal events like violence and school bullying. Camera-based human motion detection is an effective way to analyze human behavior and detect intense human motion. However, even if the camera is properly deployed, it will still generate blind spots. Moreover, camera-based methods cannot be used in places such as restrooms and dressing rooms due to privacy issues. In this paper, we propose a multi-target intense human motion detection scheme using commercial Wi-Fi infrastructures. Compared with human daily activities, intense human motion usually has the characteristics of intensity, rapid change, irregularity, large amplitude, and continuity. We studied the changing pattern of Channel State Information (CSI) influenced by intense human motion, and extracted features in the pattern by conducting a large number of experiments. Considering occlusion exists in some complex scenarios, we distinguished the Line-of-Sight (LOS) and Non-Line-of-Sight (NLOS) conditions in the case of obstacles appearing between the transmitter and the receiver, which further improves the overall performance. We implemented the intense human motion detection system using single commercial Wi-Fi devices, and evaluated it in real indoor environments. The experimental results show that our system can achieve intense human motion detection rate of 90%.

An Efficient Motion Detection and Tracking Scheme for Encrypted Surveillance Videos

Performing detection on surveillance videos contributes significantly to the goals of safety and security. However, performing detection on unprotected surveillance video may reveal the privacy of innocent people in the video. Therefore, striking a proper balance between maintaining personal privacy while enhancing the feasibility of detection is an important issue. One promising solution to this problem is to encrypt the surveillance videos and perform detection on the encrypted videos. Most existing encrypted signal processing methods focus on still images or small data volumes; however, because videos are typically much larger, investigating how to process encrypted videos is a significant challenge. In this article, we propose an efficient motion detection and tracking scheme for encrypted H.264/AVC video bitstreams, which does not require the previous decryption on the encrypted video. The main idea is to first estimate motion information from the bitstream structure and codeword length and, then, propose a region update (RU) algorithm to deal with the loss and error drifting of motion caused by the video encryption. The RU algorithm is designed based on the prior knowledge that the object motion in the video is continuous in space and time. Compared to the existing scheme, which is based on video encryption that occurs at the pixel level, the proposed scheme has the advantages of requiring only a small storage of the encrypted video and has a low computational cost for both encryption and detection. Experimental results show that our scheme performs better regarding detection accuracy and execution speed. Moreover, the proposed scheme can work with more than one format-compliant video encryption method, provided that the positions of the macroblocks can be extracted from the encrypted video bitstream. Due to the coupling of video stream encryption and detection algorithms, our scheme can be directly connected to the video stream output (e.g., surveillance cameras) without requiring any camera modifications.

Optical Camera Communication: Motion over Camera

OCC is a pragmatic version of VLC based on a smart device camera that allows easier implementation of various services in smart devices. This article presents a flexible and novel motion detection scheme over a smart device camera in OCC. The motion detection is conducted in conjunction with a static downlink optical camera communication, where a mobile phone front camera is employed as the receiver and an 8 x 8 dot matrix LED as the transmitter. In addition to the 8 x 8 dot matrix LED for data transmission, 10 white LEDs are also employed for providing illumination, acquiring camera focus, and light metering. The motion detection or MoC is designed to detect the user's finger movement through the OCC link via the camera. A simple but efficient quadrant division based motion detection algorithm is proposed for reliable and accurate detection of motion. The experiment and simulation results demonstrate that the proposed scheme is able to detect motion with a success probability of up to 96 percent in the mobile phone camera based OCC. It is envisioned that the proposed motion detection can facilitate cost-effective and convenient smart home environments in the OCC, where the provision of illumination and short-range wireless communications has already been addressed.

Robust and passive motion detection with COTS WiFi devices

Device-free Passive (DfP) detection has received increasing attention for its ability to support various pervasive applications. Instead of relying on variable Received Signal Strength (RSS), most recent studies rely on finer-grained Channel State Information (CSI). However, existing methods have some limitations, in that they are effective only in the Line-Of-Sight (LOS) or for more than one moving individual. In this paper, we analyze the human motion effect on CSI and propose a novel scheme for Robust Passive Motion Detection (R-PMD). Since traditional low-pass filtering has a number of limitations with respect to data denoising, we adopt a novel Principal Component Analysis (PCA)-based filtering technique to capture the representative signals of human motion and extract the variance profile as the sensitive metric for human detection. In addition, existing schemes simply aggregate CSI values over all the antennas in MIMO systems. Instead, we investigate the sensing quality of each antenna and aggregate the best combination of antennas to achieve more accurate and robust detection. The R-PMD prototype uses off-the-shelf WiFi devices and the experimental results demonstrate that R-PMD achieves an average detection rate of 96.33% with a false alarm rate of 3.67%.

Memory Efficient VLSI Implementation of Real-Time Motion Detection System Using FPGA Platform

Motion detection is the heart of a potentially complex automated video surveillance system, intended to be used as a standalone system. Therefore, in addition to being accurate and robust, a successful motion detection technique must also be economical in the use of computational resources on selected FPGA development platform. This is because many other complex algorithms of an automated video surveillance system also run on the same platform. Keeping this key requirement as main focus, a memory efficient VLSI architecture for real-time motion detection and its implementation on FPGA platform is presented in this paper. This is accomplished by proposing a new memory efficient motion detection scheme and designing its VLSI architecture. The complete real-time motion detection system using the proposed memory efficient architecture along with proper input/output interfaces is implemented on Xilinx ML510 (Virtex-5 FX130T) FPGA development platform and is capable of operating at 154.55 MHz clock frequency. Memory requirement of the proposed architecture is reduced by 41% compared to the standard clustering based motion detection architecture. The new memory efficient system robustly and automatically detects motion in real-world scenarios (both for the static backgrounds and the pseudo-stationary backgrounds) in real-time for standard PAL (720 × 576) size color video.

FPGA-Based Real-Time Motion Detection for Automated Video Surveillance Systems

Design of automated video surveillance systems is one of the exigent missions in computer vision community because of their ability to automatically select frames of interest in incoming video streams based on motion detection. This research paper focuses on the real-time hardware implementation of a motion detection algorithm for such vision based automated surveillance systems. A dedicated VLSI architecture has been proposed and designed for clustering-based motion detection scheme. The working prototype of a complete standalone automated video surveillance system, including input camera interface, designed motion detection VLSI architecture, and output display interface, with real-time relevant motion detection capabilities, has been implemented on Xilinx ML510 (Virtex-5 FX130T) FPGA platform. The prototyped system robustly detects the relevant motion in real-time in live PAL (720 × 576) resolution video streams directly coming from the camera.

Coping with real world data: Artifact reduction and denoising for motion-compensated cardiac C-arm CT.

Detailed analysis of cardiac motion would be helpful for supporting clinical workflow in the interventional suite. With an angiographic C-arm system, multiple heart phases can be reconstructed using electrocardiogram gating. However, the resulting angular undersampling is highly detrimental to the quality of the reconstructed images, especially in nonideal intraprocedural imaging conditions. Motion-compensated reconstruction has previously been shown to alleviate this problem, but it heavily relies on a preliminary reconstruction suitable for motion estimation. In this work, the authors propose a processing pipeline tailored to augment these initial images for the purpose of motion estimation and assess how it affects the final images after motion compensation. The following combination of simple, direct methods inspired by the core ideas of existing approaches proved beneficial: (a) Streak reduction by masking high-intensity components in projection domain after filtering. (b) Streak reduction by subtraction of estimated artifact volumes in reconstruction domain. (c) Denoising in spatial domain using a joint bilateral filter guided by an uncompensated reconstruction. (d) Denoising in temporal domain using an adaptive Gaussian smoothing based on a novel motion detection scheme. Experiments on a numerical heart phantom yield a reduction of the relative root-mean-square error from 89.9% to 3.6% and an increase of correlation with the ground truth from 95.763% to 99.995% for the motion-compensated reconstruction when the authors' processing is applied to the initial images. In three clinical patient data sets, the signal-to-noise ratio measured in an ideally homogeneous region is increased by 37.7% on average. Overall visual appearance is improved notably and some anatomical features are more readily discernible. The authors' findings suggest that the proposed sequence of steps provides a clear advantage over an arbitrary sequence of individual image enhancement methods and is fit to overcome the issue of lacking image quality in motion-compensated C-arm imaging of the heart. As for future work, the obtained results pave the way for investigating how accurately cardiac functional motion parameters can be determined with this modality.

Depth Motion Detection—A Novel RS-Trigger Temporal Logic based Method

Recently, depth data is widely used in computer vision applications such as detection and tracking, which shows great promises in complicated environments due to its complementary natures to RGB data. However, previous works mostly use depth as an auxiliary cue of RGB data and overlook its inherent advantage on motion detection. Intrinsically different from RGB data, points in depth map essentially represents 3-D positions in the world, so depth video represents the variation of these “positions,” which is motion. Motivated by this, we proposed a novel motion detection scheme based on RS-Trigger temporal logic which best fits nature of depth data on motion detection. The proposed algorithm can fast detect motion regions in the scene without statistics of background and prior knowledge of objects to detect. In following refinement modules, a depth-invariant density-constant projection is proposed which contributes to a fast spatial clustering and accurate segmentation, for it transforms dense 3-D points cloud to depth-invariant 2-D map with density-constance, not only it overcomes depth-dependent sampling of depth sensor, but also overcomes the common `scale problem' in 2-D image analysis, which makes it easy to set system parameters to de-noise and pop-out motion regions. Experimental results validate its effectiveness and efficiency.

Design and Performance Evaluation of Temporal Motion and Color Energy Features for Video Rating System

With the increasing demand for access of mobile Internet via smartphones, the need of adult images and videos filtering is escalating. In this paper, a simple and fast objectionable video classification scheme is proposed using temporal motion and color energy features (TMCEF) and the performances of accuracy and processing time are evaluated. The video segments extracted from a video clip have similar color and motion characteristics. System framework for TMCEF consists of key frame extraction, motion energy calculation, skin color energy calculation, and feature extraction based on statistical distribution metric using mean, standard deviation, and frequency analysis using DCT(discrete cosine transform). The motion energy is extracted based on foreground motion detection scheme and the color energy based on skin color detection method. In order to verify the performance of these video-based temporal features, support vector machine (SVM) classifier is used. In experiments, 64F-TMCEF (36) (the case of extracting 64 key frames and using 36 temporal motion and color energy features) yields the best performance on accuracy. However, due to excessive processing time of 64F-TMCEF, 36F-TMCEF (36) (the case of extracting 36 key frames and using 36 temporal motion and color energy features) is found to be more practical. For smartphone applications, 16FTCEF (18) (the case of extracting 16 key frames and using 18 temporal color energy features) is adequate to be used.

Motion Adaptive Compensation Approach for Deinterlacing of Video Sequences

de-interlacing algorithm using adaptive 4-field motion compensation approach is presented. It consists of block- based directional edge interpolation, same- parity 4-field motion detection, 4-field motion estimation and compensation. The intra field methods are reconstructed the frame from the current field information .but this method introduce the edge flicker problems and jitter effect. The inter field methods are depends on the previous and future fields for reconstruction of the current frame. This method introduces feathering effect. The edges are sharper when the directional edge interpolation is adopted and jitter effect and the feathering effect eliminated. The motion adaptive deinterlacing scheme is taking the advantages of both intra and inters field methods. First it finds the motion by using motion detection scheme if the field contain motion apply intra field interpolation method if the field contain stationary objects apply the inter field interpolation method. The 3-field motion detection can not detect the fast motion areas from field to field. The same parity 4-field motion adaptive deinterlacing and the 4-field motion compensation detect the static areas and fast motion by four reference fields. The Compensation recovers the interlaced videos to the progressive ones but the feathering effect is not recovered in this method. The adaptive 4-field motion compensation method removes the feathering effect along with detecting fast motion areas by using four reference fields. Experimental results show that the peak signal-to-noise ratio of our adaptive 4-field motion compensation deinterlacing algorithm is 4 to 6 dB higher than that of 3-field motion adaptive deinterlacing and 2 to 3 dB higher than 4-field motion compensation deinterlacing and attain the best quality of video.

The many facets of adaptation in fly visual motion processing

Neuronal adaptation has been studied extensively in visual motion-sensitive neurons of the fly Calliphora vicina, a model system in which the computational principles of visual motion processing are amenable on a single-cell level. Evidenced by several recent papers, the original idea had to be dismissed that motion adaptation adjusts velocity coding to the current stimulus range by a simple parameter change in the motion detection scheme. In contrast, linear encoding of velocity modulations and total information rates might even go down in the course of adaptation. Thus it seems that rather than improving absolute velocity encoding motion adaptation might bring forward an efficient extraction of those features in the visual input signal that are most relevant for visually guided course control and obstacle avoidance.

Motion detection using Fourier image reconstruction

In video surveillance, detection of moving objects from an image sequence is very important for object tracking, activity recognition and behavior understanding. The conventional background subtraction suffers from slow updating of environmental changes, and temporal difference cannot accurately extract the moving object boundaries. In this paper, a Fourier reconstruction scheme for motion detection is proposed. A series of consecutive 2D spatial images along the time axis are first reorganized as a series of 2D spatial–temporal images along a spatial axis. In each of the 2D spatial–temporal images, a static background region forms a vertical line pattern, and a moving object creates an irregular, non-vertical structure in the image. Fourier transforms are applied to remove the vertical line pattern (i.e. the background) and retain only the foreground in the reconstructed image. The proposed method is a global approach that identifies the moving objects based on structural variations in the whole patterned image. It is therefore very robust to accommodate noise and local gray-level variations. It can well extract the shapes of foreground objects at various moving speeds, and is very responsive to dynamic environments. High computational cost is the major drawback of the proposed method. However, it can still achieve 11 frames per second for small images of size 150 × 200.

Fuzzy logic recursive motion detection and denoising of video sequences

We propose a fuzzy logic recursive scheme for motion detection and spatiotemporal filtering that can deal with the Gaussian noise and unsteady illumination conditions in both the temporal and spatial directions. Our focus is on applications concerning tracking and denoising of image sequences. We process an input noisy sequence with fuzzy logic motion detection to determine the degree of motion confidence. The proposed motion detector combines the membership of the temporal intensity changes, appropriately using fuzzy rules, where the membership degree of motion for each pixel in a 2-D sliding window is determined by a proposed membership function. Both the fuzzy membership function and the fuzzy rules are defined in such a way that the performance of the motion detector is optimized in terms of its robustness to noise and unsteady lighting conditions. We simultaneously perform tracking and recursive adaptive temporal filtering, where the amount of filtering is inversely proportional to the confidence in the existence of motion. Finally, temporally filtered frames are further processed by a proposed spatial filter to obtain a denoised image sequence. Our main contribution is a robust novel fuzzy recursive scheme for motion detection and temporal filtering. We evaluate the proposed motion detection algorithm using two criteria: (1) robustness to noise and to changing illumination conditions and (2) motion blur in temporal recursive denoising. Additionally, we make comparisons in terms of noise reduction with other state of the art video denoising techniques.