Optical Flow Approach Research Articles

Optical Flow Based Object Movement Tracking

Object detection and tracking is one of the key tasks performed in video surveillance. The objects present in the area under surveillance is studied and analyzed with reference to the context. This plays a pivotal role in detecting and predicting anomalies based on the behavioral traits of objects observed under the surveillance region. Optical flow is one of the computer vision based approaches that is used for tracking the precise movement of objects. Several optical flow algorithms have been used to track and study the movement of objects. This work is motivated towards carrying out a thorough study different optical flow techniques and comparing the features of different optical flow approaches and implementing them for real time detection and tracking of objects in real time environment.

Satellite-Based Assessment of Various Cloud Microphysics Schemes in Simulating Typhoon Hydrometeors

The accurate simulation of typhoon hydrometeors remains a challenge. This study attempts to evaluate the performances of five microphysics schemes (MPSs) in the Weather Research and Forecasting (WRF) model in simulating the supertyphoon Neoguri in July 2014. The observed microwave brightness temperature, as well as retrieved data from the microwave radiometer imager (MWRI) onboard Chinese FY-3B satellite, are used to test hydrometeor simulations. In particular, two MWRI radiance indices, including the emission index (EI) and scattering index (SI), are used to assess the performance of five MPSs in simulating liquid and frozen hydrometeors, respectively. Overall, the WRF model can well reproduce the overall pattern of typhoon-produced precipitation, albeit with slightly overestimated precipitation in the inner rainband and underestimated precipitation in the stratiform rainband. Moreover, ice water paths (IWPs) from all five MPS simulations are higher than those estimated from MWRI retrieval in most areas, and the spatial pattern and values of IWP for the National Severe Storms Laboratory double-moment MPS (NSSL) are much closer to those for MWRI. The NSSL scheme reproduces a more realistic joint histogram distribution of SI and EI than other MPSs do, relative to the observation. Besides, the nonlinear Lucas–Kanade optical flow approach has been used to reflect the horizontal distribution of hydrometeors in the typhoon. The results show that the simulated EI and SI from the five MPSs show a systematic southwest bias of approximately about 10∼20 km and significant intensity bias in the convection area. Further model sensitivity tests confirm that the NSSL scheme generates more realistic graupel and supercooled water close to the observations among all MPSs. The findings suggest that satellite measurements would be helpful to assess MPSs in numeric weather models, especially for hydrometeor distributions in the whole typhoon system.

Semi-dense and robust image registration by shift adapted weighted aggregation and variational completion

This work presents a novel approach for both stereo and optical flow that deals with large displacements, depth/motion discontinuities and occlusions. The proposed method comprises two main steps. First, a novel local stereo matching algorithm is presented, whose main novelty relies in the block-matching aggregation step. We adopt an adaptive support weights approach in which the weight distribution favors pixels that share the same displacement with the reference one. State-of-the-art methods make the weight function depend only on image features. On the contrary, the proposed weight function depends additionally on the tested shift, by giving more importance to those pixels in the block-matching with smaller cost, as these are supposed to have the tested displacement. Moreover, the method is embedded into a pyramidal procedure to locally limit the search range, which helps to reduce ambiguities in the matching process and saves computational time. Second, the non-dense local estimation is filtered and interpolated by means of a new variational formulation making use of intermediate scale estimates of the local procedure. This permits to keep the fine details estimated at full resolution while being robust to noise and untextured areas using estimates at coarser scales. The introduced variational formulation as well as the block-matching algorithm are robust to illumination changes. We test our algorithm for both stereo and optical flow public datasets showing competitive results.

A laser arrays scan thermography (LAsST) for the rapid inspection of CFRP composite with subsurface defects

Laser arrays scan thermography (LAsST) as a nondestructive testing and evaluation (NDT&E) tool is presented to detect the subsurface delamination or abnormally of carbon fiber reinforced polymer (CFRP) composite. A series of woven CFRP specimens with artificial bottom flat holes (BFHs) are prepared for NDT&E by LAsST. The thermal response signal was reconstructed and in turn appeared the analogous thermal responsible behavior similar to pulsed thermography (PT). The thermal response signal was processed by using of Fast Fourier transform (FFT) and principal component analysis (PCA), and FFT amplitude and phase images, PCA image were formed. The signal noise ratios (SNRs) of defects were calculated and employed to evaluate the defect detectability of different post processing algorithms. A likeness optical flow (LOF) approach was proposed to reduce the interruption of lateral thermal diffusion and improve the defect detectability. The experimental results indicate that FFT phase based image shows a relatively high SNR, and the LOF method effectively enlarged the SNRs of defects.

On illumination-invariant variational optical flow for weakly textured scenes

This paper deals with variational optical flow approaches for motion estimation under varying illumination conditions in weakly textured scenes. It proposes a systematic and complete study on descriptor-based data-terms that lead to a robust variational optical flow model. Unlike the literature which most often only experimentally shows that a descriptor is illumination invariant, this contribution gives a theoretical proof of this invariance. First, a local illumination change model is proposed and used to mathematically check whether a descriptor is invariant or not with respect to illumination variations between images. Then, this contribution proposes two general mathematical formulations which can be used to design a wide variety of new illumination-invariant descriptors. To illustrate the interest of the proposed approach, two novel illumination-invariant descriptors are constructed using the proposed general formulations. Moreover, the performance of the descriptors was evaluated on numerous datasets with known ground truth optical flow, while the robustness of the variational optical flow approach was highlighted using complex medical image sequences without ground truth. These experimental results have shown that data-terms based on the proposed descriptors led to accurate and constant optical flow under varying illumination conditions.

Efficient Multispectral Facial Capture With Monochrome Cameras

We propose a variant to polarized gradient illumination facial scanning which uses monochrome instead of color cameras to achieve more efficient and higher-resolution results. In typical polarized gradient facial scanning, sub-millimeter geometric detail is acquired by photographing the subject in eight or more polarized spherical gradient lighting conditions made with white LEDs, and RGB cameras are used to acquire color texture maps of the subject's appearance. In our approach, we replace the color cameras and white LEDs with monochrome cameras and multispectral, colored LEDs, leveraging that color images can be formed from successive monochrome images recorded under different illumination colors. While a naive extension of the scanning process to this setup would require multiplying the number of images by number of color channels, we show that the surface detail maps can be estimated directly from monochrome imagery, so that only an additional n photographs are required, where n is the number of added spectral channels. We also introduce a new multispectral optical flow approach to align images across spectral channels in the presence of slight subject motion. Lastly, for the case where a capture system's white light sources are polarized and its multispectral colored LEDs are not, we introduce the technique of multispectral polarization promotion, where we estimate the cross- and parallel-polarized monochrome images for each spectral channel from their corresponding images under a full sphere of even, unpolarized illumination. We demonstrate that this technique allows us to efficiently acquire a full color (or even multispectral) facial scan using monochrome cameras, unpolarized multispectral colored LEDs, and polarized white LEDs.

Optical Flow Approach Followed by SVM Classification Model to Recognize Abnormal Behavior of a Crowd

A many of researches have been carried out in the field of the crowd behavior recognition system. Recognizing crowd behavior in videos is most challenging and occlusions because of irregular human movement. This paper gives an overview of optical flow model along with the SVM (Support Vector Machine) classification model. This proposed approach evaluates sudden changes in motion of an event and classifies that event to a category: Normal and Abnormal. Geometric means of location, direction, and displacement of the feature points of each frame are estimated. Harris corner Detector is used in each frame for tracking a set of feature points. Proposed approach is very effective in real time scenario like public places where security is most important. After analyzing result ROC curve (receiver operating characteristics) is plotted which gives classification accuracy. We also presented frame level comparison with Ground truth and social force model (SFM) techniques. Our proposed approach is giving a promising result compare to all state of art methods.

Towards breaking the spatial resolution barriers: An optical flow and super-resolution approach for sea ice motion estimation

Estimation of sea ice motion at fine scales is important for a number of regional and local level applications, including modeling of sea ice distribution, ocean-atmosphere and climate dynamics, as well as safe navigation and sea operations. In this study, we propose an optical flow and super-resolution approach to accurately estimate motion from remote sensing images at a higher spatial resolution than the original data. First, an external example learning-based super-resolution method is applied on the original images to generate higher resolution versions. Then, an optical flow approach is applied on the higher resolution images, identifying sparse correspondences and interpolating them to extract a dense motion vector field with continuous values and subpixel accuracies. Our proposed approach is successfully evaluated on passive microwave, optical, and Synthetic Aperture Radar data, proving appropriate for multi-sensor applications and different spatial resolutions. The approach estimates motion with similar or higher accuracy than the original data, while increasing the spatial resolution of up to eight times. In addition, the adopted optical flow component outperforms a state-of-the-art pattern matching method. Overall, the proposed approach results in accurate motion vectors with unprecedented spatial resolutions of up to 1.5 km for passive microwave data covering the entire Arctic and 20 m for radar data, and proves promising for numerous scientific and operational applications.

The Assessment of left ventricular Function in MRI using the detection of myocardial borders and optical flow approaches: A Review

The evaluation of left ventricular wall motion in Magnetic Resonance Imaging (MRI) clinical practice is based on a visual assessment of cine-MRI sequences. In fact, clinical interpreters (radiologists) proceed with a global visual evaluation of multiple cine-MRI sequences acquired in the three standard views. In addition, some functional parameters are quantified following a manual or a semi-automatic contouring of the myocardial borders. Although these parameters give information about the functional state of the left ventricle, they are not able to provide the location and the extent of wall motion abnormalities, which are associated with many cardiovascular diseases. In the past years, several approaches were developed to overcome the limitations of the classical evaluation techniques of left ventricular function. The aim of this article is to present an overview of the different methods and to summarize the relevant techniques based on myocardial contour detection and optical flow for regional assessment of left ventricular abnormalities.

An Efficient Scan-to-Map Matching Approach Based on Multi-channel Lidar

Accurately localizing the vehicle against a pre-built high precision map is an essential step for the Autonomous Land Vehicle (ALV). This paper proposes an efficient scan-to-map matching approach based on multi-channel lidar. We firstly advocate the usage of both the lidar reflectance map and the height map, as these two maps contain complementary information. Then, borrowing ideas from the Lucas-Kanade optical flow approach, we formulate the scan-to-map matching problem in a similar form, and propose an efficient gradient descent approach to solve it. Finally, the proposed approach is integrated into a filtering framework for real-time online localization. Experiments on real-world dataset have demonstrated the validity of our approach.

High-Resolution Sea Ice Motion Estimation With Optical Flow Using Satellite Spectroradiometer Data

The purpose of this paper is twofold: 1) to propose an approach based on optical flow for the estimation of sea ice motion as an accurate, dense, and computationally efficient alternative to state-of-the-art pattern matching approaches and 2) to investigate the potential of moderate resolution imaging spectroradiometer (MODIS) optical satellite data for combined daily and high-resolution motion estimation. A series of MODIS image pairs for a selected region in Arctic Ocean is employed, and sparse pairwise correspondences between nonrigid patches are calculated. An edge-preserving sparse-to-dense interpolation is applied followed by variational energy minimization to compute the final optical flow. A state-of-the-art multiresolution pattern matching method based on phase correlation and normalized cross correlation is also implemented and evaluated for comparison. Thorough experimentation with different settings of input data preprocessing as well as varying scales of motion is performed. The derived motion vectors are compared with coarser resolution operational sea ice motion vector products from combined buoy and microwave satellite data. The proposed optical flow approach clearly outperforms the pattern matching method in most cases, both in terms of accuracy and motion vector consistency. The estimated motion vectors from the MODIS images highly correlate with the operational vector products. In addition, MODIS provides a vector field with spatial resolution two orders of magnitude higher than the operational products that are able to detect significantly smaller sea ice drifts.

Fault detection and classification in automated assembly machines using machine vision

Automated assembly machines operate continuously to achieve high production rates. Continuous operation increases the potential for faults such as jams, missing parts, and electromechanical failures of subsystems. The goal of this research project was to develop and validate a machine vision inspection (MVI) system to detect and classify multiple faults using a single camera as a sensor. An industrial automated O-ring assembly machine that places O-rings on to continuously moving plastic carriers at a rate of over 100 assemblies per minute was modified to serve as the test apparatus. An industrial camera with LED panel lights for illumination was used to acquire videos of the machine’s operation. A programmable logic controller (PLC) with a human-machine interface (HMI) allowed for the generation of faults in a controlled fashion. Three MVI methods, based on computer vision techniques available in the literature, were developed for this application. The methods used features extracted from the videos to classify the machine’s condition. The first method was based on Gaussian mixture models (GMMs); the second method used an optical flow approach; and the third method was based on running average and morphological image processing operations. In order to provide a single metric to quantify relative performance, a machine vision performance index (MVPI) was developed with five measures of performance: accuracy, processing time, speed of response, robustness against noise, and ease of tuning. The MVPI for the three MVI methods is reported along with the significance of the results.

Video interpolation using optical flow and Laplacian smoothness

Non-rigid video interpolation is a common computer vision task. In this paper we present an optical flow approach which adopts a Laplacian Cotangent Mesh constraint to enhance the local smoothness. Similar to Li et al., our approach adopts a mesh to the image with a resolution up to one vertex per pixel and uses angle constraints to ensure sensible local deformations between image pairs. The Laplacian Mesh constraints are expressed wholly inside the optical flow optimization, and can be applied in a straightforward manner to a wide range of image tracking and registration problems. We evaluate our approach by testing on several benchmark datasets, including the Middlebury and Garg et al. datasets. In addition, we show application of our method for constructing 3D Morphable Facial Models from dynamic 3D data.

Estimation of sea surface currents based on ocean colour remote-sensing image analysis

ABSTRACTAccurate quantification of the sea surface current speeds with high spatial resolution and in near real-time is beneficial for many applications of physical oceanography, and can be implemented by processing ocean colour remote-sensing imagery data. The robust optical flow (ROF) approach applied to sub-image processing is described in detail in this article, and compared with a conventional maximum cross-correlation (MCC) block matching algorithm with sub-pixel operation. ROF results obtained from Geostationary Ocean Colour Imager (GOCI) imagery are shown to be in good agreement with sea surface currents derived from Ocean Surface Current Analyses-Real time (OSCAR) data, providing a validation of the ROF method.

DeepMatching: Hierarchical Deformable Dense Matching

We introduce a novel matching algorithm, called DeepMatching, to compute dense correspondences between images. DeepMatching relies on a hierarchical, multi-layer, correlational architecture designed for matching images and was inspired by deep convolutional approaches. The proposed matching algorithm can handle non-rigid deformations and repetitive textures and efficiently determines dense correspondences in the presence of significant changes between images. We evaluate the performance of DeepMatching, in comparison with state-of-the-art matching algorithms, on the Mikolajczyk (Mikolajczyk et al. A comparison of affine region detectors, 2005), the MPI-Sintel (Butler et al. A naturalistic open source movie for optical flow evaluation, 2012) and the Kitti (Geiger et al. Vision meets robotics: The KITTI dataset, 2013) datasets. DeepMatching outperforms the state-of-the-art algorithms and shows excellent results in particular for repetitive textures. We also apply DeepMatching to the computation of optical flow, called DeepFlow, by integrating it in the large displacement optical flow (LDOF) approach of Brox and Malik (Large displacement optical flow: descriptor matching in variational motion estimation, 2011). Additional robustness to large displacements and complex motion is obtained thanks to our matching approach. DeepFlow obtains competitive performance on public benchmarks for optical flow estimation.

2D Video Stabilization for Industrial High-Speed Cameras

Abstract The current research concerns the problem of video stabilization “in a point”, which aims to stabilize all video frames according to one chosen reference frame to produce a new video, as by a static camera. Similar task importance relates providing static background in the video sequence that can be usable for correct measurements in the frames when studying dynamic objects in the video. For this aim we propose an efficient combined approach, called “3×3OF9×9”. It fuses our the previous development for fast and rigid 2D video stabilization [2] with the well-known Optical Flow approach, applied by parts via Otsu segmentation, for eliminating the influence of moving objects in the video. The obtained results are compared with those, produced by the commercial software Warp Stabilizer of Adobe-After-Effects CS6.

Detection of recurrent nerve paralysis: Development of a Computer Aided Diagnosis system

Recurrent nerve paralysis (RP) is one of the most frequent complications of thyroid surgery. It reduces vocal fold mobility. Nasal endoscopy, a mini-invasive procedure, is the reference procedure to detect RP, and is based on the examination of vocal fold mobility. A new approach based on laryngeal ultrasound acquisition and using a dedicated data analysis was designed to help with the automated detection of RP.One hundred and fifty subjects were enrolled for this feasibility study: 50 controls, 50 patients with RP and 50 patients without RP, according to nasal endoscopy. The ultrasound protocol was based on a ten-second B-mode acquisition in an axial plane during normal breathing. Image processing included three steps: 1) the detection of two consecutive closing and opening images corresponding to extreme positions of vocal folds in the sequence of B-mode images; 2) the positioning of three landmarks and the robust tracking of these points using multi-pyramidal refined optical flow approaches; 3) the estimation of quantitative parameters indicating left and right fractions of mobility and motion symmetry.Results provided by the first two image processing steps were compared to those obtained by an expert. Motion symmetry and fraction of mobility indices were systematically computed using the automated procedures. Associated sensitivity and specificity values for detecting RP were then calculated. To optimize the performances of the system, a mixed CAD system, which integrates the automatic steps of image processing and a possible correction of its results by a trained operator, was developed.Laryngeal ultrasound combined with appropriate image processing helped in the diagnosis of recurrent nerve paralysis and could be proposed as a first-line method.

Anisotropic motion estimation on edge preserving Riesz wavelets for robust video mosaicing

Image mosaicing is a technique widely used for extending the field of view of industrial, medical, outdoor or indoor scenes. However, image registration can be very challenging, e.g. due to large texture variability, illumination changes, image blur and camera perspective changes. In this paper, a total variational optical flow approach is investigated to estimate dense point correspondences between image pairs. An edge preserving Riesz wavelet scale-space combined with a novel TV-regularizer is proposed for preserving motion discontinuities along the edges of weak textures and for handling strong in-plane rotations present in image sequences. An anisotropic weighted median filtering is implemented for minimizing outliers. Quantitative evaluation of the method on the Middlebury image database and simulated sequences with known ground truth demonstrates high accuracy of the proposed method in comparison with other state-of-the-art methods, including a robust graph-cut method and a patch matching approach. Qualitative results on video-sequences of difficult real scenes demonstrate the robustness of the proposed method.

An Implementation of Combined Local-Global Optical Flow

Optical Flow (OF) approaches for motion estimation calculate vector fields for the apparent velocities of objects in image sequences. In 1981 Horn and Schunck (HS) introduced two basic assumptions: 'brightness value constancy' and 'smooth variation' to estimate a smooth OF field over the entire image -global approach-. In parallel, Lucas and Kanade (LK) assumed constant motion patterns for image patches, estimating piecewise-homogeneous OF fields -local approach-. Several variations of these approaches exist today. Here we present the combined local-global (CLG) approach by Bruhn et al. which encompasses properties of HS-OF and LK-OF, aiming to improve the OF accuracy for small-scale variations, while delivering the HS-OF dense and smooth fields. A multiscale implementation is provided for 2D images, together with two numerical solvers: Successive Over-Relaxation and the faster Pointwise-Coupled Gauss-Seidel by Bruhn et al.. The algorithm works on gray-scale (single channel) images, with color images being converted prior to the OF computation.

On the optical flow model selection through metaheuristics

Optical flow methods are accurate algorithms for estimating the displacement and velocity fields of objects in a wide variety of applications, being their performance dependent on the configuration of a set of parameters. Since there is a lack of research that aims to automatically tune such parameters, in this work, we have proposed an optimization-based framework for such task based on social-spider optimization, harmony search, particle swarm optimization, and Nelder-Mead algorithm. The proposed framework employed the well-known large displacement optical flow (LDOF) approach as a basis algorithm over the Middlebury and Sintel public datasets, with promising results considering the baseline proposed by the authors of LDOF.