Robust Optical Flow Estimation Research Articles

Robust optical flow estimation method based on structure-texture aware retinex model and its application on face anti-spoofing

Robust optical flow estimation method based on structure-texture aware retinex model and its application on face anti-spoofing

Every Frame Counts: Joint Learning of Video Segmentation and Optical Flow

A major challenge for video semantic segmentation is the lack of labeled data. In most benchmark datasets, only one frame of a video clip is annotated, which makes most supervised methods fail to utilize information from the rest of the frames. To exploit the spatio-temporal information in videos, many previous works use pre-computed optical flows, which encode the temporal consistency to improve the video segmentation. However, the video segmentation and optical flow estimation are still considered as two separate tasks. In this paper, we propose a novel framework for joint video semantic segmentation and optical flow estimation. Semantic segmentation brings semantic information to handle occlusion for more robust optical flow estimation, while the non-occluded optical flow provides accurate pixel-level temporal correspondences to guarantee the temporal consistency of the segmentation. Moreover, our framework is able to utilize both labeled and unlabeled frames in the video through joint training, while no additional calculation is required in inference. Extensive experiments show that the proposed model makes the video semantic segmentation and optical flow estimation benefit from each other and outperforms existing methods under the same settings in both tasks.

Graph Laplacian Regularization for Robust Optical Flow Estimation.

This paper proposes graph Laplacian regularization for robust estimation of optical flow. First, we analyze the spectral properties of dense graph Laplacians and show that dense graphs achieve a better trade-off between preserving flow discontinuities and filtering noise, compared with the usual Laplacian. Using this analysis, we then propose a robust optical flow estimation method based on Gaussian graph Laplacians. We revisit the framework of iteratively reweighted least-squares from the perspective of graph edge reweighting, and employ the Welsch loss function to preserve flow discontinuities and handle occlusions. Our experiments using the Middlebury and MPI-Sintel optical flow datasets demonstrate the robustness and the efficiency of our proposed approach.

Optical Flow Estimation with Occlusion Detection

The dense optical flow estimation under occlusion is a challenging task. Occlusion may result in ambiguity in optical flow estimation, while accurate occlusion detection can reduce the error. In this paper, we propose a robust optical flow estimation algorithm with reliable occlusion detection. Firstly, the occlusion areas in successive video frames are detected by integrating various information from multiple sources including feature matching, motion edges, warped images and occlusion consistency. Then optimization function with occlusion coefficient and selective region smoothing are used to obtain the optical flow estimation of the non-occlusion areas and occlusion areas respectively. Experimental results show that the algorithm proposed in this paper is an effective algorithm for dense optical flow estimation.

Robust optical flow estimation based on wavelet

Concentrating on the issue that the systematic error caused by variation of illumination conditions and sensor noise leads to poor robustness and low accuracy of optical flow calculation, based on the wavelet multi-resolution theory, a robust optical flow estimation method is developed in this paper. With the multi-resolution characteristics of wavelet, the systematic error is incorporated into the calculation of optical flow to improve the robustness and estimation accuracy. In what follows, the total least squares method can be exploited to solve the obtained overdetermined wavelet optical flow equations such that the optical flow vector can be achieved. As compared to the traditional Lucas–Kanade approach, Horn–Schunck method, and the optical flow estimation algorithm in omnidirectional images using wavelet approach, simulation results show that the proposed algorithm can significantly improve the accuracy of optical flow estimation and the robustness of the optical flow field.

On robust optical flow estimation on image sequences with differently exposed frames using primal–dual optimization

Optical flow methods are used to estimate pixelwise motion information based on consecutive frames in image sequences. The image sequences traditionally contain frames that are similarly exposed. However, many real-world scenes contain high dynamic range content that cannot be captured well with a single exposure setting. Such scenes result in certain image regions being over- or underexposed, which can negatively impact the quality of motion estimates in those regions. Motivated by this, we propose to capture high dynamic range scenes using different exposure settings every other frame. A framework for OF estimation on such image sequences is presented, that can straightforwardly integrate techniques from the state-of-the-art in conventional OF methods. Different aspects of robustness of OF methods are discussed, including estimation of large displacements and robustness to natural illumination changes that occur between the frames, and we demonstrate experimentally how to handle such challenging flow estimation scenarios. The flow estimation is formulated as an optimization problem whose solution is obtained using an efficient primal–dual method.

Motion-based object tracking method for safety at level crossing

This paper presents a tracking method to detect and track independently moving targets, attempting to traverse the railway, in monocular camera sequences. This method is capable of tracking the maximum number of pixels belonging to an object. The method starts by detecting and separating moving objects due to background subtraction and an energy vector-based clustering. Next, the method performs the step of tracking locally. Tracking starts by generating initial optical flow of all object pixels by propagating the optical flow of Harris corner points (calculated by Lucas–Kanade technique) using normal distribution. An iterative procedure, including Kalman filtering with adaptive parameters, color intensity difference-based optimization, and validation constraints, is then implemented to reach precise and robust optical flow estimation for the majority of the pixels of the tracked objects. Different experimental results are presented, evaluated, and discussed to show the effectiveness of the method of tracking objects that may move in complex and overlapping trajectories.

A fractional order variational model for the robust estimation of optical flow from image sequences

In this paper, a fractional order variational model for estimating the optical flow is presented. In particular, the proposed model generalizes the integer order variational optical flow models. The fractional order derivative describes discontinuous information about texture and edges, and therefore a more suitable in estimating the optical flow. The proposed variational functional is a combination of a global model of Horn and Schunck and the classical model of Nagel and Enkelmann. This formulation yields a dense flow and preserves discontinuities in the flow field and also provides a significant robustness against outliers. The Grünwald–Letnikov derivative is used for solving complex fractional order partial differential equations. The corresponding linear system of equations is solved by an efficient numerical scheme. A detailed stability and convergence analysis is given in order to show the mathematical applicability of the numerical algorithm. Experimental results on various datasets verify the validity of the proposed model.

High accuracy optical flow estimation based on PDE decomposition

Many optical flow estimation techniques are based on the differential optical flow equation. These algorithms involve solving over-determined systems of optical flow equations. Least-squares (LS) estimation is usually used to solve these systems even though the underlying noise does not conform to the model implied by LS estimation. To tackle this problem, work has been done combining the variational partial differential equation (PDE) methods with motion estimation. However, PDE methods demonstrated powerful tools to decompose an image into its structures, textures, and noise components. The noise is eliminated systematically in the proposed scheme, and the optical flow is computed separately on both components of the decomposition. We experimentally show that very precise and robust estimation of optical flow can be achieved with a total variational approach in real time. The implementation is described in a detailed way, which enables reimplementation of this high-end method. The proposed technique has been tested upon different dataset of both synthetic and real image sequences, and compared to both well-known and state-of-the-art differential optical flow methods.

Robust optical flow estimation based on brightness correction fields

Optical flow estimation is still an important task in computer vision with many interesting applications. However, the results obtained by most of the optical flow techniques are affected by motion discontinuities or illumination changes. In this paper, we introduce a brightness correction field combined with a gradient constancy constraint to reduce the sensibility to brightness changes between images to be estimated. The advantage of this brightness correction field is its simplicity in terms of computational complexity and implementation. By analyzing the deficiencies of the traditional total variation regularization term in weakly textured areas, we also adopt a structure-adaptive regularization based on the robust Huber norm to preserve motion discontinuities. Finally, the proposed energy functional is minimized by solving its corresponding Euler-Lagrange equation in a more effective multi-resolution scheme, which integrates the twice downsampling strategy with a support-weight median filter. Numerous experiments show that our method is more effective and produces more accurate results for optical flow estimation.

無数の粒子状パターンに覆われた実環境シーンからの移動対象のフロー推定法および計数法

In real environments, the detection and motion of an interesting moving object against a complex background in computer vision are very important. However, many undesirable factors that can prevent stable detection arise in a scene. The factors are, for example, occlusion, lighting change and jittery background. In addition to these, an infinite number of randomly moving particle-like patterns can make moving objects more difficult to detect. These particles have ambiguous edges and no definite shape. In a more complicated scene, we assume that such particles cannot be smoothed out by a simple preprocessing such as a low-pass filtering. Thus, they can distribute optical flow of moving objects in time and space. To deal with these issues for matching between frames, it is first assumed that particle-like patterns and moving objects have properties of fluidity and rigidity, respectively. The image brightness change and motion smoothness between frames can constrain the estimated optical flow of a moving object. However, a local large brightness change caused by the above factors violates such constraints. Thus, on the basis of a statistical approach, a robust optical flow estimation method has been developed by adding a locally parallel flow contraint. Estimated flow is used to count the number of multiple moving objects with a clustering method, which is applied starting with a large number of centorids. Centorids are iteratively merged within a distance until converged. Experimental results show that our method outperforms previous methods, thus validating our proposed method.

Dynamic consistent correlation-variational approach for robust optical flow estimation

We present in this paper a novel combined scheme dedicated to the measurement of velocity in fluid experimental flows through image sequences. The proposed technique satisfies the Navier–Stokes equations and combines the robustness of correlation techniques with the high density of global variational methods. It can be considered either as a reenforcement of fluid dedicated optical-flow methods towards robustness, or as an enhancement of correlation approaches towards dense information. This results in a physics-based technique that is robust under noise and outliers, while providing a dense motion field. The method was applied on synthetic images and on real experiments in turbulent flows carried out to allow a thorough comparison with a state of the art variational and correlation methods.

Robust estimation of optical flow based on linear brightness model

A robust optical flow algorithm for larger motion velocity and larger velocity change under large varying illumination was presented. A general linear brightness model was embedded in the gradient constraint equation, which improved algorithmic stability under large varying illumination. The anisotropic diffusion equation was introduced in the spatial oriented smoothness constraint, so that it can improve the computation of flow velocity in the motion discontinuities, according to these improvements, the differential optical flow algorithm based on multi-scale-space focusing was used. An invariant energy function under the linear variation of scales was obtained by normalizing the parameters. Motion compensation is introduced to iterative process, and then the error of brightness was decreased. The experimental results show that the algorithm proposed in this paper provides with good computation accuracy for lager motion velocity and larger velocity change, and the flow field of 100% density can be created.

A robust optical flow computation

This paper presents a new method for robust and accurate optical flow estimation. The significance of this work is twofold. Firstly, the idea of bi-directional scheme is adopted to reduce the model error of optical flow equation, which allows the second order Taylor’s expansion of optical flow equation for accurate solution without much extra computational burden; Secondly, this paper establishs a new optical flow equation based on LSCM (Local Structure Constancy Model) instead of BCM (Brightness Constancy Model), namely the optical flow equation does not act on scalar but on tensor-valued (matrix-valued) field, due to the two reason: (1) structure tensor-value contains local spatial structure information, which provides us more useable cues for computation than scalar; (2) local image structure is less sensitive to illumination variation than intensity, which weakens the disturbance of non-uniform illumination in real sequences. Qualitative and quantitative results for synthetic and real-world scenes show that the new method can produce an accurate and robust results.

Error analysis of robust optical flow estimation by least median of squares methods for the varying illumination model

The apparent pixel motion in an image sequence, called optical flow, is a useful primitive for automatic scene analysis and various other applications of computer vision. In general, however, the optical flow estimation suffers from two significant problems: the problem of illumination that varies with time and the problem of motion discontinuities induced by objects moving with respect to either other objects or with respect to the background. Various integrated approaches for solving these two problems simultaneously have been proposed. Of these, those that are based on the LMedS (Least Median of Squares) appear to be the most robust. The goal of this paper is to carry out an error analysis of two different LMedS-based approaches, one based on the standard LMedS regression and the other using a modification thereof as proposed by us recently. While it is to be expected that the estimation accuracy of any approach would decrease with increasing levels of noise, for LMedS-like methods, it is not always clear as to how much of that decrease in performance can be attributed to the fact that only a small number of randomly selected samples is used for forming temporary solutions. To answer this question, our study here includes a baseline implementation in which all of the image data is used for forming motion estimates. We then compare the estimation errors of the two LMedS-based methods with the baseline implementation. Our error analysis demonstrates that, for the case of Gaussian noise, our modified LMedS approach yields better estimates at moderate levels of noise, but is outperformed by the standard LMedS method as the level of noise increases. For the case of salt-and-pepper noise, the modified LMedS method consistently performs better than the standard LMedS method.

Optical Flow Analysis of the Aurora Borealis

Optical observations of the aurora have traditionally focused on the structure, intensity, and wavelength of the emissions. But the apparent motion of auroral forms offers another important diagnostic tool for investigating this poorly understood phenomenon. Prior analyses of auroral motion have focused on tracing individual features. In this letter, we investigate the feasibility of deriving the entire two-dimensional velocity field automatically using robust optical flow estimation. The analysis is applied to two narrow-field video sequences. Both examples are rich in small-scale structure and motion, but appear very different to the eye. The robust optical flow estimator performed well for regions of dense turbulent motion, while sheared flow and flow which is perpendicular to image intensity gradients, was poorly resolved. The relative magnitude of the outliers provides a quantitative measure of the validity of the underlying flow model and, hence, a means of automatically differentiating among auroral forms with differing physical origins. The technique can be used to deduce ionospheric electric fields and neutral winds, and the flow fields yield important physical information about the generation mechanism.

A ROBUST OPTICAL FLOW ESTIMATION UNDER VARYING ILLUMINATION

This paper proposes a new method for robust and accurate optical flow estimation. The robustness of the new method is the ability of weakening the disturbance of varying illumination and preserving motion boundaries. The significance of this work is twofold: In the first part the generalized dynamic image model (GDIM) is adopted based on gradient field to encourage robustness to non-uniform brightness variations; otherwise, a Bayesian selecting scheme is adopted to encourage the ability of preserving motion boundaries and reduce the error, bias generated by smoothness constraint. We validated our results on a set of real and synthetic sequences. The qualitative and quantitative results indicate that the new method produces more accurate and robust results, which better sustain our conclusions.

Robust motion estimation under varying illumination

The optical-flow approach has emerged as a major technique for estimating scene and object motion in image sequences. However, the results obtained by most optical flow techniques are strongly affected by motion discontinuities and by large illumination changes. While there do exist many separate techniques for robust estimation of optical flow in the presence of motion discontinuities and for dealing with the problems caused by illumination variations, only a few integrated approaches have been proposed. However, most of these previously proposed integrated approaches use simple models of illumination variation; a common assumption being that illumination changes by either just a multiplicative factor or just an additive factor from frame to frame, but not both. Some other previously proposed integrated approaches are limited to specialized tasks such as image registration or change recovery. To remedy this shortcoming, this paper presents a new robust approach to general motion estimation in an integrated framework. Our approach deals simultaneously with motion discontinuities and large illumination variations. Our model of illumination variation is general, in the sense that it admits both multiplicative and additive effects.

Accurate optical flow computation under non-uniform brightness variations

In this paper, we present a very accurate algorithm for computing optical flow with non-uniform brightness variations. The proposed algorithm is based on a generalized dynamic image model (GDIM) in conjunction with a regularization framework to cope with the problem of non-uniform brightness variations. To alleviate flow constraint errors due to image aliasing and noise, we employ a reweighted least-squares method to suppress unreliable flow constraints, thus leading to robust estimation of optical flow. In addition, a dynamic smoothness adjustment scheme is proposed to efficiently suppress the smoothness constraint in the vicinity of the motion and brightness variation discontinuities, thereby preserving motion boundaries. We also employ a constraint refinement scheme, which aims at reducing the approximation errors in the first-order differential flow equation, to refine the optical flow estimation especially for large image motions. To efficiently minimize the resulting energy function for optical flow computation, we utilize an incomplete Cholesky preconditioned conjugate gradient algorithm to solve the large linear system. Experimental results on some synthetic and real image sequences show that the proposed algorithm compares favorably to most existing techniques reported in literature in terms of accuracy in optical flow computation with 100% density.

Reliable and Efficient Computation of Optical Flow

In this paper, we present two very efficient and accurate algorithms for computing optical flow. The first is a modified gradient-based regularization method, and the other is an SSD-based regularization method. For the gradient-based method, to amend the errors in the discrete image flow equation caused by numerical differentiation as well as temporal and spatial aliasing in the brightness function, we propose to selectively combine the image flow constraint and a contour-based flow constraint into the data constraint by using a reliability measure. Each data constraint is appropriately normalized to obtain an approximate minimum distance (of the data point to the linear flow equation) constraint instead of the conventional linear flow constraint. These modifications lead to robust and accurate optical flow estimation. We propose an incomplete Cholesky preconditioned conjugate gradient algorithm to solve the resulting large and sparse linear system efficiently. Our SSD-based regularization method uses a normalized SSD measure (based on a similar reasoning as in the gradient-based scheme) as the data constraint in a regularization framework. The nonlinear conjugate gradient algorithm in conjunction with an incomplete Cholesky preconditioning is developed to solve the resulting nonlinear minimization problem. Experimental results on synthetic and real image sequences for these two algorithms are given to demonstrate their performance in comparison with competing methods reported in literature.