Gradient-based Optical Flow Research Articles

Fast and robust volumetric refractive index measurement by unified background-oriented schlieren tomography

We propose a novel approach to background-oriented schlieren (BOS) tomography (BOST) that unifies the deflection sensing and reconstruction algorithms. BOS is a 2D flow visualization technique that renders light deflections due to refraction in the fluid. Simultaneous BOS measurements from unique views can be reconstructed by tomography to estimate the fluid’s 3D refractive index field. The cameras are focused through the fluid on textured background patterns. Deflections between an undistorted reference image and distorted image are typically determined by gradient-based optical flow (OF), which is a complex inverse problem and potential source of error in BOST. This paper presents an alternative approach to BOST that unifies the OF equations and deflection model. Our new operator simultaneously calculates the image distortions seen by each camera for a discrete refractive index distribution. Unified BOST (UBOST) thus reconstructs observed image distortions instead of inferred deflections, which are influenced by user-selected OF parameters. The UBOST operator has one third as many equations as the classical BOST operator. We show that our formulation reduces the effects of model error and the computational cost of reconstruction. These advantages are demonstrated with a numerical experiment using phantoms of varied complexity. Best practice UBOST reconstructions were more accurate than classical reconstructions of the exact deflections for each phantom. Moreover, UBOST estimates converged substantially faster, resulting in a $$\ge $$ 62.5% speedup with our solver.

Embedded Video Surveillance Based Moving Object Detection and Tracking

Background reckoning and the foreground, play prominent roles in the tasks of visual detection and tracking of objects. Moving Object Detection has been widely used in sundry discipline such as intelligent systems, security systems, video monitoring systems, banking places, provisionary systems, and so on. In this paper proposes moving objects detection and tracking method based on Embedded Video Surveillance. The method is based on using lines computed by a gradient-based optical flow and an edge detector gradient-based optical flow and edges are well matched for accurate computation of velocity, not much attention is paid to creating systems for tracking objects using this feature. The proposed method is compared with a recent work, proving its superior performance and when we want to represent high quality videos and images with, lower bit rate, and also suitable for real-world live video applications. This method reduces influences of foreground objects to the background model. The simulation results show that the background image can be obtained precisely and the moving objects recognition is achieved effectively.

22-4 輝度勾配法を用いたオプティカルフローによる動き推定の比較

22-4 輝度勾配法を用いたオプティカルフローによる動き推定の比較

Activity representation by SURF-based templates

This paper presents a method to recognise actions, which are overlapping and multi-dimensionalities. A spatio-temporal representation is illustrated on local interest points to compute global features. Motion history image (MHI) is computed and motion overwriting the motion overwriting problem of the MHI. The main contribution of this paper is that it demonstrates a higher discriminative ability of various complex actions when compared to the other MHI-based approaches. It selects local interest feature points to capture motion information using Speeded-Up Robust Features (SURF). These key interest points are exploited to compute gradient-based optical flow into four channels. RANSAC is exploited to remove outliers. It incorporates frame-subtracted accumulated image so that we can mask out points that are not required. Afterwards, feature vectors are computed based on moments. Actions are recognised by employing a nearest neighbour classifier and leave-one-out cross-validation partitioning scheme. The proposed method provides satisfactory recognition rates over several other approaches for some challenging actions in outdoor scenes.

The subpixel resolution of optical-flow-based modal analysis

This research looks at the possibilities for full-field, non-contact, displacement measurements based on high-speed video analyses. A simplified gradient-based optical flow method, optimised for subpixel harmonic displacements, is used to predict the resolution potential. The simplification assumes an image-gradient linearity, producing a linear relation between the light intensity and the displacement in the direction of the intensity gradient. The simplicity of the method enables each pixel or small subset to be viewed as a sensor. The resolution potential and the effect of noise are explored theoretically and tested in a synthetic experiment, which is followed by a real experiment. The identified displacement can be smaller than a thousandth of a pixel and subpixel displacements are recognisable, even with a high image noise. The resolution and the signal-to-noise ratio are influenced by the dynamic range of the camera, the subset size and the sampling length. Real-world experiments were performed to validate and demonstrate the method using a monochrome high-speed camera. One-dimensional mode shapes of a steel beam are recognisable even at the maximum displacement amplitude of 0.0008 pixel (equal to 0.2μm) and multiple out-of-plane mode shapes are recognisable from the high-speed video of a vibrating cymbal.

A Real-Time Microscopic PIV System Using Frame Straddling High-Frame-Rate Vision

In this paper, we propose a novel concept of realtime microscopic particle image velocimetry (PIV) for apparent high-speed microchannel flows in lab-on-achip (LOC). We introduce a frame-straddling dualcamera high-speed vision system that synchronizes two different camera inputs for the same camera view with a submicrosecond time delay. In order to improve upper and lower limits of measurable velocity in microchannel flow observation, we designed an improved gradient-based optical flow algorithm that adaptively selects a pair of images in the optimal frame-straddling time between the two camera inputs based on the amplitude of the estimated optical flow. This avoids large image displacement between frames that often generates serious errors in optical flow estimation. Our method is implemented using software on a frame-straddling dual-camera high-speed vision platform that captures real-time video and processes 512 × 512 pixel images at 2000 fps for the two camera heads and controls the frame-straddling time delay between them from 0 to 0.25 ms with 9.9 ns step. Our microscopic PIV system with frame-straddling dualcamera high-speed vision simultaneously estimates the velocity distribution of high-speed microchannel flow at 1 × 108pixel/s or more. Results of experiments using real microscopic flows on microchannels thousands of µm wide on LOCs verify the performance of the real-time microscopic PIV system we developed.

Gait Analysis: An Energy Image-Based Approach

Gait recognition and analysis are very crucial research areas in the field of computer vision for various applications. This paper presents a simple approach for gait analysis. It is based on the four-directional motion energy image calculated from gradient-based optical flow computation. The binarized energy images for horizontal directions are considered and the first order geometric moments are computed to have the concept of speed for walking or running motion. To show the robustness of this approach, we employed three standard datasets for our experiment. The experimental results both in indoor and outdoor datasets show satisfactory performance of the proposed method.

Robust motion estimation on a low-power multi-core DSP

This paper addresses the efficient implementation of a robust gradient-based optical flow model in a low-power platform based on a multi-core digital signal processor (DSP). The aim of this work was to carry out a feasibility study on the use of these devices in autonomous systems such as robot navigation, biomedical assistance, or tracking, with not only power restrictions but also real-time requirements. We consider the C6678 DSP from Texas Instruments (Dallas, TX, USA) as the target platform of our implementation. The interest of this research is particularly relevant in optical flow scope because this system can be considered as an alternative solution for mid-range video resolutions when a combination of in-processor parallelism with optimizations such as efficient memory-hierarchy exploitation and multi-processor parallelization are applied.

Quantization analysis and enhancement of a VLSI gradient-based motion estimation architecture

Despite more than 40 years of research, motion estimation is still considered an emerging field, a field especially relevant today because of its vast utility for real-world applications. Currently, even the best bio-inspired algorithms lack certain characteristics that are readily found, for example, naturally in, say, mammals. Furthermore, the vast computational resources required are not usually affordable in real-time application. We present here a useful framework for building bio-inspired systems in real-time environments, reducing computational complexity. A complete quantization study of neuromorphic robust optical flow architecture is performed, using properties found in the cortical motion pathway. This architecture is designed for VLSI systems. An extensive analysis is performed to avoid compromising the viability and the robustness of the final system. A set of simulations and techniques that can be helpful for designing real-time artificial vision embedded systems and, specifically, gradient-based optical flow systems is shown. This work includes the final error results, resource usage, and performance data.

Bulk wind estimation and prediction for adaptive optics control systems

We present a wind-predictive controller for astronomical adaptive optics (AO) systems that is able to predict the motion of a single windblown layer in the presence of other, more slowly varying phase aberrations. This controller relies on fast, gradient-based optical flow estimation to identify the velocity of the translating layer and a recursive mean estimator to account for turbulence that varies on a time scale much slower than the operating speed of the AO loop. We derive the Cramer-Rao lower bound for the wind estimation problem and show that the proposed estimator is very close to achieving theoretical minimum-variance performance. We also present simulations using on-sky data that show significant Strehl increases from using this controller in realistic atmospheric conditions.

A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets

Expert landmark correspondences are widely reported for evaluating deformable image registration (DIR) spatial accuracy. In this report, we present a framework for objective evaluation of DIR spatial accuracy using large sets of expert-determined landmark point pairs. Large samples (>1100) of pulmonary landmark point pairs were manually generated for five cases. Estimates of inter- and intra-observer variation were determined from repeated registration. Comparative evaluation of DIR spatial accuracy was performed for two algorithms, a gradient-based optical flow algorithm and a landmark-based moving least-squares algorithm. The uncertainty of spatial error estimates was found to be inversely proportional to the square root of the number of landmark point pairs and directly proportional to the standard deviation of the spatial errors. Using the statistical properties of this data, we performed sample size calculations to estimate the average spatial accuracy of each algorithm with 95% confidence intervals within a 0.5 mm range. For the optical flow and moving least-squares algorithms, the required sample sizes were 1050 and 36, respectively. Comparative evaluation based on fewer than the required validation landmarks results in misrepresentation of the relative spatial accuracy. This study demonstrates that landmark pairs can be used to assess DIR spatial accuracy within a narrow uncertainty range.

A Contour-Based Moving Object Detection

We propose a fast and robust approach to the detection of moving objects. Our method is based on using lines computed by a gradient-based optical flow and an edge detector. While it is known among researchers that gradient-based optical flow and edges are well matched for accurate computation of velocity, not much attention is paid to creating systems for detecting objects using this feature. In our method, extracted edges by using optical flow and the edge detector are restored as lines, and background lines of the previous frame are subtracted. Contours of objects are obtained by using snakes to clustered lines. The experimental results on outdoor-scenes show fast and robust performance of our method. The computation time of our method is 0.089 s/frame on a 900 MHz processor.

A Robust Docking Strategy for a Mobile Robot Using Flow Field Divergence

We present a robust strategy for docking a mobile robot in close proximity with an upright surface using optical flow field divergence and proportional feedback control. Unlike previous approaches, we achieve this without the need for explicit segmentation of features in the image, and using complete gradient-based optical flow estimation (i.e., no affine models) in the optical flow computation. A key contribution is the development of an algorithm to compute the flow field divergence, or time-to-contact, in a manner that is robust to small rotations of the robot during ego-motion. This is done by tracking the focus of expansion of the flow field and using this to compensate for ego rotation of the image. The control law used is a simple proportional feedback, using the unfiltered flow field divergence as an input, for a dynamic vehicle model. Closed-loop stability analysis of docking under the proposed feedback is provided. Performance of the flow field divergence algorithm is demonstrated using offboard natural image sequences, and the performance of the closed-loop system is experimentally demonstrated by control of a mobile robot approaching a wall.

Real-time Architecture for Robust Motion Estimation under Varying Illumination Conditions

Motion estimation from image sequences is a complex problem which requires high computing resources and is highly affected by changes in the illumination conditions in most of the existing approaches. In this contribution we present a high performance system that deals with this limitation. Robustness to varying illumination conditions is achieved by a novel technique that combines a gradient-based optical flow method with a non-parametric image transformation based on the Rank transform. The paper describes this method and quantitatively evaluates its robustness to different illumination changing patterns. This technique has been successfully implemented in a real-time system using reconfigurable hardware. Our contribution presents the computing architecture, including the resources consumption and the obtained performance. The final system is a real-time device capable to computing motion sequences in real-time even in conditions with significant illumination changes. The robustness of the proposed system facilitates its use in multiple potential application fields.

Optical flow based vehicle tracking strengthened by statistical decisions

Reliable tracking of cars from aerial video imagery is one of the main ingredients of microscopic traffic monitoring. Current tracking methods however are not able yet to track all the vehicles in all frames of video imagery taken by e.g. a helicopter. Several problem scenarios can be distinguished, like situations with many similar cars in congested traffic areas, cars that appear in low contrast compared to the background and cars that are occluded in some frames by other cars or by traffic signs. In this paper an improved method is described that continuously tracks all vehicles from their appearance into the viewing area until their exit. Our algorithm starts by separately tracking individual car pixels and the complete car region as a whole using the gradient-based optical flow method. A scale space approach is used to initiate the optical flow method. The best result as obtained from the intermediate results is used in the following statistical decision making step. Finally, either the best results are accepted and by applying a rigid body assumption, one displacement result is adapted for the car as a whole, or the best results are rejected, because even the best results fail a quality criterion. Continuation of these steps for all frames constitutes the final tracking result. This method solves most of the sketched problem scenarios as is illustrated by applying it on suited helicopter video imagery.

Optical flow for large motion using gradient technique

In this paper, we present the gradient-based optical flow method that estimates the two-dimensional velocity of object motion. a multi-resolution smoothing operation proposes in this paper as a pre-processing step for overcoming the difficulty of large motion estimation by gradient-based optical flow techniques. the effectiveness of the proposed method has confirmed by applying image sequence of large motion experimental results with an image sequence show a qualitative improvement.

Analog Integrated 2-D Optical Flow Sensor

I present a new focal-plane analog very-large-scale-integrated (aVLSI) sensor that estimates optical flow in two visual dimensions. Its computational architecture consists of a two-layer network of locally connected motion units that collectively estimate the optimal optical flow field. The applied gradient-based optical flow model assumes visual motion to be translational and smooth, and is formulated as a convex optimization problem. The model also guarantees that the estimation problem is well-posed regardless of the visual input by imposing a bias towards a preferred motion under ambiguous or noisy visual conditions. Model parameters can be globally adjusted, leading to a rich output behavior. Varying the smoothness strength, for example, can provide a continuous spectrum of motion estimates, ranging from normal to global optical flow. The non-linear network conductances improve the resulting optical flow estimate because they reduce spatial smoothing across large velocity differences and minimize the bias for reliable stimuli. Extended characterization and recorded optical flow fields from a 30 × 30 array prototype sensor demonstrate the validity of the optical flow model and the robustness and functionality of the computational architecture and its implementation.

Errors-in-variables modeling in optical flow estimation

Gradient-based optical flow estimation methods typically do not take into account errors in the spatial derivative estimates. The presence of these errors causes an errors-in-variables (EIV) problem. Moreover, the use of finite difference methods to calculate these derivatives ensures that the errors are strongly correlated between pixels. Total least squares (TLS) has often been used to address this EIV problem. However, its application in this context is flawed as TLS implicitly assumes that the errors between neighborhood pixels are independent. In this paper, a new optical flow estimation method (EIVM) is formulated to properly treat the EIV problem in optical flow. EIVM is based on Sprent's (1966) procedure which allows the incorporation of a general EIV model in the estimation process. In EIVM, the neighborhood size acts as a smoothing parameter. Due to the weights in the EIVM objective function, the effect of changing the neighborhood size is more complex than in other local model methods such as Lucas and Kanade (1981). These weights, which are functions of the flow estimate, can alter the effective size and orientation of the neighborhood. In this paper, we also present a data-driven method for choosing the neighborhood size based on Stein's unbiased risk estimators (SURE).

Filtering requirements for gradient-based optical flow measurement

The accuracy of gradient-based optical flow algorithms depends on the ability to measure intensity gradients accurately. We show how the temporal gradient can be compromised by temporal aliasing arising from motion and how appropriate post-sampling spatial filtering improves the situation. We also demonstrate a benefit of using higher-order gradient estimators.

Bandpass optical flow for tagged MRI.

MR tagging has shown great promise for detailed noninvasive cardiac motion imaging. Our research uses low-frequency tags coupled with gradient-based optical flow estimation to compute cardiac motion. We develop here a novel, fast, fully automated optical flow method for tagged MRI by exploiting the Fourier content of the tagged images. This new method, called bandpass optical flow, works by extracting various subband images from tagged cardiac data, and then formulating multiple optical flow constraints for each subband. The resulting system is solved by least squares pseudo-inversion. The proposed method is validated on simulated and real tagged data.