Optical Flow Equation Research Articles

A massively parallel semi-Lagrangian algorithm for solving the transport equation

The scalar transport equation underpins many models employed in science, engineering, technology and business. Application areas include, but are not restricted to, pollution transport, weather forecasting, video analysis and encoding (the optical flow equation), options and stock pricing (the Black-Scholes equation) and spatially explicit ecological models. Unfortunately finding numerical solutions to this equation which are fast and accurate is not trivial. Moreover, finding such numerical algorithms that can be implemented on high performance computer architectures efficiently is challenging. In this paper the authors describe a massively parallel algorithm for solving the advection portion of the transport equation. We present an approach here which is different to that used in most transport models and which we have tried and tested for various scenarios. The approach employs an intelligent domain decomposition based on the vector field of the system equations and thus automatically partitions the computational domain into algorithmically autonomous regions. The solution of a classic pure advection transport problem is shown to be conservative, monotonic and highly accurate at large time steps. Additionally we demonstrate that the algorithm is highly efficient for high performance computer architectures and thus offers a route towards massively parallel application.

Motion estimation of tagged cardiac magnetic resonance images using variational techniques

This work presents a new method for motion estimation of tagged cardiac magnetic resonance sequences based on variational techniques. The variational method has been improved by adding a new term in the optical flow equation that incorporates tracking points with high stability of phase. Results were obtained through simulated and real data, and were validated by manual tracking and with respect to a reference state-of-the-art method: harmonic phase imaging (HARP). The error, measured in pixels per frame, obtained with the proposed variational method is one order of magnitude smaller than the one achieved by the reference method, and it requires a lower computational cost.

Optical-Flow-Based B-Mode Elastography: Application in the Hypertensive Rat Carotid

Background-Ultrasound elastography is now used worldwide in tissue characterization. The primary premises of elastography are that speckle kinematics reproduces underlying tissue kinematics and that tissue motion can be inferred from speckle tracking. This implicitly assumes that speckle pattern is a material property that can be tracked with respect to time and space. It is then convenient to express the motion of such a material property in terms of total derivative, also known as optical flow (OF) equations. Aims-The present paper introduces a new iterative OF-based elastography (OFBE) method devoted to B-mode data. The first OFBE iteration computes axial and lateral displacement fields. Such displacement fields are used for data rigid registration, prior to the second OFBE iteration which computes the 2-D strain tensor. Methods-The OFBE method was validated in the common carotid artery of rat hypertension models. The effect of aging on carotid stiffness was investigated in female recombinant inbred rats (RI-17, (n=2)) in the first experiment. The outcomes of low/high-salt diets were examined in young male Dahl salt-sensitive rats (SS, n=6; SM12, n=6; SM9, n=6) in the second experiment. Results-Good concordance was observed between left and right carotid axial strain measurements with 11.4% relative error, whereas 4.6% relative error occurred between diastolic and systolic axial strain measurements. Old (80 and 85 weeks) RI-17 carotids were determined to be twice as stiff with 5.70 +/- 0.97% (strain+/-std) as young carotids (30 and 34 weeks) with 13.26 +/- 2.73%, p < 0.001. Carotid axial strain measurement also indicated that salt diets had a significant impact on SS (p=0.008) and SM12 (p < 0.001) but not on SM9 (p=0.881) rats.

Learning visual motion and structure as latent explanations from huge motion data sets

Event Abstract Back to Event Learning visual motion and structure as latent explanations from huge motion data sets Rudolf Mester1*, Alvaro Guevara1, Christian Conrad1 and Holger Friedrich1 1 Goethe-Universität Frankfurt, Bernstein Focus Neurotechnology (BFNT), Germany The term ‘motion’ denotes a simple explanation for a possibly complicated change of the illumination pattern sensed in an eye or a camera. Motion explains these changes by transformation of the brightness pattern such as e.g. shift, rotation, and scaling. Other important components of these ‘explanations’ are segmentation (= grouping into spatially connected objects) and scene depth. In that sense, ‘motion’ is an, in an information-theoretic sense, ’cheap’ description of the second and further image in a sequence, conditioned on the first image to be given. The question is, based on which principles, purely based on long-term observation, perception is capable to distill concepts such as motion, depth, and segmentation from the continuous stream of visual input data, without having a priori access to mathematical models of the world and to models of the signals observed which a priori employ these ‘explanatory entities’. Optical flow is the representation of motion in the image plane, and ‘technical’ equations such as the brightness constancy constraint equation (BCCE) state an explicit relation between the observable entities (spatio-temporal derivates of the image signal), and the unknown explanatory variable, i.e. motion. However: how can such a relation be learnt, instead of constructed on the basis of an already available ‘higher insight’? We suggest that the emergence of concepts such as motion in a visual perception system is largely supported by (ego)motoric information, and by discovering statistical correlations (not necessarily only linear ones) between motoric data and characteristics of the instantaneous spatio-temporal characteristics of the visual motion field. Both local motion (as it appears in the optical flow equations) as well as global motion (e.g. parametric descriptions of the complete visual motion field) are claimed to be informative ‘latent variables’ that emerge from a statistical analysis of observable sensory information, in particular the spatio-temporal image signal as well as motoric signals. We are currently exploring this hypothesis on the basis of a large-scale experiment where an autonomous robot continuously ‘explores’ an indoor environment (using standard methods), and collects huge multi-channel video data streams that will be subject to an in-depth analysis in the spirit of the approach sketched above. In contrast to [1], we address specifically to learn (=identify) the latent variables, not primarily to learn the distribution of the parameters. This is an approach that is more in the spirit of [2] where also transformations are learnt.

Generalized least squares-based parametric motion estimation

The estimation of parametric global motion had a significant attention during the last two decades, but despite the great efforts invested, there are still open issues. The most important ones are related to the accuracy of the estimation and to the ability to recover large deformation between images. In this paper, a new generalized least squares-based motion estimator is proposed. The non-linear Brightness Constancy Assumption is directly used instead of using the classical approach by linearizing the minimization problem using the optical flow equation. In addition, the proposed formulation of the motion estimation problem provides an additional constraint that helps to match the pixels by using the image gradient in the matching process. That is achieved by means of a weight for each observation, assigning high weight values to the observations considered as inliers, i.e. the ones that support the motion model, and low values to the ones considered as outliers. The accuracy of our approach has been tested using challenging real images using both affine and projective motion models. Two motion estimator techniques that uses iteratively reweighted least squares-based (IRLS) techniques to deal with outliers, have been selected for comparison purposes. The results obtained show that the proposed motion estimator can obtain, in most cases, more accurate estimates that the IRLS-based techniques.

Fluid flow and optical flow

The connection between fluid flow and optical flow is explored in typical flow visualizations to provide a rational foundation for application of the optical flow method to image-based fluid velocity measurements. The projected-motion equations are derived, and the physics-based optical flow equation is given. In general, the optical flow is proportional to the path-averaged velocity of fluid or particles weighted with a relevant field quantity. The variational formulation and the corresponding Euler–Lagrange equation are given for optical flow computation. An error analysis for optical flow computation is provided, which is quantitatively examined by simulations on synthetic grid images. Direct comparisons between the optical flow method and the correlation-based method are made in simulations on synthetic particle images and experiments in a strongly excited turbulent jet.

Experimental Surface Strain Mapping of Porcine Peripapillary Sclera Due to Elevations of Intraocular Pressure

To experimentally characterize 2D surface mapping of the deformation pattern of porcine peripapillary sclera following acute elevations of intraocular pressure (IOP) from 5 mm Hg to 45 mm Hg. Four porcine eyes were obtained within 48 h postmortem and dissected to the sclera. After the anterior chamber was removed, each posterior scleral shell was individually mounted at the equator on a custom-built pressurization device, which internally pressurized the scleral samples with isotonic saline at 22 degrees C. Black polystyrene microspheres (10 microm in diameter) were randomly scattered and attached to the scleral surface. IOP was incrementally increased from 5 mm Hg to 45 mm Hg (+/-0.15 mm Hg), and the surface deformation of the peripapillary sclera immediately adjacent to the dural insertion was optically tracked at a resolution of 2 micrompixel one quadrant at a time, for each of four quadrants (superior, nasal, inferior, and temporal). The 2D displacement data of the microsphere markers were extracted using the optical flow equation, smoothed by weighting function interpolation, and converted to the corresponding Lagrangian finite surface strain. In all four quadrants of each eye, the principal strain was highest and primarily circumferential immediately adjacent to the scleral canal. Average maximum Lagrangian strain across all quadrants for all eyes was 0.013+/-0.005 from 5 mm Hg to 10 mm Hg, 0.014+/-0.004 from 10 mm Hg to 30 mm Hg and 0.004+/-0.001 from 30 mm Hg to 45 mm Hg, demonstrating the nonlinearity in the IOP-strain relationship. For each scleral shell, the observed surface strain mapping implied that the scleral stiffness was relatively low between 5 mm Hg and 10 mm Hg, but dramatically increased for each IOP elevation increment beyond 10 mm Hg. Peripapillary deformation following an acute IOP elevation may be governed by the underlying scleral collagen microstructure and is likely in the high-stiffness region of the scleral stress-strain curve when IOP is above 10 mm Hg.

Evaluating an optical-flow-based registration algorithm for contrast-enhanced magnetic resonance imaging of the breast

Dynamic contrast-enhanced magnetic resonance imaging studies of the breast are frequently degraded by patient motion. In order to correct for this, any registration algorithm must overcome two major challenges: the highly deformable nature of the breast itself and the need to remove changes in signal intensity due to patient motion whilst leaving potentially significant changes in signal intensity due to changes in contrast agent concentration unchanged. In this paper, we evaluate the use of a non-rigid registration method that uses optical flow equations to drive the displacement of a grid of control points. With conventional optical flow techniques it is assumed that changes in image intensity are solely due to motion, making it unsuitable for use with contrast-enhanced studies. The registration algorithm evaluated in this paper overcomes this problem by including an additional term to account for changes in image intensity. Studies simulating physiologically plausible deformations of the breast together with realistic changes in contrast-enhancement derived from patient studies demonstrate that the algorithm is capable of registering images to sub-voxel accuracy within minutes. This technique has now been successfully incorporated into a breast cancer screening protocol allowing registered images to be provided routinely to the radiologist immediately after the scanning session.

A HYBRID AND HIERARCHICAL APPROACH TO AERIAL IMAGE REGISTRATION

This paper proposes a hybrid approach to image registration for inferring the affine transformation that best matches a pair of partially overlapping aerial images. The image registration is formulated as a two-stage hybrid approach combining both phase correlation method (PCME) and optical flow equation (OFE) based estimation algorithm in a coarse-to-fine manner. With PCME applied at the highest level of decomposition, the initial affine parameter model could be first estimated. Subsequently, the OFE-based estimation algorithm is incorporated into the proposed hybrid approach using a multi-resolution mechanism. PCME is characterized by its insensitivity to large geometric transform between images, which can effectively guide the OFE-based registration. For image pairs under salient brightness variations, we propose a nonlinear image representation that emphasizes common intensity information, suppresses the non-common information between an image pair, and is suitable for the proposed coarse-to-fine hierarchical iterative processing. Experimental results demonstrate the accuracy and efficiency of our proposed approach using different types of aerial images.

Non-invasive high-frequency vascular ultrasound elastography

Non-invasive vascular elastography (NIVE) was recently introduced to characterize mechanical properties of superficial arteries. In this paper, the feasibility of NIVE and its applicability in the context of high-frequency ultrasound imaging is investigated. First, experiments were performed in vitro on vessel-mimicking phantoms. Polyvinyl alcohol cryogel was used to create two double-layer vessels with different mechanical properties. In both cases, the stiffness of the inner layer was made softer. Radial stress was applied within the lumen of the phantoms by applying incremental static pressure steps with a column of a flowing mixture of water–glycerol. The vessel phantoms were insonified at 32 MHz with an ultrasound biomicroscope to provide cross-section sequences of radio-frequency (RF) ultrasound data. The Lagrangian speckle model estimator (LSME) was used to assess the two-dimensional-strain tensors, and the composite Von Mises elastograms were computed. A new implementation of the LSME based on the optical flow equations was introduced. Deformation parameters were estimated using an inversion algorithm. For each in vitro experiment, both layers of approximately 1 mm were distinguished. Second, the use of the method for the purpose of studying small vessels (MicroNIVE) in genetically engineered rodents was introduced. Longitudinal scans of the carotid artery were performed at 40 MHz. The in vivo results give confidence in the feasibility of MicroNIVE as a potential tool to non-invasively study the impact of targeted genes on vascular remodelling in rodents.

On the potential of the Lagrangian speckle model estimator to characterize atherosclerotic plaques in endovascular elastography: In vitro experiments using an excised human carotid artery

Endovascular ultrasound (US) elastography (EVE) was introduced to supplement endovascular US echograms in the assessment of vessel lesions and for endovascular therapy planning. Indeed, changes in the vascular tissue stiffness are characteristic of vessel wall pathologies and EVE appears as a very appropriate imaging technique to outline the elastic properties of vessel walls. Recently, a model-based approach was proposed to assess tissue motion in EVE. It specifically consists of a nonlinear minimization algorithm that was adapted to speckle motion estimation. Regarding the theoretical framework, such an approach considers the speckle as a material property; this assumption then led to the derivation of the optical flow equations, which were suitably combined with the Lagrangian speckle model estimator to provide the full 2-D polar strain tensor. In this study, the proposed algorithm was validated in vitro using a fresh excised human carotid artery. The experimental setup consisted of a cardiovascular imaging system (CVIS) US scanner, working with a 30-MHz mechanical rotating single-element transducer, a digital oscilloscope and a pressuring system. A sequence of radiofrequency (RF) images was collected while incrementally adjusting the intraluminal static pressure steps. The results showed the potential of this 2-D algorithm to characterize and to distinguish an atherosclerotic plaque from the normal vascular tissue. Namely, the geometry as well as some mechanical characteristics of the detected plaque were in good agreement with histology. The results also suggested that there might exist a range of intraluminal pressures for which plaque detectability is optimal. (E-mail: guy.cloutier@umontreal.ca)

Adapting the Lagrangian speckle model estimator for endovascular elastography: theory and validation with simulated radio-frequency data.

Intravascular ultrasound (IVUS) is known to be the reference tool for preoperative vessel lesion assessments and for endovascular therapy planning. Nevertheless, IVUS echograms only provide subjective information about vessel wall lesions. Since changes in the vascular tissue stiffness are characteristic of vessel pathologies, catheter-based endovascular ultrasound elastography (EVE) has been proposed in the literature as a method for outlining the elastic properties of vessel walls. In this paper, the Lagrangian Speckle Model Estimator (LSME) is formulated for investigations in EVE, i.e., using a polar coordinate system. The method was implemented through an adapted version of the Levenberg-Marquardt minimization algorithm, using the optical flow equations to compute the Jacobbian matrix. The theoretical framework was validated with simulated ultrasound rf data of mechanically complex vessel wall pathologies. The results, corroborated with Ansys finite element software, demonstrated the potential of EVE to provide useful information about the heterogeneous nature of atherosclerotic plaques.

Shape and motion from simultaneous equations with closed-loop solution

This paper proposes a method for simultaneously estimating 2D image motion and 3D object shape and motion from only two frames. The problem is formulated in a system of equations, including the differential epipolar constraint, a newly derived optical flow equation and surface normal constraint, under the assumption of perspective projection, rigid motion, Lambertian reflectance and distant lighting. A closed-loop solver is constructed based on the simultaneous equations to export accurate estimate for optical flow as well as dense shape and motion. Experimental results are also provided.

An Information Theoretic Criterion for Evaluating the Quality of 3-D Reconstructions From Video

Even though numerous algorithms exist for estimating the three-dimensional (3-D) structure of a scene from its video, the solutions obtained are often of unacceptable quality. To overcome some of the deficiencies, many application systems rely on processing more data than necessary, thus raising the question: how is the accuracy of the solution related to the amount of data processed by the algorithm? Can we automatically recognize situations where the quality of the data is so bad that even a large number of additional observations will not yield the desired solution? Previous efforts to answer this question have used statistical measures like second order moments. They are useful if the estimate of the structure is unbiased and the higher order statistical effects are negligible, which is often not the case. This paper introduces an alternative information-theoretic criterion for evaluating the quality of a 3-D reconstruction. The accuracy of the reconstruction is judged by considering the change in mutual information (MI) (termed as the incremental MI) between a scene and its reconstructions. An example of 3-D reconstruction from a video sequence using optical flow equations and known noise distribution is considered and it is shown how the MI can be computed from first principles. We present simulations on both synthetic and real data to demonstrate the effectiveness of the proposed criterion.

A fast parametric motion estimation algorithm with illumination and lens distortion correction

Methods for estimating motion in video sequences that are based on the optical flow equation (OFE) assume that the scene illumination is uniform and that the imaging optics are ideal. When these assumptions are appropriate, these methods can be very accurate, but when they are not, the accuracy of the motion field drops off accordingly. This paper extends the models upon which the OFE methods are based to include irregular, time-varying illumination models and models for imperfect optics that introduce vignetting, gamma, and geometric warping, such as are likely to be found with inexpensive PC cameras. The resulting optimization framework estimates the motion parameters, illumination parameters, and camera parameters simultaneously. In some cases these models can lead to nonlinear equations which must be solved iteratively; in other cases, the resulting optimization problem is linear. For the former case an efficient, hierarchical, iterative framework is provided that can be used to implement the motion estimator.

Motion estimation based on the energy flow*

Abstract By taking pixels of image sequences as moving gas molecules a novel concept image temperature is proposed to describe the natural property of the image motion. The idea comes from the revelation of the Maxwell- Boltzmann Distribution Law in gas dynamic theories. Furthermore another concept of energy flow corresponding to the optical flow is developed and the method of the energy flow equation EFE is established to estimate imagemotion. The experiment indicates a better performance of the proposed EFE scheme with significantly reduced false motion estimates when compared to the traditional optical flow equation OFE.

Recursive estimation of motion and shape parameters under perspective projection

The problem of estimating motion and shape of a moving rigid object has been considered from a time series of image data obtained via perspective projection. First it is shown that, for a class of motion parameters, the shape parameters are asymptotically decoupled from the optical flow dynamics, hence become unobservable asymptotically. For other choices of motion parameters, shape parameters remain asymptotically obseravble. Secondly we show that, if the surface is almost planar, then uncertainty in the shape of the surface propagates as an optical flow equation with additive state dependent uncertainty. Finally, we develop a recursive estimation algorithm, and present the numerical simulation results.

Finding the epipole from uncalibrated optical flow

This article presents a novel method for determining the location of the instantaneous epipole in a sequence of images acquired by an uncalibrated camera and containing a single, rigid motion (e.g. the camera moves in a static environment). The method uses the full perspective camera model and requires the estimation of the optical flow at a minimum of six image locations. The key observation is that the optical flow equations can be written in terms of the epipole in a strikingly simple form if the translational and rotational flow components are not separated as performed usually. The epipole location can then be obtained as the minimum of a least-square residual function associated to the computed optical flow. We report and discuss initial experiments on both synthetic and real data and illustrate possible developments of this method towards the use of uncalibrated optical flow for 3-D motion and structure reconstruction.

Non Uniform Multiresolution Method for Optical Flow and Phase Portrait Models: Environmental Applications

In this paper we define a complete framework for processing large image sequences for a global monitoring of short range oceanographic and atmospheric processes. This framework is based on the use of a non quadratic regularization technique for optical flow computation that preserves flow discontinuities. We also show that using an appropriate tessellation of the image according to an estimate of the motion field can improve optical flow accuracy and yields more reliable flows. This method defines a non uniform multiresolution approach for coarse to fine grid generation. It allows to locally increase the resolution of the grid according to the studied problem. Each added node refines the grid in a region of interest and increases the numerical accuracy of the solution in this region. We make use of such a method for solving the optical flow equation with a non quadratic regularization scheme allowing the computation of optical flow field while preserving its discontinuities. The second part of the paper deals with the interpretation of the obtained displacement field. For this purpose a phase portrait model used along with a new formulation of the approximation of an oriented flow field allowing to consider arbitrary polynomial phase portrait models for characterizing salient flow features. This new framework is used for processing oceanographic and atmospheric image sequences and presents an alternative to complex physical modeling techniques.

Fast motion estimation for 2D triangular mesh basedimage coding

A fast motion estimation method for 2D triangular mesh based image coding is presented. To estimate the motions of image sequences, this method is based on the combination of optical flow equation, affine transform and fast iteration using a multiple linear regression model. The method proposed has significantly reduced computational cost compared with conventional 2D triangular mesh based image coding, without producing any additional degradation in the quality of the reconstructed image.