Optical Flow Computation Research Articles

Coalescence analysis for evolving foamsviaoptical flow computation on projection image sequences

A novel image-processing procedure is proposed for the analysis of sequences of two-dimensional projection images. Sudden events like the merging of bubbles in an evolving foam can be detected and spatio-temporally located in a given projection image sequence. The procedure is based on optical flow computations extended by a forward-backward check for each time step. Compared with prior methods, efficient suppression of noise or false events is achieved owing to uniform foam motion, and the reliability of detection is thus increased. The applicability of the proposed procedure in combination with synchrotron radiography is illustrated by a series of characteristic studies of foams of different kind. First, the detection of single-bubble collapses in aqueous foams is considered. Second, a spatial distribution of coalescence events in metals foamed in casting molds is estimated. Finally, the structural stability of polymer foams containing admixed solid nanoparticles is examined.

Weighted root mean square approach to select the optimal smoothness parameter of the variational optical flow algorithms

Variational methods are the most widely used approaches for optical flow computation. Many complicated algorithms have been proposed to improve their performance, yet little work has focused on how to select the optimal smoothness parameter λ of the variational optical flow algorithm itself. We present a weighted root mean square error method to automatically select the optimal smoothness parameter λ. Furthermore, we detail a scientific method for selecting the reference λ0 based on the quality of the frame, and propose an efficient brute-force approach to assign a group of λ, that will reduce the number of λ candidates to be tested by cutting down the search range. Experimental results validate the effectiveness of our methods.

Flow-Based Local Optimization for Image-to-Geometry Projection

The projection of a photographic data set on a 3D model is a robust and widely applicable way to acquire appearance information of an object. The first step of this procedure is the alignment of the images on the 3D model. While any reconstruction pipeline aims at avoiding misregistration by improving camera calibrations and geometry, in practice a perfect alignment cannot always be reached. Depending on the way multiple camera images are fused on the object surface, remaining misregistrations show up either as ghosting or as discontinuities at transitions from one camera view to another. In this paper we propose a method, based on the computation of Optical Flow between overlapping images, to correct the local misalignment by determining the necessary displacement. The goal is to correct the symptoms of misregistration, instead of searching for a globally consistent mapping, which might not exist. The method scales up well with the size of the data set (both photographic and geometric) and is quite independent of the characteristics of the 3D model (topology cleanliness, parametrization, density). The method is robust and can handle real world cases that have different characteristics: low level geometric details and images that lack enough features for global optimization or manual methods. It can be applied to different mapping strategies, such as texture or per-vertex attribute encoding.

Real-time Optical-flow Computation for Motion Estimation under Varying Illumination Conditions

<p>The optical flow approach has emerged as a major technique for estimating object motion in image sequences. However, the obtained results by most optical flow techniques are poor because they are strongly affected by large illumination changes and by motion discontinuities. On the other hand, there have been two thrusts in the development of optical flow algorithms. One has emphasized higher accuracy; the other faster implementation. These two thrusts have been independently pursed, without addressing the accuracy vs. efficiency trade-offs. The optical flow computation requires high computing resources and is highly affected by changes in the illumination conditions in most of the existing techniques. In this paper, a new strategy for image sequence processing is proposed. The data reduction achieved with this strategy allows a faster optical flow computation. In addition, the proposed architecture is a hardware custom implementation in EP1S60F1020 FPGA showing the achieved performance.</p>

Real-time Optical-flow Computation for Motion Estimation under Varying Illumination Conditions

Real-time Optical-flow Computation for Motion Estimation under Varying Illumination Conditions

Path Curvature Discrimination: Dependence on Gaze Direction and Optical Flow Speed

Many experimental approaches to the control of steering rely on the tangent point (TP) as major source of information. The TP is a good candidate to control self-motion. It corresponds to a singular and salient point in the subject's visual field, and its location depends on the road geometry, the direction of self-motion relative to the road and the position of the driver on the road. However, the particular status of the TP in the optical flow, as a local minimum of flow speed, has often been left aside. We therefore assume that the TP is actually an optimal location in the dynamic optical array to perceive a change in the trajectory curvature. In this study, we evaluated the ability of human observers to detect variations in their path curvature from optical flow patterns, as a function of their gaze direction in a virtual environment. We simulated curvilinear self-motion parallel to a ground plane. Using random-dot optic flow stimuli of brief duration and a two-alternative forced-choice adaptive procedure, we determined path curvature discrimination thresholds, as a function of gaze direction. The discrimination thresholds are minimal for a gaze directed toward a local minimum of optical flow speed. A model based on Weber fraction of the foveal velocities () correctly predicts the relationship between experimental thresholds and local flow velocities. This model was also tested for an optical flow computation integrating larger circular areas in central vision. Averaging the flow over five degrees leads to an even better fit of the model to experimental thresholds. We also found that the minimal optical flow speed direction corresponds to a maximal sensitivity of the visual system, as predicted by our model. The spontaneous gazing strategies observed during driving might thus correspond to an optimal selection of relevant information in the optical flow field.

A Shift-Tolerant Dissimilarity Measure for Surface Defect Detection

Template matching has been widely used in image processing for visual inspection of complicated, patterned surfaces. The currently existing methods such as golden template matching and normalized cross correlation are very sensitive to the displacement even the object under test is carefully aligned with respect to the template. This paper proposes a dissimilarity measure based on the optical-flow technique for surface defect detection, and aims at light-emitting diode (LED) wafer die inspection. The dissimilarity measure of each pixel derived from the optical flow field does not represent the true translation distance, but is reliable enough to indicate the degree of difference between an image pair. It is well tolerated to misalignment and random product variation. The integral image technique is applied to replace the sum operations in optical flow computation, and speeds up the intensive computation. We also point out the pitfall of the Lucas-Kanade optical flow when it is applied for defect detection, and propose a swapping process to tackle the problem. The experiment on LED wafer dies has shown that the proposed method can achieve a 100% recognition rate based on a test set of 357 die images.

Sonification of Face

In this paper, the integration of face segments detection and extraction, and facial expression recognition and Sonification are discussed. In this paper, we propose an algorithm that utilizes haar classifiers detecting face followed by dividing face into Regions of interest (ROI) to extract facial segments. Furthermore, in this project, we propose a statistical approach to process the optical flow data to obtain the overall value for the respective feature region in the face. This approach has eliminated the requirement of accurate identification of the feature boundary. Optical flow computations are utilized to identify the directions and the amount of motions in image sequences that are caused by human facial expressions. These calculated values are then compared with the values from table to estimate the expression .Furthermore after getting the expression; these are then converted to sound by mapping functions. Sounds are generated by simple MIDI tunes based on instruments assigned to each region.

F-Actin Mediated Chromosome Transport

F-actin meshwork contraction is a novel mechanism of intracellular transport that was recently shown to transport chromosomes during meiosis in starfish oocytes. The starfish model system is well suited to high-resolution fluorescence microscopy imaging. We use a high-throughput volume filtering procedure as well as optical flow calculations and spatio-temporal image correlation spectroscopy (STICS). We determine the meshwork pore sizes, node distributions, and branching angles, and relate these structural features to predictions of the recently-proposed “homogeneous contraction” model of transport. We also perform indirect structure analysis by simulating diffusing particles migrating though the F-actin meshwork to probe local F-actin structures and to investigate how F-actin bundle remodeling or binding to F-actin bundles would affect particle motion. We analyze these results in the context of percolation theory and well-understood diffusion model systems. We furthermore compare the simulated particle trajectories to experimental observations of chromosomes and inert PEG-coated beads diffusing within both active and utrophin-stabilized F-actin meshworks. The experimental data are in good agreement with our simulations, and both suggest that chromosome-sized particles can be transported by the F-actin meshwork as passive cargo undergoing confined diffusion.

Modified optical flow technique for cardiac motions analysis in echocardiography images

The quantitative analysis of cardiac motions in echocardiography images is a noteworthy issue in processing of these images. Cardiac motions can be estimated by optical flow (OF) computation in different regions of image which is based on the assumption that the intensity of a moving pattern remains constant in consecutive frames. However, in echocardiographic sequences, this assumption may be violated because of unique specifications of ultrasound. There are some methodsapplying the brightness variation effect in OF. Almost all of them have presented a mathematical brightness variation model globally in the images. Nevertheless, there is not a brightness variation model for echocardiographic images in these methods. Therefore, we are looking for a method to apply brightness variations locally in different regions of the image. In this study, we proposed a method to modify ausual OF technique by considering intensity variation. To evaluate this method, we implement two other OF-based methods, one usual OF method and a modified OF method applying brightness variation as a multiplier and an offset (generalized dynamic imaging model [GDIM]) and compare them with ours. These algorithms and ours were implemented on real 2D echocardiograms. Our method resulted in more accurate estimations than two others. At last, we compared our method with expert'spoint of view and observed that three distance metrics between them was appropriately smaller than other methods. The Haussdorff distance between the estimated curve defined by the proposed method and the expert defined curve is 4.81 pixels less than this distance for Lucas-Kanade and 2.28 pixels less than GDIM.

An energy model approach to people counting for abnormal crowd behavior detection

Abnormal crowd behavior detection plays an important role in surveillance applications. We propose a camera parameter independent and perspective distortion invariant approach to detect two types of abnormal crowd behavior. The two typical abnormal activities are people gathering and running. Since people counting is necessary for detecting the abnormal crowd behavior, we present an potential energy-based model to estimate the number of people in public scenes. Building histograms on the X- and Y-axes, respectively, we can obtain probability distribution of the foreground object and then define crowd entropy. We define the Crowd Distribution Index by combining the people counting results with crowd entropy to represent the spatial distribution of crowd. We set a threshold on Crowd Distribution Index to detect people gathering. To detect people running, the kinetic energy is determined by computation of optical flow and Crowd Distribution Index. With a threshold, kinetic energy can be used to detect people running. To test the performance of our algorithm, videos of different scenes and different crowd densities are used in the experiments. Without camera calibration and training data, our method can robustly detect abnormal behaviors with low computation load.

Locally Oriented Optical Flow Computation

This paper proposes the use of an adaptive locally oriented coordinate frame when calculating an optical flow field. The coordinate frame is aligned with the least curvature direction in a local window about each pixel. This has advantages to both fitting the flow field to the image data and in imposing smoothness constraints between neighboring pixels. In terms of fitting, robustness is obtained to a wider variety of image motions due to the extra invariance provided by the coordinate frame. Smoothness constraints are naturally propagated along image boundaries which often correspond to motion boundaries. In addition, moving objects can be efficiently segmented in the least curvature direction. We show experimentally the benefits of the method and demonstrate robustness to fast rotational motion, such as what often occurs in human motion.

Asynchronous frameless event-based optical flow

This paper introduces a process to compute optical flow using an asynchronous event-based retina at high speed and low computational load. A new generation of artificial vision sensors has now started to rely on biologically inspired designs for light acquisition. Biological retinas, and their artificial counterparts, are totally asynchronous and data driven and rely on a paradigm of light acquisition radically different from most of the currently used frame-grabber technologies. This paper introduces a framework for processing visual data using asynchronous event-based acquisition, providing a method for the evaluation of optical flow. The paper shows that current limitations of optical flow computation can be overcome by using event-based visual acquisition, where high data sparseness and high temporal resolution permit the computation of optical flow with micro-second accuracy and at very low computational cost.

Directionally Weakened Diffusion for Optical Flow

Optical flow estimation from image sequences is of paramount importance to computer vision applications. Many optical flow algorithms have been proposed for optical flow computation, which minimize a certain energy function involving a data term and a smoothness term. In this paper, we propose a new method to compute optical flow by decomposing the Laplacian operator along the tangential and gradient direction of the image. Experimental results show that the new method could gain more accurate estimation of optical flow around motion discontinuities compared with the classical methods.

Loveparade 2010: Automatic video analysis of a crowd disaster

On July 24, 2010, 21 people died and more than 500 were injured in a stampede at the Loveparade, a music festival, in Duisburg, Germany. Although this tragic incident is but one among many terrible crowd disasters that occur during pilgrimage, sports events, or other mass gatherings, it stands out for it has been well documented: there were a total of seven security cameras monitoring the Loveparade and the chain of events that led to disaster was meticulously reconstructed. In this paper, we present an automatic, video-based analysis of the events in Duisburg. While physical models and simulations of human crowd behavior have been reported before, to the best of our knowledge, automatic vision systems that detect congestions and dangerous crowd turbulences in real world settings were not reported yet. Derived from lessons learned from the video footage of the Loveparade, our system is able to detect motion patterns that characterize crowd behavior in stampedes. Based on our analysis, we propose methods for the detection and early warning of dangerous situations during mass events. Since our approach mainly relies on optical flow computations, it runs in real-time and preserves privacy of the people being monitored.

Mesoscale surface current fields in the Baltic Sea derived from multi-sensor satellite data

We demonstrate the use of multi-sensor satellite images for the computation of mesoscale surface currents in the Northern and Southern Baltic Proper by enhancing and combining image-processing techniques. The sequential satellite images were acquired by the Thematic Mapper (TM), the ERS-2 Synthetic Aperture Radar (SAR), the Wide-Field Scanner (WiFS) and the Sea-viewing Wide Field-of-view Sensor (SeaWiFS) during extensive cyanobacterial blooms in July 1997 and in July/August 1999. We also used a pair of Advanced SAR (ASAR) images from May 2005 showing imprints of singular oil spills in the Southern Baltic Proper. Different marine surface films and accumulated algae at the water surface were taken as tracers for the local motion of the sea surface. Data from sensors working at different electromagnetic bands (e.g. TM and SAR) were used to apply high-speed feature-matching (cross-correlation) techniques for motion detection. The sufficiently short time lags between the multiple image acquisitions (from less than 1 h to approximately 1 day) and the high spatial coverage allowed for the calculation of optical flow (i.e. surface motion) fields, which include small-scale turbulent structures that are not resolved by operational numerical models. Our computed surface currents range from 4 to 35 cm s−1 and are generally larger than those provided by the numerical models for the same dates and areas. We attribute this difference to local wind forcing, causing higher drift velocities at the very sea surface, which is seen from space, but which is not resolved by the numerical models.

Multiscale NeighborhoodWise Decision Fusion for Redundancy Detection in Image Pairs

To develop better image change detection algorithms, new models able to capture spatio-temporal regularities and geometries present in an image pair are needed. In this paper, we propose a multiscale formulation for modeling semilocal interimage interactions and detecting local or regional changes in an image pair. By introducing dissimilarity measures to compare patches and binary local decisions, we design collaborative decision rules that use the total number of detections obtained from the neighboring pixels for different patch sizes. We study the statistical properties of the nonparametric detection approach that guarantees small probabilities of false alarms. Experimental results on several applications demonstrate that the detection algorithm (with no optical flow computation) performs well at detecting occlusions and meaningful changes for a variety of illumination conditions and signal-to-noise ratios. The number of control parameters of the algorithm is small, and the adjustment is intuitive in ...

Warped Optical-Flow Inter-Frame Reconstruction for Ultrasound Image Enhancement

Problem statement: Optical flow inter-frame enhancement is one of the techniques to improve the quality of ultrasound images by reducing the speckle noise. The performance depends on accuracy of the optical flow inter-frame reconstruction which is a part of the technique. The speckle noise of an ultrasound image forms certain pattern, of which size of the speckle varies depends on the distance from the transducer. Approach: This study proposed that the use of warping technique prior to as well as after the optical flow to improve the accuracy of the optical flow calculation. The warping function was derived to transform the image from circular grid to rectangular grid. Results: The technique was applied to a series of synthetic moving frames generated from actual ultrasound image taken from a patient heart. The technique improved the accuracy of motion vectors generated from the optical flow. The MSE of the reconstructed frame was also smaller using warped technique compared to that of the non-warped technique. Conclusion: The proposed warping operation was shown to be successful in increasing the accuracy of the frame reconstruction for the optical flow inter-frame enhancement technique.

Bootstrap optical flow confidence and uncertainty measure

We address the problem of estimating the uncertainty of optical flow algorithm results. Our method estimates the error magnitude at all points in the image. It can be used as a confidence measure. It is based on bootstrap resampling, which is a computational statistical inference technique based on repeating the optical flow calculation several times for different randomly chosen subsets of pixel contributions. As few as ten repetitions are enough to obtain useful estimates of geometrical and angular errors. For demonstration, we use the combined local–global optical flow method (CLG) which generalizes both Lucas–Kanade and Horn–Schunck type methods. However, the bootstrap method is very general and can be applied to almost any optical flow algorithm that can be formulated as a pixel-based minimization problem. We show experimentally on synthetic as well as real video sequences with known ground truth that the bootstrap method performs better than all other confidence measures tested.

Dense versus Sparse Approaches for Estimating the Fundamental Matrix

There are two main strategies for solving correspondence problems in computer vision: sparse local feature based approaches and dense global energy based methods. While sparse feature based methods are often used for estimating the fundamental matrix by matching a small set of sophistically optimised interest points, dense energy based methods mark the state of the art in optical flow computation. The goal of our paper is to show that this separation into different application domains is unnecessary and can be bridged in a natural way. As a first contribution we present a new application of dense optical flow for estimating the fundamental matrix. Comparing our results with those obtained by feature based techniques we identify cases in which dense methods have advantages over sparse approaches. Motivated by these promising results we propose, as a second contribution, a new variational model that recovers the fundamental matrix and the optical flow simultaneously as the minimisers of a single energy functional. In experiments we show that our coupled approach is able to further improve the estimates of both the fundamental matrix and the optical flow. Our results prove that dense variational methods can be a serious alternative even in classical application domains of sparse feature based approaches.