The estimation of velocity vector fields from time-varying image sequences

Abstract

Changes in successive images from a time-varying image sequence of a scene can be characterized by velocity vector fields. The estimate of the velocity vector field is determined as a compromise between optical flow and directional smoothness constraints. The optical flow constraints relate the values of the time-varying image function at the corresponding points of the successive images of the sequence. The directional smoothness constraints relate the values of neighboring velocity vectors. To achieve the compromise, we introduce a system of nonlinear equations of the unknown estimate of the velocity vector field using a novel variational principle applied to the weighted average of the optical flow and the directional smoothness constraints. A stable iterative method for solving this system is developed. The optical flow and the directional smoothness constraints are selectively suppressed in the neighborhoods of the occluding boundaries by implicitly adjusting their weights. These adjustments are based on the spatial variations of the estimates of the velocity vectors and the spatial variations of the time-varying image function. The system of nonlinear equations is defined in terms of the time-varying image function and its derivatives. The initial image functions are in general discontinuous and cannot be directly differentiated. These difficulties are overcome by treating the initial image functions as generalized functions and their derivatives as generalized derivatives. These generalized functions are evaluated (observed) on the parametric family of testing (smoothing) functions to obtain parametric families of secondary images, which are used in the system of nonlinear equations. The parameter specifies the degree of smoothness of each secondary image. The secondary images with progressively higher degrees of smoothness are sampled with progressively lower resolutions. Then coarse-to-fine control strategies are used to obtain the estimate.