Spatio-Temporal Image Pattern Prediction Method Based on a Physical Model With Time-Varying Optical Flow

Abstract

This paper proposes an image-based prediction method that can physically predict near-future spatio-temporal image changes using fluid-like image sequences, i.e., dynamic texture, from different image sources such as ground-based radar imagers, satellite sensors, and lightning detectors. Previous alternatives, i.e., tracking radar echo by correlation or thunderstorm identification, tracking, analysis, and nowcasting, employ pattern matching or linear extrapolation of the centroid of an image object to predict the next time image with many tuning model parameters. However, such methods fail to handle the high degree of motion and deformation of fluid-like images, i.e., vortex. To remedy this issue, this paper presents a spatio-temporal prediction method based on a computer vision framework; it employs a physics-based model with time-variant optical flow. Initial local motions from image sequences are estimated by the extended optical flow method, where a locally optimal weighting parameter and a statistically robust function are applied to Horn and Schunck's model. The next time image sequence from the past image sequence is physically predicted by the extended advection equation for image intensities and the Navier–Stokes equation with a continuity equation for varying optical flow over time. For different source images, our method offers no prior knowledge of size, shape, texture, and motion of moving objects. Experiments demonstrate that the proposed prediction method outperforms a previous prediction method with respect to prediction accuracy.