Real-time frame-straddling-based optical flow detection

Abstract

This paper proposes a novel conception of realtime optical flow detection for high-speed moving objects based on high-frame-rate (HFR) video; it introduces a frame-straddling function to select only a pair of images with a small interval for optical flow estimation from an HFR image sequence even when the output of the estimated optical flow is required as fast as video-rate images (NTSC 30 fps/PAL 25 fps). This frame-straddling function can remarkably improve the upper limit of measurable velocity in optical flow estimation without heavy computation, because it can avoid large image displacements between frames that often create serious errors in optical flow estimation. Our method was implemented using software on a high-speed vision platform, IDP Express. This method can execute real-time video capturing and processing of 512×512-pixel images at 2000 fps. In our implementation, optical flows can be outputted at an interval of 40 ms in real time by selecting a pair of 512×512-pixel images with a frame-straddling interval of 0.5 ms from a 2000 fps image sequence. Compared to optical flows estimated for 25 fps videos with the Lucas-Kanade method, the upper limit of measurable velocity in optical flow estimation is remarkably improved, because the frame-straddling interval in optical flow estimation was 80 times smaller than its output interval of 40 ms. Several experiments were performed for rapidly moving objects to verify the performance of our optical flow method.