An Improved Fractional-Order Variational Optical Flow Model Combining Structure Tensor

Bin Zhu,Jameel Bhutto,Jinmei Guo,Zhaodong Wang,Lianfang Tian,Lingjian Wang,Erik Cuevas

doi:10.1155/2021/9960784

Abstract

Dealing with problems of illumination changes in optical flow estimation, an improved variational optical flow model is proposed in this paper. The local structure constancy constraint (LSCC) is applied in the data term of the traditional HS (Horn & Schunck) optical flow model to substitute the brightness constancy constraint. The fractional-order smoothness constraint (FSC) is applied in the smoothness term of the HS model. Then, the detailed calculation processes from the optical flow model to the optical flow value are explained. The structure tensor in LSCC is an image feature that is constant in the illumination changes scene. The fractional differential coefficient in FSC can fuse the local neighborhood optical flow vector into the optical flow vector of the target pixel, which can improve the integrity of the motion region with the same motion speed. Combining LSCC with FSC, our improved optical flow model can obtain an accurate optical flow field with clear outline in the illumination abnormity scene. The experimental results show that, compared with other optical flow models, our model is more suitable for the illumination changes scene and can be employed in outdoor motion detection projects.

Highlights

Motion detection in the image sequence or video is one of the basic tasks of various image processing projects, which has been widely used in the research and engineering practice of 3D reconstruction, moving object segmentation, moving object tracking, video compression, automatic driving, and so on. e variational optical flow method is one of the most commonly used methods of motion detection. e HS optical flow model [1] is the most classical variational optical flow model
It is composed of a brightness constancy constraint and global smoothness constraint. e brightness constancy constraint requires that the intensity of pixels remains constant in the process of motion, while the smoothness constraint supposes that the motion speed of all pixels in the image changes smoothly
Fractional Smoothness Constraint Equation e fractional smoothness constraint (FSC) is applied in our model to further improve the performance of optical flow estimation

Summary

Introduction

Motion detection in the image sequence or video is one of the basic tasks of various image processing projects, which has been widely used in the research and engineering practice of 3D reconstruction, moving object segmentation, moving object tracking, video compression, automatic driving, and so on. e variational optical flow method is one of the most commonly used methods of motion detection. e HS optical flow model [1] is the most classical variational optical flow model. Several other constraints on the data term were proposed to improve the performance of the variational optical flow model, such as gradient constancy constraint [2], Laplacian constancy constraint [3], and Hessian constancy constraint [4], but these constraints depend heavily on the intensity difference Illumination invariant descriptors such as binarybased [5], real value-based [6, 7], and neighborhood-based [8, 9] were applied to substitute the brightness constancy constraint. E model proposed in this paper combines the advantages of the two constraints and solves the illumination change problem and the motion discontinuity problem in optical flow estimation, and different motion parts in a structure can be identified. PWC-Net [32], which consists of pyramidal processing, warping, and a cost volume, is designed to estimate optical flow. e network uses the current optical flow estimate to warp the CNN features of the second image and uses the warped features and features of the first image to construct a cost volume. e drawback of deep learning-based methods is large number of ground truth are needed, and retraining the model is needed for different application scenarios

HS Optical Flow Model e HS model can be written as follows

Improved Fractional Optical Flow Model

Experimental Result and Analysis

Findings

Conclusion