A video object segmentation algorithm based on the feature learning and shape tracking

Abstract

This paper proposes a video object segmentation algorithm based on the conditional random field (CRF) framework. A foreground object in the first frame is segmented by training the CRF on user interaction, i.e., by using user scribbles corresponding to foreground and background respectively for CRF training. The data term of the energy function in this CRF framework is designed as a function of the score of texture-color classifier trained by AdaBoost. From the second frame, a weighted data term that encodes the shape of the object is added to this energy function. The boundary pixels of the current frame are predicted by the optical flow, and a smaller cost is given to a pixel closer to the boundary and vice versa. Also, a confidence of optical flow is defined, and a larger weight is given to the data term when the confident is high. As a result, the data term related with the shape becomes important when the motion estimation is reliable, and conversely the color-texture term becomes important otherwise. Experimental results show that the proposed data term keeps the boundary correctly in most cases and provides comparable result when compared to a state-of-the-art method.