Displacement field estimation and image segmentation using block matching enhanced by a neural network.

Abstract

The block-matching method plays an important role in displacement field estimation due to its simplicity, achievement of long-range motion, and robustness to noise. In this paper, a single-layer feedback neural network model is proposed that enhances block matching, estimates the displacement field, and simultaneously performs image segmentation from consecutive images. In this paper, image segmentation is defined as partitioning each image into a set of moving objects and the background. For any two consecutive images, a neural network is created that learns the connection relationship of the pixels in an object from the displacement field and stores the relationship in the network. A modified block matching is used to compute a more accurate displacement field by utilizing the segmentation information embedded in the neural network. The displacement vector at the edge of an object or occluding boundary is hard to estimate, but the proposed model performs satisfactorily because it learns and uses the connection information. Furthermore, a flood-fill algorithm is used to compute the dense displacement field more efficiently and correctly than the exhaustive search does. The most important aspect of this paper is that image segmentation is performed simultaneously with the displacement-field estimation by the neural-network model. The novel idea of the work is to embed the segmentation information (connection relations) in the neural network and to perform the displacement-field estimation and image segmentation simultaneously. Two methods for retrieving segmentation information from the neural network with any two consecutive images are also presented.