Combining static and dynamic features using neural networks and edge fusion for video object extraction

Abstract

Semantic object representation is an important step for digital multimedia applications such as object-based coding, content-based access and manipulations. The authors propose an image sequence segmentation scheme which provides region information for the semantic object representation of those applications. The objective is to develop a hardware-friendly segmentation algorithm by combining static and dynamic features simultaneously in one scheme. In the initial stage, a multiple feature space is transformed to one-dimensional label space by using self-organising feature map (SOFM) neural networks. The next stage is an edge fusion process in which edge information is incorporated into the neural network outputs to generate more precisely located boundaries of segmentation. The proposed algorithm differs from existing methods as follows: it can segment textured images with low-dimensional features; leads to more meaningful segmentation region boundaries; and is easier to map into hardware than existing methods. Experimental results are compared with an existing segmentation method using evaluation metrics to clarify the advantages of the proposed algorithm objectively.