Affine motion estimation using a neural network

Abstract

A neural network based motion-estimation technique (for successive video frames) is developed that is applicable to purely translational as well as affine movements. In addition to yielding highly accurate results, a benefit of this technique is the elimination of fractional-pixel interpolation and corresponding search. The technique is formulated in terms of a modified Hopfield neural network. The procedure consists of two stages: estimation of the neural network parameters from the present and previous frames, followed by the estimation of the affinity weights for the candidate macroblocks. The reconstructed macroblock then consists of the sum of the candidate macroblocks weighted by their affinity values as defined. Experiments on translational motion indicate that this technique produces a 5 dB higher peak signal to noise ratio (PSNR) than the full search algorithm, 6 dB higher than the logarithmic block matching algorithms, and 2 dB higher than some of the very recent motion estimation algorithms. Preliminary experiments using the neural network affine algorithm on synthetically sheared images produce an 11 dB higher PSNR than the full search and LBM. Further, due to the neural network's fault tolerant nature and inherent parallelism, potential exists for VLSI implementation as an interconnection of small cells on a single chip.