Video colorization based on optical flow and edge-oriented color propagation

Abstract

ABSTRACT We propose a novel video colorization method based on sparse optical ow and edge-oriented color propagation.Colorization is a process of adding color to monochrome images or videos. In our video colorization method, it isassumed that key frames are appropriately selected out of a grayscale video stream and are properly colorized inadvance. Once key frames are colorized, our method colorizes all the remaining grayscale frames automatically.It is also possible to colorize key frames semi-automatically by our method. For colorizing a grayscale framebetween a pair of colorized key frames, sparse optical ow is computed rst. The optical ow consists of reliablemotion vectors around strong feature points. The colors around feature points of key frames are then copied tothe grayscale frame according to the estimated motion vectors. Those colors are then propagated to the restof the grayscale frame. Colors are blended appropriately during the propagation process. A pair of accuracyand priority measures is introduced to control how the color propagation proceeds. To successfully propagatecolors, it is important not to wrongly spread colors across edges. For this purpose, a set of neighboring pixelsis adaptively selected not to include edge-like areas and thus not to spread colors across edges. To evaluatee ectiveness of our method, image colorization and video colorization were performed. Experimental resultsshow that our method can colorize images and videos better than previous methods when there are edges. Wealso show that the proposed method enables us to easily modify colors in colored video streams.Keywords: colorization, video colorization, color propagation