Robust Image Chroma-Keying: A Quadmap Approach Based on Global Sampling and Local Affinity

Abstract

Chroma-keying is a technique used to replace solid-colored background of images or video frames. This technique is widely used in TV broadcasting, film production, augmented reality, and virtual environment. This paper proposes a new chroma-keying method, which can automatically remove the background color in an image and accurately segment the foreground objects along with their transparency property. Compared to conventional chroma-keying methods based on color clustering, color difference, or thresholding, the proposed method analyzes the color statistics and color confidence of the image in global range. By analyzing image color statistics, local lightness variation, and experience from human visual perception in HSV color space, a segmentation map called quadmap is automatically generated to segment the image into four types of regions: 1) foreground; 2) background; 3) transparent; and 4) reflective regions. By using quadmap, the proposed chroma-keying system can differentiate between transparent and reflective regions. This has always been a challenging problem in conventional chroma-keying or $\alpha $ matting systems. This improvement generates more reliable foreground colors in reflective regions. As a result, there can be less constrains for foreground scene used in TV-broadcasting or film making. The procedures of the proposed method consist of the following five steps: 1) background region detection based on color statistics and local lightness variation; 2) absolute foreground region detection based on the knowledge of background color and predefined thresholds in Hue channel of HSV color space; 3) reflective region detection based on experience from human visual system; 4) background color propagation based on Laplacian equation; and 5) transparency value and foreground color estimation based on global sampling and color confidence.