Image matting from a physical perspective

Abstract

Image and video matting is used to extract objects from their original backgrounds in order to place them on a different background. The traditional matting model is a combination of foreground and background colors, i.e., I = αF + (1 − α)B. Even with good cameras, limited depth-of-focus means that often both the object and background are blurred at the boundaries. Does the matting model still apply? To understand this and other cases better, we investigate image matting from a physical perspective. We start with 3D objects and examine the mechanism of image matting due to geometrical defocus. We then derive a general matting model using radiometry and geometric optics. The model accounts for arbitrary surface shapes, defocus, transparency, and directional radiance. Under certain conditions, the physical framework subsumes the traditional matting equation. The new formulation reveals a fundamental link between parameter α and object depth, and this establishes a framework for designing new matting algorithms.