3‐D shape reconstruction of human face from a 2‐d facial image and provision for change of expression

Abstract

AbstractVarious studies are presently being made on the facial image (e.g., highly efficient coding and the discrimination of faces) for computer animation and TV conference systems. This paper describes a method in which the three‐dimensional (3‐D) shape of the face is estimated from a single 2‐D facial image, and the expression or the direction of the face can be modified. The original 2‐D facial image need not be taken from the front, and the image from any direction can be used. The curved surface model, not the usual polyhedron model, is employed as the 3‐D shape model.The surface model generates the curved surface passing through the feature points of the face (e.g., the eyes and both ends of the mouth) and produces the 3‐D shape of the face. Compared to the conventional method, in the surface model it suffices to specify only a small number of feature points on the facial image. The 3‐D shape of the face can be estimated with sufficient accuracy for image generation, and the expression can be varied in a flexible way. Using this method, if only a single picture or image of a face is available, dynamic images with varying expressions and directions can be generated. An application, for example, is a broadcast program in which a historical figure changes expression or speaks. Another basic application will be to realize the intelligent image coding, which is of recent concern.