Joint face alignment and segmentation via deep multi-task learning

Abstract

Face alignment and segmentation are challenging problems which have been extensively studied in the field of multimedia. These two tasks are closely related and their learning processes are supposed to benefit each other. Hence, we present a joint multi-task learning algorithm for both face alignment and segmentation using deep convolutional neural network (CNN). The proposed multi-task learning approach allows CNN model to simultaneously share visual knowledge between different tasks. With a carefully designed refinement residual module, the cross-layer features are fused in a collaborative manner. To the best of our knowledge, this is the first time that face alignment and segmentation are learned together via deep multi-task learning. Our experiments show that learning these two related tasks simultaneously builds a synergy between them, improves the performance of each individual task, and rivals recent approaches. Furthermore, we demonstrate the effectiveness of our model in two practical applications: virtual makeup and face swap.