Face De-identification Method Research Articles

Examining StyleGAN as a Utility-Preserving Face De-identification Method

Several face de-identification methods have been proposed to preserve users’ privacy by obscuring their faces. These methods, however, can degrade the quality of photos, and they usually do not preserve the utility of faces, i.e., their age, gender, pose, and facial expression. Recently, advanced generative adversarial network models, such as StyleGAN [ 33], have been proposed, which generate realistic, high-quality imaginary faces. In this paper, we investigate the use of StyleGAN in generating de-identified faces through style mixing, where the styles or features of the target face and an auxiliary face get mixed to generate a de-identified face that carries the utilities of the target face. We examined this de-identification method for preserving utility and privacy by implementing several face detection, verification, and identification attacks and conducting a user study. The results from our extensive experiments, human evaluation, and comparison with two state-of-the-art face de-identification methods, i.e., CIAGAN and DeepPrivacy, show that StyleGAN performs on par or better than these methods, preserving users’ privacy and images’ utility. In particular, the results of the machine learning-based experiments show that StyleGAN0-4 preserves utility better than CIAGAN and DeepPrivacy while preserving privacy at the same level. StyleGAN 0-3 preserves utility at the same level while providing more privacy. In this paper, for the first time, we also performed a carefully designed user study to examine both privacy and utility-preserving properties of StyleGAN 0-3, 0-4, and 0-5, as well as CIAGAN and DeepPrivacy from the human observers’ perspectives. Our statistical tests showed that participants tend to verify and identify StyleGAN 0-5 images easier than DeepPrivacy images. All the methods but StyleGAN 0-5 had significantly lower identification rates than CIAGAN. Regarding utility, as expected, StyleGAN 0-5 performed significantly better in preserving some attributes. Among all methods, on average, participants believe gender has been preserved the most while naturalness has been preserved the least.

The UU-Net: Reversible Face De-Identification for Visual Surveillance Video Footage

We propose a reversible face de-identification method for video surveillance data, where landmark-based techniques cannot be reliably used. Our solution generates a photorealistic de-identified stream that meets the data protection regulations and can be publicly released under minimal privacy concerns. Notably, such stream still encapsulates the information required to later reconstruct the original scene, which is useful for scenarios, such as crime investigation, where subjects identification is of most importance. Our learning process jointly optimizes two main components: 1) a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">public</i> module, that receives the raw data and generates the de-identified stream; and 2) a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">private</i> module, designed for security authorities, that receives the public stream and reconstructs the original data, disclosing the actual IDs of the subjects in a scene. The proposed solution is landmarks-free and uses a conditional generative adversarial network to obtain synthetic faces that preserve pose, lighting, background information and even facial expressions. Also, we keep full control over the set of soft facial attributes to be preserved/changed between the raw/de-identified data, which extends the range of applications for the proposed solution. Our experiments were conducted in three visual surveillance datasets (BIODI, MARS and P-DESTRE) plus one video face data set (YouTube Faces), showing highly encouraging results. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hugomcp/uu-net</uri> .

SF-GAN: Face De-Identification Method Without Losing Facial Attribute Information

This paper discusses the face de-identification without losing facial attribute information and provides a solution called SF-GAN (Secret Face Generative Adversarial Network). The proposed model aims to realize face de-identification effectively and generate visually reasonable images while retaining the facial attribute information of original images, such as facial expression, gender, hairstyle and wearing glasses or not, as much as possible. To optimize the face de-identification, we construct a variety of external mechanisms to balance the influence of multiple factors on the effect of face de-identification. The SF-GAN differs from the face replacement and face swapping because the later two fails actually in the privacy protection. As for the multi-attribute retention, we use the shallow face attribute information and deep face attribute information, and adopt different processing strategies for different face attribute information based on the uniqueness of each kind of face attribute information. Finally, we train and test the model on two high-definition datasets Celeba-HQ and FFHD. Compared with existing face de-identification methods, the proposed model proved to perform better in protecting the facial privacy.

Privacy preservation through facial de-identification with simultaneous emotion preservation

Due to the availability of low-cost internet and other data transmission media, a high volume of multimedia data get shared very quickly. Often, the identity of individuals gets revealed through images or videos without their consent, which affects their privacy. Since face is the only biometric feature that reveals the most identifiable characteristics of a person in an image or a video frame, the need for the development of an effective face de-identification algorithm for privacy preservation cannot be over-emphasized. Existing solutions to face de-identification are either non-formal or are unable to obfuscate identifiable features completely. In this paper, we propose an automated face de-identification algorithm that takes as input a facial image and generates a new face that preserves the emotion and non-biometric facial attributes of a target face. We consider a proxy set of a large collection of artificial faces generated by StyleGAN and select the most appropriate face from the proxy set that has a facial expression and pose similar to that of the target face. The faces in the proxy set are artificially generated, and hence the face selected from this set is completely anonymous. To retain the non-biometric attributes of the target face, we employ a generative adversarial network (GAN) with a suitable loss function that fuses the non-biometric attributes of the target face with the face selected from the proxy set to obtain the final de-identified face. Experimental results emphasize the superiority of our approach over state-of-the-art face de-identification methods.

Disentangled Representation Learning for Multiple Attributes Preserving Face Deidentification.

Face is one of the most attractive sensitive information in visual shared data. It is an urgent task to design an effective face deidentification method to achieve a balance between facial privacy protection and data utilities when sharing data. Most of the previous methods for face deidentification rely on attribute supervision to preserve a certain kind of identity-independent utility but lose the other identity-independent data utilities. In this article, we mainly propose a novel disentangled representation learning architecture for multiple attributes preserving face deidentification called replacing and restoring variational autoencoders (R2VAEs). The R2VAEs disentangle the identity-related factors and the identity-independent factors so that the identity-related information can be obfuscated, while they do not change the identity-independent attribute information. Moreover, to improve the details of the facial region and make the deidentified face blends into the image scene seamlessly, the image inpainting network is employed to fill in the original facial region by using the deidentified face as a priori. Experimental results demonstrate that the proposed method effectively deidentifies face while maximizing the preservation of the identity-independent information, which ensures the semantic integrity and visual quality of shared images.

FPGAN: Face de-identification method with generative adversarial networks for social robots

In this paper, we propose a new face de-identification method based on generative adversarial network (GAN) to protect visual facial privacy, which is an end-to-end method (herein, FPGAN). First, we propose FPGAN and mathematically prove its convergence. Then, a generator with an improved U-Net is used to enhance the quality of the generated image, and two discriminators with a seven-layer network architecture are designed to strengthen the feature extraction ability of FPGAN. Subsequently, we propose the pixel loss, content loss, adversarial loss functions and optimization strategy to guarantee the performance of FPGAN. In our experiments, we applied FPGAN to face de-identification in social robots and analyzed the related conditions that could affect the model. Moreover, we proposed a new face de-identification evaluation protocol to check the performance of the model. This protocol can be used for the evaluation of face de-identification and privacy protection. Finally, we tested our model and four other methods on the CelebA, MORPH, RaFD, and FBDe datasets. The results of the experiments show that FPGAN outperforms the baseline methods.

CLEANIR: Controllable Attribute-Preserving Natural Identity Remover

We live in an era of privacy concerns. As smart devices such as smartphones, service robots and surveillance cameras spread, preservation of our privacy becomes one of the major concerns in our daily life. Traditionally, the problem was resolved by simple approaches such as image masking or blurring. While these provide effective ways to remove identities from images, there are certain limitations when it comes to a matter of recognition from the processed images. For example, one may want to get ambient information from scenes even when privacy-related information such as facial appearance is removed or changed. To address the issue, our goal in this paper is not only to modify identity from faces but also keeps facial attributes such as color, pose and facial expression for further applications. We propose a novel face de-identification method based on a deep generative model in which we design the output vector from an encoder to be disentangled into two parts: identity-related part and the rest representing facial attributes. We show that by solely modifying the identity-related part from the latent vector, our method effectively modifies the facial identity to a completely new one while the other attributes that are loosely related to personal identity are preserved. To validate the proposed method, we provide results from experiments that measure two different aspects: effectiveness of personal identity modification and facial attribute preservation.

Defeating face de-identification methods based on DCT-block scrambling

Face de-identification aims at preserving the privacy of people by concealing faces in images and videos. In this paper, we propose a defeating algorithm for face de-identification methods that are based on DCT-block scrambling. These methods protect faces by scrambling the AC and DC coefficients of the DCT blocks corresponding to a face region in the compressed domain. The proposed approach does not make use of the protection key utilized in the de-identification process. It consists of the following stages. First, random unprotected faces are generated based on a random alteration of the sign of AC coefficients with a fixed value of DC coefficients. Then, the best unprotected faces are selected by an eigenfaces model trained with facial images from a repository of potentially protected people. A single facial image is then generated by merging the selected images through median stacking. Finally, the eigenfaces model is utilized again to choose the face from the repository that is closest to the resulting image in order to improve the aspect of the unprotected face. Experimental results using a proprietary database and the public CALTECH, Utrecht and LFW face databases show the effectiveness of the proposed technique.