Original Face Research Articles

Facial expression morphing: enhancing visual fidelity and preserving facial details in CycleGAN-based expression synthesis

Recent advancements in facial expression synthesis using deep learning, particularly with Cycle-Consistent Adversarial Networks (CycleGAN), have led to impressive results. However, a critical challenge persists: the generated expressions often lack the sharpness and fine details of the original face, such as freckles, moles, or birthmarks. To address this issue, we introduce the Facial Expression Morphing (FEM) algorithm, a novel post-processing method designed to enhance the visual fidelity of CycleGAN-based outputs. The FEM method blends the input image with the generated expression, prioritizing the preservation of crucial facial details. We experimented with our method on the Radboud Faces Database (RafD) and evaluated employing the Fréchet Inception Distance (FID) standard benchmark for image-to-image translation and introducing a new metric, FSD (Facial Similarity Distance), to specifically measure the similarity between translated and real images. Our comprehensive analysis of CycleGAN, UNet Vision Transformer cycle-consistent GAN versions 1 (UVCGANv1) and 2 (UVCGANv2) reveals a substantial enhancement in image clarity and preservation of intricate details. The average FID score of 31.92 achieved by our models represents a remarkable 50% reduction compared to the previous state-of-the-art model’s score of 63.82, showcasing the significant advancements made in this domain. This substantial enhancement in image quality is further supported by our proposed FSD metric, which shows a closer resemblance between FEM-processed images and the original faces.

Diversity-driven Privacy Protection Masks Against Unauthorized Face Recognition

Face recognition (FR) technologies have enabled many life-enriching applications but have also opened doors for potential misuse. Governments, private companies, or even individuals can scrape the web, collect facial images, and build a face database to fuel the FR system to identify human faces without their consent. This paper introduces PMask to combat such a privacy threat against unauthorized FR. It provides a holistic approach to enable privacy-preserving sharing of facial images. PMask preprocesses the facial image and hides its unique facial signature through iterative optimization with dual goals: (i) minimizing the amount of noise to ensure high image quality and (ii) minimizing the perception loss between the privacy-protected face and the original face to ensure the face is recognizable to be the same person by humans. Extensive experiments are conducted on eight representative FR models to evaluate PMask against unauthorized FR. The results validate that PMask provides much stronger protection, introduces less perceptible changes to facial images, and runs faster than state-of-the-art methods to provide privacy protection with a better user experience.

Improved Face Image Authentication Scheme based on Embedding in Adjacent Coefficients

Face image authentication (FIA) schemes have recently been developed using face detection and image watermarking technology. The research in this direction proved the presented schemes' efficiency in accurately detecting the manipulated face regions and recovering the original face region. Recovering the original face region is very important in practical applications. Still, it was at the cost of increasing the secret data that must be embedded in the face image. The increment in the secret data required a large embedding capacity, which was not available in some images. To overcome this limitation, an improved FIA scheme based on a new data embedding algorithm is presented in this paper. The suggested FIA scheme consists of two main algorithms applied at the sender and receiver sides, where both start by detecting the face region and dividing and classifying the image into blocks that belong to the face region or outside the face region. At the sender side, the secret data are generated from the face region and embedded in the blocks outside the face region using the suggested algorithm called Embedding in Adjacent Coefficients (EAC) for three subbands obtained after applying the Slantlet transform of the blocks. On the receiver side, the secret data are extracted from the blocks outside the face region using the suggested algorithm called Extraction from Adjacent Coefficients (ExAC). The extracted data is used to authenticate the face region and recover the original one when manipulations occur. The proposed FIA scheme obtained higher embedding capacity than previous ones, making it applicable to protect more face images that could not be protected using previous FIA schemes.

Toward identity preserving in face sketch-photo synthesis using a hybrid CNN-Mamba framework

The synthesis of facial sketch-photo has important applications in practical life, such as crime investigation. Many convolutional neural networks (CNNs) based methods have been proposed to address this issue. However, due to the substantial modal differences between sketch and photo, the CNN’s insensitivity to global information, and insufficient utilization of hierarchical features, synthesized photos struggle to balance both identity preservation and image quality. Recently, State Space Sequence Models (SSMs) have achieved exciting results in computer vision (CV) tasks. Inspired by SSMs, we design a hybrid CNN–SSM model called FaceMamba for the Face Sketch-Photo Synthesis (FSPS) task. It includes an original Face Vision Mamba Attention for modeling in latent space using SSM. Additionally, it incorporates a general auxiliary method called Attention Feature Injection that combines encoding features, decoding features, and external auxiliary features using attention mechanisms. FaceMamba combines Mamba’s modeling ability for long-range dependencies with CNN’s powerful local feature extraction ability, and utilizes hierarchical features at the appropriate position. Adequate experimental and evaluation results reveal that FaceMamba has strong competitiveness in FSPS task, achieving the best balance between identity preservation and image quality.

Face swapping in seizure videos for patient deidentification

ObjectiveThis study aimed to test different AI-based face-swapping models applied to videos of epileptic seizures, with the goal of protecting patient privacy while retaining clinically useful seizure semiology. We hypothesized that specific models would show differences in semiologic fidelity compared to the original clinical videos. MethodsThree open-source models, SimSwap, MobileFaceSwap and GHOST were adopted for face-swapping. For every model, an AI generated male and female image were used to replace the original faces. One representative seizure per patient from three patients with epilepsy was chosen (3 seizure videos x 3 AI models x 2 M/F swap) and remade to 18 transformed video clips. To evaluate the performance of the three models, we used both objective (AI-based) and subjective (expert clinician) evaluation. The objective assessment included four metrics for facial appearance and four metrics for facial expression changes. Four experienced epileptologists reviewed the clips and scoring according to deidentification and preservation of semiology. Kruskal-Wallis H test was used for statistical analysis among the models. ResultsIn the reproduced videos, the swapped face cannot be recognized as the original face, with no significant difference in scores of deidentification either by objective or subjective assessment. Regarding semiology preservation, no significant differences between models were observed in the objective evaluations. The subjective evaluations revealed that the GHOST model outperformed the other two models (p=0.028). ConclusionThis is the first study evaluating AI face swapping models in epileptic seizure video clips. Optimization of AI face-swapping models could enhance the accessibility of seizure videos for education and research while protecting patient privacy and maintaining semiology.

Quaternion-based 2D-DOST and stacked principal component analysis network for multimodal face recognition

Face recognition is widely utilized and has become a ubiquitous part of daily lives. However, original face images are often sensitive and unauthorized access could imperil personal information. To protect privacy and security of multimodal face data, we propose an original approach that integrates differential privacy and a lightweight convolutional neural network with unsupervised predefined quaternion-type filters. Firstly, different modalities of face images are banded together by means of full quaternion matrix representation for simultaneously processing. Then the proposed quaternion two-dimensional discrete orthogonal Stockwell transform is exploited for implementing difference privacy, which is disturbed by virtue of Laplace and exponential mechanism. Afterwards, we devise a stacked quaternion two-dimensional principal component analysis network with three-stages for learning representative features. The proposed network model can enhance the discriminative characteristics through inserting nonlinearity, together with union of feature maps in different levels to achieve information complementation. Experiments conducted on four multimodal face datasets unveil that the proposed method achieves superior recognition performance in comparison with other state-of-the-art approaches.

Design and Additive Manufacturing of Medical Face Shield for Healthcare Workers Battling Coronavirus (COVID-19).

During the coronavirus disease-19 pandemic, the demand for specific medical equipment such as personal protective equipment has rapidly exceeded the available supply around the world. Specifically, simple medical equipment such as medical gloves, aprons, goggles, surgery masks, and medical face shields have become highly in demand in the health-care sector in the face of this rapidly developing pandemic. This difficult period strengthens the social solidarity to an extent parallel to the escalation of this pandemic. Education and government institutions, commercial and noncommercial organizations and individual homemakers have produced specific medical equipment by means of additive manufacturing (AM) technology, which is the fastest way to create a product, providing their support for urgent demands within the health-care services. Medical face shields have become a popular item to produce, and many design variations and prototypes have been forthcoming. Although AM technology can be used to produce several types of noncommercial equipment, this rapid manufacturing approach is limited by its longer production time as compared to conventional serial/mass production and the high demand. However, most of the individual designer/maker-based face shields are designed with little appreciation of clinical needs and nonergonomic. They also lack of professional product design and are not designed according to AM (Design for AM [DfAM]) principles. Consequently, the production time of up to 4 – 5 h for some products of these designs is needed. Therefore, a lighter, more ergonomic, single frame medical face shield without extra components to assemble would be useful, especially for individual designers/makers and noncommercial producers to increase productivity in a shorter timeframe. In this study, a medical face shield that is competitively lighter, relatively more ergonomic, easy to use, and can be assembled without extra components (such as elastic bands, softening materials, and clips) was designed. The face shield was produced by AM with a relatively shorter production time. Subsequently, finite element analysis-based structural design verification was performed, and a three-dimensional (3D) prototype was produced by an original equipment manufacturer 3D printer (Fused Deposition Modeling). This study demonstrated that an original face shield design with <10 g material usage per single frame was produced in under 45 min of fabrication time. This research also provides a useful product DfAM of simple medical equipment such as face shields through advanced engineering design, simulation, and AM applications as an essential approach to battling coronavirus-like viral pandemics.

Advanced deep learning for masked individual surveillance

During Covid-19 pandemic, face masks have become a ubiquitous protective measure. This poses new challenges for surveillance systems that heavily rely on facial recognition. To address this critical issue, we present a novel enhanced surveillance system that leverages deep learning techniques to tackle two crucial tasks simultaneously: anomaly detection of masked individuals' activities and masked face completion for accurate recognition. For anomaly detection, we employ a custom-designed deep neural network capable of processing real-time video streams. Finding a dataset of anomaly events of masked individuals is a big challenge for us. We handle this challenge using efficient techniques such as Dlib library and other image processing techniques. The network is trained on a diverse dataset encompassing normal and abnormal activities of masked individuals, enabling it to identify suspicious behaviors effectively. The surveillance cameras will exchange information, using a suitable network protocol, about detected anomalies and share relevant image data to aid in decision-making and choose the best images for further processing. In the context of masked face completion, we present a novel architecture called CCGAN network that is a combination of convolutional neural network (CNN) and conditioned generative adversarial network (CGAN) to generate the hidden parts of the face in a form that is accurate and close to the original face shape, as shown in our results. We conduct extensive experiments on publicly available datasets, demonstrating superior performance in both anomaly detection and masked face completion tasks. We have achieved 90% accuracy for anomaly detection of masked people.

Generation of Face Privacy-Protected Images Based on the Diffusion Model.

In light of growing concerns about the misuse of personal data resulting from the widespread use of artificial intelligence technology, it is necessary to implement robust privacy-protection methods. However, existing methods for protecting facial privacy suffer from issues such as poor visual quality, distortion and limited reusability. To tackle this challenge, we propose a novel approach called Diffusion Models for Face Privacy Protection (DIFP). Our method utilizes a face generator that is conditionally controlled and reality-guided to produce high-resolution encrypted faces that are photorealistic while preserving the naturalness and recoverability of the original facial information. We employ a two-stage training strategy to generate protected faces with guidance on identity and style, followed by an iterative technique for improving latent variables to enhance realism. Additionally, we introduce diffusion model denoising for identity recovery, which facilitates the removal of encryption and restoration of the original face when required. Experimental results demonstrate the effectiveness of our method in qualitative privacy protection, achieving high success rates in evading face-recognition tools and enabling near-perfect restoration of occluded faces.

Eksistensi dan Fungsi Kampung (Lewo) dalam Pemerintahan Desa di Hinga Kecamatan Kelubagolit Kabupaten Flores Timur

This research is motivated by the existence and function of the village (Lewo) in village government, which shows that the existence of a village as a village is still classified as a strong criterion. This is because the village (Lewo) can give identity to a village where the original face of the village is still visible, which is marked by the existence of customs that are still firmly attached. So that the formulation of the problem in this research: (1) How is the existence and function of the village (Lewo) in village governance? (2) What is the relevance of village (Lewo) in Village Administration? This research is normative legal research supported by empirical legal research. Aspects of research include the existence and function of the village (Lewo) in village administration, the relationship between the existence and function of the village (Lewo) and the village government in village government, the dynamics of the existence and function of the village (Lewo) with the village government in village governance. The results of the study show that: (1) The function of the village or lewotanah is carried out by a triad of collegial leadership consisting of landlords called Tanah alapen, village heads called mehene lewo, and customary priests called rerawulan alapen. The lewotanah leadership then formed a governance structure and placed leadership in each corner of lewotanah. (2) that the function of the village or lewotanah is carried out by a triumvirate of collegial leadership consisting of landlords called Tanah Alapen, village heads called mehene lewo, and customary priests called rera wulan alapen and other village functions are to assist the village government in solving problems that occurs between communities in order to maintain harmony between community members and assist the village government in village development.

DESIGNING AND VALIDATION OF DOSHA VRUDDHI KSHAYA SCALE (SDVK SCALE)

The concept of Dosha Vruddhi and Kshaya in Ayurveda is intricate. It revolves around the balance of three Doshas: Vata, pitta, and Kapha. These biological entities are fundamental to the body’s constitution, and their equilibrium is essential for maintaining health. Dosha Vruddhi refers to an increase or aggravation of Doshas. At the same time, Dosha Kshaya indicates a decreased or diminished state. Both states can lead to health issues. And the Doshas can vitiate themselves, and it vitiates other entities like Dhatus and Malas. It helps treat the patients early, so this study has been conducted to prepare the Dosha Vruddhi Kshaya Scale. Aim &amp;Objectives: This study aims to design and validate the Dosha Vruddhi Kshaya Scale (SDVK Scale) in 100 healthy individuals. Design: A Cross-Sectional Survey Study. Methods: The Dosha Vruddhi Kshaya Scale has been designed and validated by reliability tests like Face, Content, and Construct Validity. The reliability analysed by Cronbach’s alpha coefficient, i.e. 0.752, and KMO Bartlett’s test valve is 0. 529. The Exploratory Factor Analysis used Kaiser’s criteria for extracting factors with Varimax rotation. Results: Dosha Vruddhi Kshaya Lakshana reproduced the structure of the original face validity in this study sample. Content validity met the standards. Internal consistency was excellent, with KMO Bartlett’s test valve at 0.529 and Cronbach’s alpha coefficient at 0.752, which is highly significant. Conclusion: The Dosha Vruddhi Kshaya Scale is a valid and reliable instrument for assessing Dosha Vruddhi Kshaya Lakshanas in healthy individuals.

The influence of featural and configural changes on the attractiveness evaluation of one’s own face

ABSTRACT In the present study, featural changes and configural changes were manipulated to explore the influence of such changes on the attractiveness evaluation of participants' own faces and other faces. Thirty female and 30 male participants were asked to complete three tasks, i.e., similarity rating of a given face to the original face, paired comparison of facial attractiveness, and subjective evaluation of facial attractiveness, on the same-sex original faces, featural faces and configural faces. The results revealed that the rank order of the attractiveness ratings of the three types of faces was generally identical: original faces > featural faces > configural faces, for both self-faces and other faces in male and female participants. More importantly, faces with configural changes were rated as less attractive than faces with featural changes, which is consistent with the configurational hypothesis. These findings have important implications for facial beautification.

AI-assisted deepfake detection using adaptive blind image watermarking

This paper proposes a new adaptive blind watermarking technology for deepfake detection, which can embed deepfake detection information into the image and verify the image's authenticity without requiring additional information. The proposed scheme utilizes mixed modulation combined with partly sign-altered mean value to embed a set of coefficients that enhance robustness against attacks while maintaining high image quality. Additionally, blind adaptive deepfake detection technology with the tamper detection mean value is employed to detect relative positions adaptively, even when face images are slightly modified or deepfaked. To further improve the performance of the proposed scheme, a gray wolf optimizer is introduced to optimize parameters, and a denoising autoencoder is employed to facilitate the identification of extracted watermarks. This technology will adaptively embed watermark information while preserving the original face image, thereby maintaining the authenticity of the face in the image and verifying the owner of the image. The code is available at https://github.com/lyhsu01/AwDD.

Face De-Identification Using Convolutional Neural Network (CNN) Models for Visual-Copy Detection

The proliferation of media-sharing platforms has led to issues with illegally edited content and the distribution of pornography. To protect personal information, de-identification technologies are being developed to prevent facial identification. Existing de-identification methods directly alter the pixel values in the face region, leading to reduced feature representation and identification accuracy. This study aims to develop a method that minimizes the possibility of personal identification while effectively preserving important features for image- and video-copy-detection tasks, proposing a new deep-learning-based de-identification approach that surpasses traditional pixel-based alteration methods. We introduce two de-identification models using different approaches: one emphasizing the contours of the original face through feature inversion and the other generating a blurred version of the face using D2GAN (Dual Discriminator Generative Adversarial Network). Both models were evaluated on their performance in image- and video-copy-detection tasks before and after de-identification, demonstrating effective feature preservation. This research presents new possibilities for personal-information protection and digital-content security, contributing to digital-rights management and law enforcement.

Head model reconstruction and animation method using color image with depth information

The article presents a method for reconstructing and animating a digital model of a human head from a single RGBD image, a color RGB image with depth information. An approach is proposed for optimizing the parametric FLAME model using a point cloud of a face corresponding to a single RGBD image. The results of experimental studies have shown that the proposed optimization approach makes it possible to obtain a head model with more prominent features of the original face compared to optimization approaches using RGB images or the same approaches generalized to RGBD images.

DESIGNING AND VALIDATION OF ARDHASHAKTI VYAYAMA SCALE

Background: Vyayama is essential for the maintenance of health and also to eradicate the disease. Acharya Charaka says regular and proper exercise makes strongness and steadiness of body entities. Acharya Sushruta says people should practice physical exercise daily up to their Ardhashakti, i.e., half of their physical capacity. In Ayurveda, it is mentioned that excess Vyayama leads to severe complications, so during Vyayama, a person should follow Ardha Shakti Vyayama Lakshana. The Ardha Shakti Vyayama scale is needed to access the Ardha Shakti Vyayama Lakshana, so a study has been conducted to prepare the Ardhashakti Vyayama scale. Aim &amp; Objectives: This study aims to design and validate the Ardhashakti Vyayama scale (ASV scale) in a sample of 50 healthy individuals. Design: A cross-sectional survey study. Methods: The Ardhashakti Vyayama scale has been designed and validated by reliability and validity tests like Face Validity, Content Validity, and Construct validity. The reliability analyzed by Cronbach's α coefficient I.e0.824 and KMO Bartlett's test value is 0.799. The Exploratory Factor Analysis was carried out using Kaiser's criteria for extracting factors with Varimax rotation. Results: Ardhashakti Vyayama Lakshana in this study sample reproduced the similar structure of the original Face validity, Content validity, and Construct validity met the standards, internal consistency was excellent with KMO Bartlett's Test value 0 .799 and Cronbach's α coefficient 0.824 which is highly significant. Conclusion: The Ardhashakti Vyayama scale is a valid and reliable instrument for assessing Ardhashakti Vyayama Lakshana in healthy individuals.

Eyes See Hazy while Algorithms Recognize Who You Are

Facial recognition technology has been developed and widely used for decades. However, it has also made privacy concerns and researchers’ expectations for facial recognition privacy-preserving technologies. To provide privacy, detailed or semantic contents in face images should be obfuscated. However, face recognition algorithms have to be tailor-designed according to current obfuscation methods, as a result the face recognition service provider has to update its commercial off-the-shelf (COTS) products for each obfuscation method. Meanwhile, current obfuscation methods have no clearly quantified explanation. This paper presents a universal face obfuscation method for a family of face recognition algorithms using global or local structure of eigenvector space. By specific mathematical explanations, we show that the upper bound of the distance between the original and obfuscated face images is smaller than the given recognition threshold. Experiments show that the recognition degradation is 0% for global structure based and 0.3%-5.3% for local structure based, respectively. Meanwhile, we show that even if an attacker knows the whole obfuscation method, he/she has to enumerate all the possible roots of a polynomial with an obfuscation coefficient, which is computationally infeasible to reconstruct original faces. So our method shows a good performance in both privacy and recognition accuracy without modifying recognition algorithms.

Deep Cross-view Reconstruction GAN based on Correlated Subspace for Multi-view Transformation.

In scenarios where identifying face information in the visible spectrum (VIS) is challenging due to poor lighting conditions, the use of near-infrared (NIR) and thermal (TH) cameras can provide viable alternatives. However, the unique data distribution of images captured by these cameras compared to VIS images presents challenges in matching face identities. To address these challenges, we propose a novel image transformation framework. The framework includes feature extraction from the input image, followed by a transformation network that generates target domain images with perceptual fidelity. Additionally, a reconstruction network preserves original information by reconstructing the original domain image from the extracted features. By considering the correlation between features from both domains, our framework utilizes paired data obtained from the same individual. We apply this framework to two well-established image-to-image transformation models, pix2pix and CycleGAN, known as CRC-pix2pix and CRC-CycleGAN respectively. The versatility of our approach allows extension to other models based on pix2pix or CycleGAN architectures. Our models generate high-quality images while preserving the identity information of the original face. Performance evaluation on TFW and BUAA NIR-VIS datasets demonstrates the superiority of our models in terms of generated image face matching and evaluation metrics such as SSIM, MSE, PSNR, and LPIPS. Moreover, we introduce the CQUPT-VIS-TH dataset, which enriches the paired dataset with thermal-visual face data capturing various angles and expressions.

The Influence of the Other-Race Effect on Susceptibility to Face Morphing Attacks.

Facial morphs created between two identities resemble both of the faces used to create the morph. Consequently, humans and machines are prone to mistake morphs made from two identities for either of the faces used to create the morph. This vulnerability has been exploited in "morph attacks" in security scenarios. Here, we asked whether the "other-race effect" (ORE)-the human advantage for identifying own- vs. other-race faces-exacerbates morph attack susceptibility for humans. We also asked whether face-identification performance in a deep convolutional neural network (DCNN) is affected by the race of morphed faces. Caucasian (CA) and East-Asian (EA) participants performed a face-identity matching task on pairs of CA and EA face images in two conditions. In the morph condition, different-identity pairs consisted of an image of identity "A" and a 50/50 morph between images of identity "A" and "B". In the baseline condition, morphs of different identities never appeared. As expected, morphs were identified mistakenly more often than original face images. Of primary interest, morph identification was substantially worse for cross-race faces than for own-race faces. Similar to humans, the DCNN performed more accurately for original face images than for morphed image pairs. Notably, the deep network proved substantially more accurate than humans in both cases. The results point to the possibility that DCNNs might be useful for improving face identification accuracy when morphed faces are presented. They also indicate the significance of the race of a face in morph attack susceptibility in applied settings.

Re-construction of original face images from plastic surgery images using harmony search algorithm and IRBFN model

Face Reconstruction is the only biometric that allows you to perform passive identification in one to many environments. By changing the geometry and texture of the facial region, there can be an increased intraclass variability between images of an individual before and after surgery. Even so, the nonlinear changes caused by plastic surgery have a hard time being modelled by facial reconstruction systems. An algorithm for plastic surgery face reconstruction is presented here, that uses Harmony Search to confront the challenges involved. A non-disjoint algorithm generates the face granules, each representing different information with varying resolutions and sizes. A Scale-Invariant Feature Transform (SIFT) and Co-occurrence of Adjacent Local Binary Patterns were applied to extract discriminating information from granules of face images. As the concluding step, we used Harmony Search Method (HSM) to combine the various responses to attain the optimal solution. Finally Improved Radial Basis Function Network (IRBFN) algorithm is used for feature matching. The feature matching predict, where the face is recognized or not. The modified method, proposed in this work, has the better performance in classifying more images with higher accuracy. An obtained sensitivity of 100 % and specificity of 97.5 % was achieved by the proposed classifier. The results proved that it is possible to use the developed algorithms understand different pre and post surgical image conditions, evaluate their strength and future development trends, and provide all-facial features basis for detection of pre and post surgical images.