CUHK Face Sketch Database Research Articles

Method for generating masks on face images and systems for their recognition

The problem of masked face recognition is investigated. It is shown that real masks of various shapes, textures and colors have become a problem for state-of-the-art face recognition systems. A reason for this is the lack of the necessary real training datasets. Creation of new data based on simple methods of forming masks on face images could solve this problem. An original method is proposed including the generation of various types, shapes, and colors of masks directly on the original texture of face images. The formation of the masks on the faces of individuals, on faces in group photos, and in scenes with streams of people was taken into account. Based on 100 original face images from the CUHK Face Sketch Database, a test database was created that includes more than 20,000 masked faces images which available for use. Experiments were carried out to recognize faces from the test database within the implemented four systems, among which three are state-of-the-art systems based on “deep learning” and one is deterministic system based on the cosine-transform. The performance of these systems was evaluated, the obtained results of masked face recognition were interpreted, and the masks that were a problem for selected four systems were noted. The proposed mask generation method can be used to create corpora and test databases of images with masks. The obtained results will be useful to researchers and specialists in the field of image processing and analysis.

Face photo-sketch synthesis via full-scale identity supervision

Face photo-sketch synthesis refers transforming a face image between photo domain and sketch domain. It plays a crucial role in law enforcement and digital entertainment. A great deal of efforts have been devoted on face photo-sketch synthesis. However, limited by the weak identity supervision, existing methods mostly yield indistinct details or great deformation, resulting in poor perceptual appearance or low recognition accuracy. In the past several years, face identification achieved great progress, which represents the face images much more precisely than before. Considering the face image translation is also a type of face image re-representation, we attempt to introduce face recognition models to improve the synthesis performance. First, we applied existing synthesis models to augment the training set. Then, we proposed a full-scale identity supervision method to reduce redundant information introduced by these pseudo samples and take the valid information to enhance the intra-class variations. The proposed framework consists of two sub-networks: cross-domain translation (CT) network and intra-domain adaptation (IA) network. The CT network translates the input image from source domain to latent image of target domain, which overcomes the great gap between two domains with less structural deformation. The IA network adapts the perceptual appearance of latent image to target image by adversarial learning. Experimental results on CUHK Face Sketch Database and CUHK Face Sketch FERET Database demonstrate the proposed method preserved best perceptual appearance and more distinct details with less deformation.

Face recognition based on full convolutional neural network based on transfer learning model

Deep learning has achieved a great success in face recognition (FR), however, little work has been done to apply deep learning for face photo-sketch recognition. This paper proposes an adaptive scale local binary pattern extraction method for optical face features. The extracted features are classified by Gaussian process. The most authoritative optical face test set LFW is used to train the trained model. Test, the test accuracy is 98.7%. Finally, the face features extracted by this method and the face features extracted from the convolutional neural network method are adapted to sketch faces through transfer learning, and the results of the adaptation are compared and analyzed. Finally, the paper tested the open-source sketch face data set CUHK Face Sketch database(CUFS) using the multimedia experiment of the Chinese University of Hong Kong. The test result was 97.4%. The result was compared with the test results of traditional sketch face recognition methods. It was found that the method recognized High efficiency, it is worth promoting.

Pengenalan sketsa wajah menggunakan principle component analysis sebagai aplikasi forensik

Recognition of human faces in forensics applications can be identified through the Sketch recognition method by matching sketches and photos. The system gives five criminal candidates who have similarities to the sketch given. This study aims to perform facial recognition on photographs and sketches using Principal Component Analysis (PCA) as feature extraction and Euclidean distance as a calculation of the distance of test images to training images. The PCA method was used to recognize facial images from pencil sketch drawings. The system dataset is in the form of photos and sketches in the CUHK Face Sketch database consists of 93 photos and 93 sketches, and personal documentation consists of five photos and five sketches. The sketch matching application to training data produces an accuracy of 76.14 %, precision of 91.04 %, and recall of 80.26 %, while testing with sketch modifications produces accuracy and recall of 95 % and precision of 100 %.

Fine Tuning Dual Streams Deep Network with Multi-scale Pyramid Decision for Heterogeneous Face Recognition

In this paper, we propose a novel method called fine tuning dual streams deep network (FTDSDN) with multi-scale pyramid decision (MsPD) for solving heterogeneous face recognition task. As an extension of classical CNNs, FTDSDN can remove highly non-linear modality information and reserve the discriminative information using Rayleigh quotient objective function. Furthermore, we develop a powerful joint decision strategy called MsPD to adaptively adjust the weight of sub structure and obtain more robust classification performance. Experimental results show our proposed method achieves better performance on the challenging CASIA NIR-VIS 2.0 database, the heterogeneous face biometrics database, the CUHK face sketch FERET database, and the CUHK face sketch database, which demonstrates the effectiveness of our proposed approach.

Fast Face Sketch–Photo Image Synthesis and Recognition

Face sketch recognition has great practical value in the criminal detection, security and other fields. Especially, it can help the police narrow down potential suspects in criminal detection effectively. Face sketch represents the original photos in a simple and recognizable form, so sketch and photo are images of two different modes. In order to identify the corresponding sketch face image in a lot of photo face images, this paper presents an improved sketch–photo transformation algorithm, and it uses the effective characteristics of the photo image more reasonably during transforming a photo image into sketch. In this way, it can reduce the difference between the sketch and photo image to improve the matching effect, and save the recognition time. Many experiments on CUHK Face Sketch database including 188 sketch–photos prove the effectiveness of the method in this paper.

Unified framework for face sketch synthesis

Face sketch synthesis (FSS) has great significance to sketch based face retrieval or recognition and digital entertainment. Recently, great progress has been made in face sketch synthesis. Most state-of-the-art FSS methods work at patch level. However, these methods only consider either position constraint or global search when selecting candidate image patches. Furthermore, the common used weighting combination in these methods leads to the lost of reasonable high frequency details. We argue that all these factors are necessary for face sketch synthesis. To this end, we propose a simple yet effective unified approach considering position constraint, global search and high frequency compensation to infer pseudosketches from input test photos. Firstly, a nearest-neighbor search is conducted by combining both position constraint and global search. After obtaining the candidate image patches, a Markov network is applied to generate the pseudosketch. Secondly, the residue between an original sketch and the synthesized pseudosketch is modeled by the same Markov network to compensate the high frequency details. The effectiveness of the proposed method is demonstrated on the CUHK face sketch database by comparing with state-of-the-art FSS methods.

Nearest orthogonal matrix representation for face recognition

This paper presents a simple but effective method for face recognition, named nearest orthogonal matrix representation (NOMR). Specifically, the specific individual subspace of each image is estimated and represented uniquely by the sum of a set of basis matrices generated via singular value decomposition (SVD), i.e. the nearest orthogonal matrix (NOM) of original image. Then, the nearest neighbor criterion is introduced for recognition. Compared with the current specific individual subspace based methods (e.g. the sparse representation based classifier, the linear regression based classifier and so on), the proposed NOMR is more robust for alleviating the effect of illumination and heterogeneous (e.g. sketch face recognition), and more intuitive and powerful for handling the small sample size problem. To evaluate the performance of the proposed method, a series of experiments were performed on several face databases: Extended Yale B, CMU-PIE, FRGCv2, AR and CUHK Face Sketch database (CUFS). Experimental results demonstrate that the proposed method achieves encouraging performance compared with the state-of-the-art methods.

基于人脸画像的伪照片合成及修正

A method was proposed to transform a face sketch into a pseudophoto and to amend the synthesized pseudophoto.It consists of three procedures for the face pseudophoto:synthesizing,enhancement and local amendment.Firstly,the initial face pseudophoto was synthesized by Local Eigentransformation Technique(LET).Then,the edge features and textures of the face sketch were enhanced to improve the visual effect of synthesized pseudophoto.Finally,the local synthesis error of the face pseudophoto was amended by moving least squares algorithm.The experiment was performed on the CUHK face sketch database and the transformation was compared with real photos.Result shows that the method can effectively synthesize and amend the pseudophoto and can guarantee the similarity with sufficient amendment.As the proposed method only involves the deformation of a local region,it is independent on the neural network to predict the target location in control points.So,it has lowered the computing complexity.