Bosphorus Database Research Articles

A fast and efficient pre-training method based on layer-by-layer maximum discrimination for deep neural networks

In this paper, through extension of the present methods and based on error minimization, two fast and efficient layer-by-layer pre-training methods are proposed for initializing deep neural network (DNN) weights. Due to confrontation with a large number of local minima, DNN training often does not converge. By proper initializing of DNN weights instead of random values at the beginning of the training, it is possible to avoid many local minima. The first version of the proposed method is for pre-training the deep bottleneck neural network (DBNN) in which the DBNN is broken down to some corresponding single-hidden-layer bottleneck neural networks (BNN) which must be trained first. The weight values resulting from their training are then applied in the DBNN. The proposed method was utilized to pre-train a five-hidden-layer DBNN to extract the non-linear principal components of face images in the Bosphorus database. A comparison of the randomly initialized DBNN result with pre-trained DBNN by the layer-by-layer pre-training method shows that this method not only increased the convergence rate of training but also improved its generalizability. Furthermore, it has been shown that this method yields higher efficiency and convergence speed in comparison with some of the previous pre-training methods. This paper also presents the bidirectional version of the layer-by-layer pre-training method for hetero-associative DNN pre-training. This method pre-trains DNN weights in forward and backward manner in parallel. Bidirectional layer-by-layer pre-training was utilized to pre-train the classifier DNN weights, and revealed that both the training speed and the recognition rate were improved in Bosphorus and CK+ databases.

Depth based Occlusion Detection and Localization from 3D Face Image

In this paper, authors have proposed two novel techniques for occlusion detection and then localization of the occluded section from a given 3D face image if occlusion is present. For both of these methods, at first, the 2.5D or range face images are created from input 3D face images. Then for detecting the occluded faces, two approaches have been followed, namely: block based and threshold based. These two methods have been investigated individually on Bosphorus database for localization of occluded portion. Bosphorus database consists of different types of occlusions, which have been considered during our research work. If 2D and 3D images are compared then 3D images provide more reliable, accurate, valid information within digitized data. In case of 2D images each point, named as pixel, is represented by a single value. One byte for gray scale and three byte for color images in a 2D grid whereas in case of 3D, there is no concept of 2D grid. Each point is represented by three values, namely X, Y and Z. The ‘Z’ value in X-Y plane does not contain the reflected light energy like 2D images. The facial surface’s depth data is included in Z’s point set. The threshold or cutoff based technique can detect the occluded faces with the accuracy 91.79% and second approach i.e. block based approach can successfully detect the same with the success rate of 99.71%. The accuracy of the proposed occlusion detection scheme has been measured as a qualitative parameter based on subjective fidelity criteria.

Illumination, Pose and Occlusion Invariant Face Recognition from Range Images Using ERFI Model

In this paper the pivotal contribution of the authors is to recognize the 3D face images from range images in the unconstrained environment i.e. under varying illumination, pose as well as occlusion that are considered to be the most challenging task in the domain of face recognition. During this investigation, face images have been normalized in terms of pose registration as well as occlusion restoration using ERFI (Energy Range Face Image) model. 3D face images are inherently illumination invariant due its point-based representation of data along three axes. Here, other than quantitative analysis, a subjective analysis is also carried out. However, synthesized datasets have been accomplished to investigate the performance of recognition rate from Frav3D and Bosphorus databases using SIFT and SURF like features. Moreover, weighted fusion of these individual feature sets is also done. Later these feature sets have been classified by K-NN and Sequence Matching Technique and achieved maximum recognition rates of 99.17% and 98.81% for Frav3D and GavabDB databases respectively.

3D Face Model Estimation based on Similarity Transform using Differential Evolution Optimization

Abstract 3D Face model reconstruction from a single 2D image is fundamentally important for face recognition because the 3D model is invariant to changes of viewpoint, illumination, background clutter and occlusions. In this paper, an efficient 3D Face Reconstruction algorithm is proposed based on multi-view 2D images of human face based on similarity transform measurements. In this algorithm, the pose and depth estimation from 2D feature points of the respective face images is considered as an optimization problem and solved using Differential Evolution optimization. Further different strategies of differential evolution are used to optimize and the results are compared. Simple model integration method is proposed to improve the estimation accuracy of the 3D structure of face, when more than one non-frontal-view face images are available. Furthermore, Pearson Linear Correlation is computed to show the efficiency of the proposed approach. Experimental results on 2D Head Pose and 3D Bosphorus databases demonstrate the feasibility and efficiency of the proposed methods.

Automatic Analysis of Smoothing Techniques by Simulation Model Based Real-Time System for Processing 3D Human Faces

The pivotal research work that has been carried out and described in this literature acknowledges the importance of various smoothing techniques for processing 3D human faces from 2.5D range face images. The smoothing techniques have been developed and implemented using MATLAB-Simulink for real time processing in embedded system. In addition, the significance of smoothed 2.5D range image over original face range image has been discovered as well as its time complexity has also been reported with array of experiments. The variations in time complexities are also accomplished using different optimization levels and execution modes. A set of filtering techniques such as, Max filter, Min filter, Median filter, Mean filter, Mid-point filter and Gaussian filter, have been designed and illustrated using Simulink model. The model takes depth face image (i.e. the range face image) as input in real time and presents the improvement over original face images. In the design flow, the performance of every block has also been characterized by range face images from Frav3D, GavabDB, and Bosphorus databases. In the experimental section of this research article, an array of performance analysis for these smoothing techniques with variation of frameworks is explained.

Three-dimensional face recognition under expression variation

In this paper, we introduce a fully automatic framework for 3D face recognition under expression variation. For 3D data preprocessing, an improved nose detection method is presented. The small pose is corrected at the same time. A new facial expression processing method which is based on sparse representation is proposed subsequently. As a result, this framework enhances the recognition rate because facial expression is the biggest obstacle for 3D face recognition. Then, the facial representation, which is based on the dual-tree complex wavelet transform (DT-CWT), is extracted from depth images. It contains the facial information and six subregions’ information. Recognition is achieved by linear discriminant analysis (LDA) and nearest neighbor classifier. We have performed different experiments on the Face Recognition Grand Challenge database and Bosphorus database. It achieves the verification rate of 98.86% on the all vs. all experiment at 0.1% false acceptance rate (FAR) in the Face Recognition Grand Challenge (FRGC) and 95.03% verification rate on nearly frontal faces with expression changes and occlusions in the Bosphorus database.

Towards 3D Face Recognition in the Real: A Registration-Free Approach Using Fine-Grained Matching of 3D Keypoint Descriptors

Registration algorithms performed on point clouds or range images of face scans have been successfully used for automatic 3D face recognition under expression variations, but have rarely been investigated to solve pose changes and occlusions mainly since that the basic landmarks to initialize coarse alignment are not always available. Recently, local feature-based SIFT-like matching proves competent to handle all such variations without registration. In this paper, towards 3D face recognition for real-life biometric applications, we significantly extend the SIFT-like matching framework to mesh data and propose a novel approach using fine-grained matching of 3D keypoint descriptors. First, two principal curvature-based 3D keypoint detectors are provided, which can repeatedly identify complementary locations on a face scan where local curvatures are high. Then, a robust 3D local coordinate system is built at each keypoint, which allows extraction of pose-invariant features. Three keypoint descriptors, corresponding to three surface differential quantities, are designed, and their feature-level fusion is employed to comprehensively describe local shapes of detected keypoints. Finally, we propose a multi-task sparse representation based fine-grained matching algorithm, which accounts for the average reconstruction error of probe face descriptors sparsely represented by a large dictionary of gallery descriptors in identification. Our approach is evaluated on the Bosphorus database and achieves rank-one recognition rates of 96.56, 98.82, 91.14, and 99.21 % on the entire database, and the expression, pose, and occlusion subsets, respectively. To the best of our knowledge, these are the best results reported so far on this database. Additionally, good generalization ability is also exhibited by the experiments on the FRGC v2.0 database.

Towards a robust affect recognition: Automatic facial expression recognition in 3D faces

Facial expressions are a powerful tool that communicates a person’s emotional state and subsequently his/her intentions. Compared to 2D face images, 3D face images offer more granular cues that are not available in the 2D images. However, one major setback of 3D faces is that they impose a higher dimensionality than 2D faces. In this paper, we attempt to address this problem by proposing a fully automatic 3D facial expression recognition model that tackles the high dimensionality problem in a twofold solution. First, we transform the 3D faces into the 2D plane using conformal mapping. Second, we propose a Differential Evolution (DE) based optimization algorithm to select the optimal facial feature set and the classifier parameters simultaneously. The optimal features are selected from a pool of Speed Up Robust Features (SURF) descriptors of all the prospective facial points. The proposed model yielded an average recognition accuracy of 79% using the Bosphorus database and 79.36% using the BU-3DFE database. In addition, we exploit the facial muscular movements to enhance the probability estimation (PE) of Support Vector Machine (SVM). Joint application of feature selection with the proposed enhanced PE (EPE) yielded an average recognition accuracy of 84% using the Bosphorus database and 85.81% using the BU-3DFE database, which is statistically significantly better (at p<0.01 and p<0.001, respectively) if compared to the individual exploit of the optimal features only.

3-D Facial Landmark Localization With Asymmetry Patterns and Shape Regression from Incomplete Local Features.

We present a method for the automatic localization of facial landmarks that integrates nonrigid deformation with the ability to handle missing points. The algorithm generates sets of candidate locations from feature detectors and performs combinatorial search constrained by a flexible shape model. A key assumption of our approach is that for some landmarks there might not be an accurate candidate in the input set. This is tackled by detecting partial subsets of landmarks and inferring those that are missing, so that the probability of the flexible model is maximized. The ability of the model to work with incomplete information makes it possible to limit the number of candidates that need to be retained, drastically reducing the number of combinations to be tested with respect to the alternative of trying to always detect the complete set of landmarks. We demonstrate the accuracy of the proposed method in the face recognition grand challenge database, where we obtain average errors of approximately 3.5 mm when targeting 14 prominent facial landmarks. For the majority of these our method produces the most accurate results reported to date in this database. Handling of occlusions and surfaces with missing parts is demonstrated with tests on the Bosphorus database, where we achieve an overall error of 4.81 and 4.25 mm for data with and without occlusions, respectively. To investigate potential limits in the accuracy that could be reached, we also report experiments on a database of 144 facial scans acquired in the context of clinical research, with manual annotations performed by experts, where we obtain an overall error of 2.3 mm, with averages per landmark below 3.4 mm for all 14 targeted points and within 2 mm for half of them. The coordinates of automatically located landmarks are made available on-line.

Adaptive 3D facial action intensity estimation and emotion recognition

Automatic recognition of facial emotion has been widely studied for various computer vision tasks (e.g. health monitoring, driver state surveillance and personalized learning). Most existing facial emotion recognition systems, however, either have not fully considered subject-independent dynamic features or were limited to 2D models, thus are not robust enough for real-life recognition tasks with subject variation, head movement and illumination change. Moreover, there is also lack of systematic research on effective newly arrived novel emotion class detection. To address these challenges, we present a real-time 3D facial Action Unit (AU) intensity estimation and emotion recognition system. It automatically selects 16 motion-based facial feature sets using minimal-redundancy–maximal-relevance criterion based optimization and estimates the intensities of 16 diagnostic AUs using feedforward Neural Networks and Support Vector Regressors. We also propose a set of six novel adaptive ensemble classifiers for robust classification of the six basic emotions and the detection of newly arrived unseen novel emotion classes (emotions that are not included in the training set). A distance-based clustering and uncertainty measures of the base classifiers within each ensemble model are used to inform the novel class detection. Evaluated with the Bosphorus 3D database, the system has achieved the best performance of 0.071 overall Mean Squared Error (MSE) for AU intensity estimation using Support Vector Regressors, and 92.2% average accuracy for the recognition of the six basic emotions using the proposed ensemble classifiers. In comparison with other related work, our research outperforms other state-of-the-art research on 3D facial emotion recognition for the Bosphorus database. Moreover, in on-line real-time evaluation with real human subjects, the proposed system also shows superior real-time performance with 84% recognition accuracy and great flexibility and adaptation for newly arrived novel (e.g. ‘contempt’ which is not included in the six basic emotions) emotion detection.

Multiple Regions Matching with Spin Images for 3D Face Recognition

This paper presents a novel method for 3D face recognition under expression and occlusion variations. It is based on the combination of the matching results from matching multiple partial regions on the 3D face. Spin image features are used for comparison between faces. Meanwhile, an efficient approach is proposed to detect nose tip and the nose tip determines the location of each candidate region. To fuse the contribution of each region, score-based and rank-based fusion scheme are performed and compared. The proposed method is evaluated on the Bosphorus database and achieves good results. It obtains 96.68% recognition rate in the presence of expression and occlusion variations.

Selecting stable keypoints and local descriptors for person identification using 3D face scans

3D face identification based on the detection and comparison of keypoints of the face is a promising solution to extend face recognition approaches to the case of 3D scans with occlusions and missing parts. In fact, approaches that perform sparse keypoints matching can naturally allow for partial face comparison. However, such methods typically use a large number of keypoints, locally described by high-dimensional feature vectors: This, combined with the combinatorial number of keypoint comparisons required to match two face scans, results in a high computational cost that does not scale well with large datasets. Motivated by these considerations, in this paper, we present a 3D face recognition approach based on the meshDOG keypoints detector and local GH descriptor, and propose original solutions to improve keypoints stability and select the most effective features from the local descriptors. Experiments have been performed to assess the validity of the proposed optimizations for stable keypoints detection and feature selection. Recognition accuracy has been evaluated on the Bosphorus database, showing competitive results with respect to existing 3D face identification solutions based on 3D keypoints.

A Novel Biometric System Based Hand Vein Recognition

Abstract As a reliable and universal biometric characteristic, hand vein identification has attracted many interested researchers. The hand vein identification system exhibits several excellent advantages in the biometric domain because it meets the increasing demand of accuracy and robustness. In this paper, we propose a new biometric recognition system based on hand vein features. The detection and extraction of the region of interest is based on Voronoi Decomposition. Furthermore, contrast enhancement is based on the Contrast Limited Adaptive Histogram Equalization (CLAHE) technique. Starting with the basic Gaussian Matched Filter (GMF) and its variant, we propose a new technique called the Improved Gaussian Matched Filter method (IMPGMF) surmounting the false detection of hand vessels with the traditional GMF. Feature points are then detected based on ending and bifurcation structures in the image map obtained with the proposed IMPGMF method and taken as signature for our biometric system. Then, Artificial Neural Networks (ANN) are used for the classification step. In the validation step, we used a 1500 hand vein image from the BOSPHORUS database. The Error Equal Rate is 0.01 % and the Area Under curve of the corresponding system is approximately 0.98, demonstrating a very high security level.

Simultaneous Learning of Nonlinear Manifolds Based on the Bottleneck Neural Network

Manifold learning methods are important techniques for nonlinear extraction of high-dimensional data structures. These methods usually extract a global manifold for data. However, in many real-world problems, there is not only one global manifold, but also additional information about the objects is shared by a large number of manifolds. These manifolds can share information for data reconstruction. To simultaneously extract these data manifolds, this paper proposes a nonlinear method based on the deep neural network (NN) named nonlinear manifold separator NN (NMSNN). Unlike unsupervised learning of bottleneck NN, data labels were used for simultaneous manifold learning. This paper makes use of NMSNN for extracting both expression and identity manifolds for facial images of the CK+ database. These manifolds have been evaluated by different metrics. The identity manifold is used for changing image identity. The result of identity recognition by K-nearest neighbor classifier shows that virtual identities are exactly sanitized. The virtual images for different expressions of test subjects are generated by expression manifold. The facial expression recognition rate of 92.86 % is achieved for virtual expressions of test persons. It is shown that NMSNN can be used to enrich datasets by sanitizing virtual images. As a result, 8 and 19 % improvements are gained in the face recognition task by a single image of each person on CK+ and Bosphorus databases, respectively.

MeshSIFT: Local surface features for 3D face recognition under expression variations and partial data

Matching 3D faces for recognition is a challenging task caused by the presence of expression variations, missing data, and outliers. In this paper the meshSIFT algorithm and its use for 3D face recognition is presented. This algorithm consists of four major components. First, salient points on the 3D facial surface are detected as mean curvature extrema in scale space. Second, orientations are assigned to each of these salient points. Third, the neighbourhood of each salient point is described in a feature vector consisting of concatenated histograms of shape indices and slant angles. Fourth, the feature vectors of two 3D facial surfaces are reliably matched by comparing the angles in feature space. This results in an algorithm which is robust to expression variations, missing data and outliers.As a first contribution, we demonstrate that the number of matching meshSIFT features is a reliable measure for expression-invariant face recognition, as shown by the rank 1 recognition rate of 93.7% and 89.6% for the Bosphorus and FRGC v2 database, respectively. Next, we demonstrate that symmetrising the feature descriptors allows comparing two 3D facial surfaces with limited or no overlap. Validation on the data of the “SHREC’11: Face Scans” contest, containing many partial scans, resulted in a recognition rate of 98.6%, clearly outperforming all other participants in the challenge. Finally, we also demonstrate the use of meshSIFT for two other problems related with 3D face recognition: pose normalisation and symmetry plane estimation. For both problems, applying meshSIFT in combination with RANSAC resulted in a correct solution for ±90% of all Bosphorus database meshes (except ±90° and ±45° rotations).

Impact of action unit detection in automatic emotion recognition

In this paper, we investigate the interest of action unit (AU) detection for automatic emotion recognition. We propose and compare two emotion detectors: the first works directly on a high-dimensional feature space and the second projects facial image in the low-dimensional space of AU intensities before recognizing emotion. In both approaches, facial images are coded by local Gabor binary pattern (LGBP) histogram differences. These features reduce the sensitivity to subject identity by computing differences between two LGBP histograms: one computed on an expressive image and the other synthesized and approaching the one we would compute on a neutral face of the same subject. As classifiers, we test support vector machines with different kernels. A new kernel is proposed, the histogram difference intersection kernel that increases classification performances. This kernel is well suited when exploiting the proposed histogram differences. Thorough experiments on three challenging databases (respectively, the Cohn-Kanade, MMI and Bosphorus databases) show the accuracy of our AU and emotion detectors. They lead to significant conclusions on three critical issues: (1) the interest of combining different training databases labeled by different AU coders, (2) the influence of each AU according to its type and detection accuracy on emotion recognition and (3) the sensitivity to identity variations.

2.5D face recognition using Patch Geodesic Moments

In this paper, we propose a novel Patch Geodesic Distance (PGD) to transform the texture map of an object through its shape data for robust 2.5D object recognition. Local geodesic paths within patches and global geodesic paths for patches are combined in a coarse to fine hierarchical computation of PGD for each surface point to tackle the missing data problem in 2.5D images. Shape adjusted texture patches are encoded into local patterns for similarity measurement between two 2.5D images with different viewing angles and/or shape deformations. An extensive experimental investigation is conducted on 2.5 face images using the publicly available BU-3DFE and Bosphorus databases covering face recognition under expression and pose changes. The performance of the proposed method is compared with that of three benchmark approaches. The experimental results demonstrate that the proposed method provides a very encouraging new solution for 2.5D object recognition.

A Statistical Method for 2-D Facial Landmarking

Many facial-analysis approaches rely on robust and accurate automatic facial landmarking to correctly function. In this paper, we describe a statistical method for automatic facial-landmark localization. Our landmarking relies on a parsimonious mixture model of Gabor wavelet features, computed in coarse-to-fine fashion and complemented with a shape prior. We assess the accuracy and the robustness of the proposed approach in extensive cross-database conditions conducted on four face data sets (Face Recognition Grand Challenge, Cohn-Kanade, Bosphorus, and BioID). Our method has 99.33% accuracy on the Bosphorus database and 97.62% accuracy on the BioID database on the average, which improves the state of the art. We show that the method is not significantly affected by low-resolution images, small rotations, facial expressions, and natural occlusions such as beard and mustache. We further test the goodness of the landmarks in a facial expression recognition application and report landmarking-induced improvement over baseline on two separate databases for video-based expression recognition (Cohn-Kanade and BU-4DFE).

3D Face Recognition Using Geodesic PZM Array from a Single Model per Person

For the 3D face recognition numerous methods have been proposed, but little attention has been given to the local-based representation for the texture map of the 3D models. In this paper, we propose a novel 3D face recognition approach based on locally extracted Geodesic Pseudo Zernike Moment Array (GPZMA) of the texture map when only one exemplar per person is available. In the proposed method, the function of the PZM is controlled by the geodesic deformations to tackle the problem of face recognition under the expression and pose variations. The feasibility and effectiveness investigation for the proposed method is conducted through a wide range of experiments using publicly available BU-3DFE and Bosphorus databases including samples with different expression and pose variations. The performance of the proposed method is compared with the performance of three state-of-the-art benchmark approaches. The encouraging experimental results demonstrate that the proposed method achieves much higher accuracy than the benchmarks in single-model databases.