Kinship Verification Research Articles

Distance and Direction Based Deep Discriminant Metric Learning for Kinship Verification

Image-based kinship verification is an important task in computer vision and has many applications in practice, such as missing children search and family album construction, among others. Due to the differences in age, gender, expression and appearance, there usually exists a large discrepancy between the facial images of parent and child. This makes kinship verification a challenging task. In this article, we propose a Distance and Direction Based Deep Discriminant Metric Learning (D 4 ML) approach for kinship verification. The basic idea of D 4 ML is to make full use of the discriminant information contained in the facial images of parent and child such that the network can learn more a discriminating distance metric. Specifically, D 4 ML learns the metric by utilizing the discriminant information from two perspectives: distance-based perspective and direction-based perspective. From the distance-based perspective, the designed loss function is used to minimize the distance between images having kinship and maximize the distance between images without kinship. In practice, the gender difference and large age gap may significantly increase the distance between facial images of parent and child. Therefore, learning the metric only from a distance-based perspective is insufficient. Considering that two vectors with a large distance may appear with high similarity in direction, D 4 ML also employs the direction-based loss function in the training process. Both kinds of loss function work together to improve the discriminability of the learned metric. Experimental results on four small size publicly available datasets demonstrate the effectiveness of our approach. Source code of our approach can be found at https://github.com/lclhenu/D4ML .

Facial kinship verification and searching for genetic origins in gamete/embryo donor conception—an overview of potential legal and ethical issues

Abstract The aim of Facial Kinship Verification (FKV) technologies is to determine, by comparing images of their faces, whether two people are related genetically. FKV is developing rapidly, and it could be used to search for genetic relatives in a variety of ways and settings, such as searching for missing children or unknown parents. In this article, we focus on one area where this technology might have significant implications, the searching for gamete (egg and sperm) and embryo donor(s) by donor-conceived people. In many jurisdictions, donor-conceived people do not have access to information about their donor’s identity, and laws differ significantly in this area. We offer an initial overview of the legal and related ethical issues raised by FKV in this context, and touch on other areas where it might be used to find genetic relatives, as a starting point for further analysis and research.

ChildPredictor: A Child Face Prediction Framework With Disentangled Learning

The appearances of children are inherited from their parents, which makes it feasible to predict them. Predicting realistic children's faces may help settle many social problems, such as age-invariant face recognition, kinship verification, and missing child identification. It can be regarded as an image-to-image translation task. Existing approaches usually assume domain information in the image-to-image translation can be interpreted by "style", i.e., the separation of image content and style. However, such separation is improper for the child face prediction, because the facial contours between children and parents are not the same. To address this issue, we propose a new disentangled learning strategy for children's face prediction. We assume that children's faces are determined by genetic factors (compact family features, e.g., face contour), external factors (facial attributes irrelevant to prediction, such as moustaches and glasses), and variety factors (individual properties for each child). On this basis, we formulate predictions as a mapping from parents' genetic factors to children's genetic factors, and disentangle them from external and variety factors. In order to obtain accurate genetic factors and perform the mapping, we propose a ChildPredictor framework. It transfers human faces to genetic factors by encoders and back by generators. Then, it learns the relationship between the genetic factors of parents and children through a mapping function. To ensure the generated faces are realistic, we collect a large Family Face Database to train ChildPredictor and evaluate it on the FF-Database validation set. Experimental results demonstrate that ChildPredictor is superior to other well-known image-to-image translation methods in predicting realistic and diverse child faces. Implementation codes can be found at https://github.com/zhaoyuzhi/ChildPredictor.

A survey on kinship verification

In this survey, kinship verification is defined as the automatic process of verifying whether two or more persons are blood relatives (kin) by analyzing images of their faces. Kinship verification is an important research field in computer vision with many applications such as finding missing persons, family album organization, and online image search. Although substantial progress has been made in kinship verification in the past decade, there are still challenges such as intrinsic (face i.e., differences in facial appearance) and extrinsic (acquisition i.e., varying imaging conditions) problems. And there is still a demand for more diverse datasets.Therefore, this paper provides a survey on kinship verification methods and datasets. The survey starts with the definition of kinship verification and its corresponding intrinsic and extrinsic challenges. Then, an overview of kinship verification methods and datasets is given. Finally, a new multi-modal dataset (Nemo-Kinship Dataset) is proposed as a benchmark dataset addressing large inter-subject age variations consisting of 4216 videos of 248 persons from 85 families. The newly collected dataset is used to systematically test and analyze state-of-the-art methods.

ACLMHA and FML: A brain-inspired kinship verification framework.

As an extended research direction of face recognition, kinship verification based on the face image is an interesting yet challenging task, which aims to determine whether two individuals are kin-related based on their facial images. Face image-based kinship verification benefits many applications in real life, including: missing children search, family photo classification, kinship information mining, family privacy protection, etc. Studies presented thus far provide evidence that face kinship verification still offers many challenges. Hence in this paper, we propose a novel kinship verification architecture, the main contributions of which are as follows: To boost the deep model to capture various and abundant local features from different local face regions, we propose an attention center learning guided multi-head attention mechanism to supervise the learning of attention weights and make different attention heads notice the characteristics of different regions. To combat the misclassification caused by single feature center loss, we propose a family-level multi-center loss to ensure a more proper intra/inter-class distance measurement for kinship verification. To measure the potential similarity of features among relatives better, we propose to introduce the relation comparison module to measure the similarity among features at a deeper level. Extensive experiments are conducted on the widely used kinship verification dataset-Family in the Wild (FIW) dataset. Compared with other state-of-art (SOTA) methods, encouraging results are obtained, which verify the effectiveness of our proposed method.

Family classification and kinship verification from facial images in the wild

Family classification and kinship verification from facial images in the wild

Binary cross coupled discriminant analysis for visual kinship verification

Kinship verification refers to comparing similarities between two different individuals through their facial images. In this context, feature descriptors play a crucial role, and few feature descriptors are present in literature to extract kin features from facial images. In this paper, we propose a binary cross-coupled discriminant analysis (BC2DA) based feature descriptor which is able to extract effective kin features from input facial image pairs. This method reduces the discrimination between kin pairs at the feature extraction stage itself. BC2DA converts original kin image pairs to encoded image pairs to reduce the discrimination between them. To make better use of tri-subject kin relations, we further propose multi cross-coupled discriminant analysis (MC2DA). This method reduces the discrimination between child and both parents’ images at the feature extraction stage. Extensive experiments were conducted on six kinship datasets such as KinfaceW-I/II, Cornell, FIW, TSKinface UBKinface to show the efficacy of the proposed algorithm.

A performance evaluation of convolution neural networks for kinship discernment: An application in digital forensics

Kinship Verification from facial images is known to have attracted major attention since time immemorial. Identifying the underlying patterns that exist between images and analysing the relationship hidden between them have enabled the multitudes of applications to utilize kinship relationships. This work serves as a study on the amount of influence that hereditary features can exert on the families tied together by lineage in identifying the relationship prevailing between them and whether it holds true or not. The approach employed involves detecting the relationship existing between the provided facial images using Siamese Network, which comprises two identical convolutional neural networks that share common weight values. A difference vector is computed from this Siamese CNN, which is then fed into a network of fully connected linear layers. This extended layer will determine whether the two individuals in the input images are related to each other or not.

Kinship identification using age transformation and Siamese network.

Facial images are used for kinship verification. Traditional convolutional neural networks and transfer learning-based approaches are presently used for kinship identification. The transfer-learning approach is useful in many fields. However, it does not perform well in the identification of humans’ kinship because transfer-learning models are trained on a different type of data that is significantly different as compared to human face image data, a technique which may be able for kinship identification by comparing images of parents and their children with transformed age instead of comparing their actual images is required. In this article, a technique for kinship identification using a Siamese neural network and age transformation algorithm is proposed. The results are satisfactory as an overall accuracy of 76.38% has been achieved. Further work can be carried out to improve the accuracy by improving the Life Span Age Transformation (LAT) algorithm for kinship identification using facial images.

Multi-View Cosine Similarity Learning with Application to Face Verification

An instance can be easily depicted from different views in pattern recognition, and it is desirable to exploit the information of these views to complement each other. However, most of the metric learning or similarity learning methods are developed for single-view feature representation over the past two decades, which is not suitable for dealing with multi-view data directly. In this paper, we propose a multi-view cosine similarity learning (MVCSL) approach to efficiently utilize multi-view data and apply it for face verification. The proposed MVCSL method is able to leverage both the common information of multi-view data and the private information of each view, which jointly learns a cosine similarity for each view in the transformed subspace and integrates the cosine similarities of all the views in a unified framework. Specifically, MVCSL employs the constraints that the joint cosine similarity of positive pairs is greater than that of negative pairs. Experiments on fine-grained face verification and kinship verification tasks demonstrate the superiority of our MVCSL approach.

Siamese convolutional neural network and fusion of the best overlapping blocks for kinship verification

Siamese convolutional neural network and fusion of the best overlapping blocks for kinship verification

Facial Kinship Verification: A Comprehensive Review and Outlook

The goal of Facial Kinship Verification (FKV) is to automatically determine whether two individuals have a kin relationship or not from their given facial images or videos. It is an emerging and challenging problem that has attracted increasing attention due to its practical applications. Over the past decade, significant progress has been achieved in this new field. Handcrafted features and deep learning techniques have been widely studied in FKV. The goal of this paper is to conduct a comprehensive review of the problem of FKV. We cover different aspects of the research, including problem definition, challenges, applications, benchmark datasets, a taxonomy of existing methods, and state-of-the-art performance. In retrospect of what has been achieved so far, we identify gaps in current research and discuss potential future research directions.

Knowledge-based tensor subspace analysis system for kinship verification

Most existing automatic kinship verification methods focus on learning the optimal distance metrics between family members. However, learning facial features and kinship features simultaneously may cause the proposed models to be too weak. In this work, we explore the possibility of bridging this gap by developing knowledge-based tensor models based on pre-trained multi-view models. We propose an effective knowledge-based tensor similarity extraction framework for automatic facial kinship verification using four pre-trained networks (i.e., VGG-Face, VGG-F, VGG-M, and VGG-S). Therefore, knowledge-based deep face and general features (such as identity, age, gender, ethnicity, expression, lighting, pose, contour, edges, corners, shape, etc.) were successfully fused by our tensor design to understand the kinship cue. Multiple effective representations are learned for kinship verification statements (children and parents) using a margin maximization learning scheme based on Tensor Cross-view Quadratic Exponential Discriminant Analysis. Through the exponential learning process, the large gap between distributions of the same family can be reduced to the maximum, while the small gap between distributions of different families is simultaneously increased. The WCCN metric successfully reduces the intra-class variability problem caused by deep features. The explanation of black-box models and the problems of ubiquitous face recognition are considered in our system. The extensive experiments on four challenging datasets show that our system performs very well compared to state-of-the-art approaches.

FuseKin: Weighted image fusion based kinship verification under unconstrained age group

Kinship verification using facial images is mainly performed with a single face sample per person. To perform with a single sample, it is very difficult to specify an age group where kin pairs may have higher similarities. To address the above problem, we propose a novel weighted multi sample fusion (WMSF) method. The proposed WMSF method combines kin signals present in multiple samples per person (MSPP) to form a FuseKin image. To select the most discriminant features from the extracted feature vector, we propose a patch based discriminative analysis (PDA) method. Weights are calculated using the PDA method so as to reduce the discrimination between positive FuseKin pairs. Experiments were conducted on two different datasets which contain multiple face image samples per person, namely Family101 and Family in the Wild (FIW) to validate the performance of the proposed methods. Our method achieves competitive results as compared to other state-of-the-art methods.

A novel factor analysis-based metric learning method for kinship verification

A novel factor analysis-based metric learning method for kinship verification

Age-Invariant Adversarial Feature Learning for Kinship Verification

Kinship verification aims to determine whether two given persons are blood relatives. This technique can be leveraged in many real-world scenarios, such as finding missing people, identification of kinship in forensic medicine, and certain types of interdisciplinary research. Most existing methods extract facial features directly from given images and examine the full set of features to verify kinship. However, most approaches are easily affected by the age gap among faces, with few methods taking age into account. This paper accordingly proposes an Age-Invariant Adversarial Feature learning module (AIAF), which is capable of factoring in full facial features to create two uncorrelated components, i.e., identity-related features and age-related features. More specifically, we harness a type of adversarial mechanism to make the correlation between these two components as small as possible. Moreover, to pay different attention to identity-related features, we present an Identity Feature Weighted module (IFW). Only purified identity features are fed into the IFW module, which can assign different weights to the features according to their importance in the kinship verification task. Experimental results on three public popular datasets demonstrate that our approach is able to capture useful age-invariant features, i.e., identity features, and achieve significant improvements compared with other state-of-the-art methods on both small-scale and large-scale datasets.

Kinship verification and recognition based on handcrafted and deep learning feature-based techniques.

Background and ObjectivesKinship verification and recognition (KVR) is the machine’s ability to identify the genetic and blood relationship and its degree between humans’ facial images. The face is used because it is one of the most significant ways to recognize each other. Automatic KVR is an interesting area for investigation. It greatly affects real-world applications, such as searching for lost family members, forensics, and historical and genealogical studies. This paper presents a comprehensive survey that describes KVR applications and kinship types. It presents a literature review of current studies starting from handcrafted passing through shallow metric learning and ending with deep learning feature-based techniques. Furthermore, kinship mostly used datasets are discussed that in turn open the way for future directions for the research in this field. Also, the KVR limitations are discussed, such as insufficient illumination, noise, occlusion, and age variations problems. Finally, future research directions are presented, such as age and gender variation problems.MethodsWe applied a literature survey methodology to retrieve data from academic databases. An inclusion and exclusion criteria were set. Three stages were followed to select articles. Finally, the main KVR stages, along with the main methods in each stage, were presented. We believe that surveys can help researchers easily to detect areas that require more development and investigation.ResultsIt was found that handcrafted, metric learning, and deep learning were widely utilized in kinship verification and recognition problem using facial images.ConclusionsDespite the scientific efforts that aim to address this hot research topic, many future research areas require investigation, such as age and gender variation. In the end, the presented survey makes it easier for researchers to identify the new areas that require more investigation and research.

Deep discriminant generation-shared feature learning for image-based kinship verification

Kinship verification based on facial images in the wild is an interesting and challenging research topic in the fields of computer vision. It has many potential applications, e.g., missing children search, image annotation, etc. In practice, the biggest obstacle in kinship verification is that there usually exists large divergence between the images of parent and children. How to effectively tackle such challenge is important for improving kinship verification performance. To this end, we proposed a novel Deep Discriminant Generation-Shared Feature Learning for Image-Based Kinship Verification (D2GFL) method for kinship verification, which consists of a two-stream generation-specific feature learning module and a generation-shared discrepancy reducing module. The generation-specific feature learning module can learn parent-specific and child-specific features. The generation-shared module aims to reduce the divergence between facial images of parent and child. In order to further relieve the cross-generation difference and improve the discriminability of learned features, we also design a cross-generation difference loss term and an intra-generation discriminant loss term, which simultaneously make use of the local and holistic features, as well as the family identity information. Experimental results on several widely used kinship datasets are presented to validate the effectiveness of our proposed approach by comparing with the state-of-the-art kinship verification methods. Specially, our approach can improve the average verification accuracy at least by 3.5%, 1.5% and 0.8% on KinFaceW-I, Cornell, and TSKinFace datasets, respectively.

Component-based metric learning for fully automatic kinship verification

This paper introduces a fully automatic method for kinship verification from facial images. Recently, a number of methods have been proposed to verify kinship from facial images, however, most of these methods are needed to exactly align face images before feature extraction in a manual manner. Unlike these methods, our method does not depend on face alignment. Firstly, we localize several facial feature points by utilizing a facial feature detector to extract SIFT descriptor around each feature point of a face image. Lastly, two ways, feature combination and distance metric learning, are used to verify the kinship of a pair of face images. For feature combination, three simple strategies of feature combination and support vector machine classifier are used for kinship verification. For metric learning, we propose a component-based metric learning (CML) method to measure the distance of each face pair, which jointly learns multiple local distance metrics, and one specific distance metric for each facial feature point. Experimental results show the effectiveness of our proposed approach on two popular kinship datasets.

Robust discriminative feature subspace analysis for kinship verification

Kinship verification using single image of a person is a challenging task for real-world applications. In this paper, we propose a novel robust discriminative feature subspace analysis (RDFSA) method to address single sample per person (SSPP) problem in kinship verification. The proposed RDFSA method takes advantages of facial symmetry and patch-based analysis to extract discriminative features for kinship verification. Each face image is firstly divided into two halves about bilateral symmetry axis, and each halved face is then partitioned into equal sized non-overlapping patches. Multiple image-sets are formed by grouping these patches according to their positions at each halved face. Then, an SSPP is formulated as an RDFSA problem and a feature subspace is learned by maximizing inter-class separation and minimizing intra-class variance for different patches in each image-set. For a given test image pair, similarity is computed for each feature subspace and majority voting strategy is employed to determine if a given image pair is kin related or not. Proposed RDFSA method is extensively evaluated on different publicly available kinship datasets to validate kinship accuracy. Experimental results show that RDFSA achieves competitive accuracy on all kinship datasets while performing kinship verification under unconstrained environment.