Face Clustering Research Articles

A novel two-stage omni-supervised face clustering algorithm

A novel two-stage omni-supervised face clustering algorithm

Block diagonal representation learning with local invariance for face clustering

Block diagonal representation learning with local invariance for face clustering

Accelerated Stochastic Variance Reduction Gradient Algorithms for Robust Subspace Clustering.

Robust face clustering enjoys a wide range of applications for gate passes, surveillance systems and security analysis in embedded sensors. Nevertheless, existing algorithms have limitations in finding accurate clusters when data contain noise (e.g., occluded face clustering and recognition). It is known that in subspace clustering, the ℓ1- and ℓ2-norm regularizers can improve subspace preservation and connectivity, respectively, and the elastic net regularizer (i.e., the mixture of the ℓ1- and ℓ2-norms) provides a balance between the two properties. However, existing deterministic methods have high per iteration computational complexities, making them inapplicable to large-scale problems. To address this issue, this paper proposes the first accelerated stochastic variance reduction gradient (RASVRG) algorithm for robust subspace clustering. We also introduce a new momentum acceleration technique for the RASVRG algorithm. As a result of the involvement of this momentum, the RASVRG algorithm achieves both the best oracle complexity and the fastest convergence rate, and it reaches higher efficiency in practice for both strongly convex and not strongly convex models. Various experimental results show that the RASVRG algorithm outperformed existing state-of-the-art methods with elastic net and ℓ1-norm regularizers in terms of accuracy in most cases. As demonstrated on real-world face datasets with different manually added levels of pixel corruption and occlusion situations, the RASVRG algorithm achieved much better performance in terms of accuracy and robustness.

Preparation and Mechanism of Calcium Carbonate Whiskers from DoLOMITE Refined Solution

AbstractThe preparation of calcium carbonate whiskers by gas‐liquid contact method using low‐grade dolomite refining solution and CO2 as raw materials has attracted widespread attention. The effects of reaction temperature, Mg2+ concentration and pH value on the morphology, particle size, aspect ratio and crystal form of CaCO3（Calcium carbonate） whiskers are investigated in detail. SEM (Scanning Electron Microscope) and XRD (X‐ray powder diffraction) are combined to analyze the calcium carbonate whiskers. The results demonstrated that under the conditions such as 100 °C, Mg2+ concentration of 0.05 mol L−1 and pH value of 9.5, calcium carbonate whiskers with uniform distribution, aspect ratio of 15–20, and purity of 99.38% can be prepared. Through Material Studios simulation software and critical nucleation energy analysis, it is confirmed that the morphology of calcium carbonate whiskers emerged to be a long hexagonal prism. Mg2+ in the refined solution would adhere to the surface of calcium carbonate during carbonization, inhibit the formation of calcite phase, and promote the growth of face clusters connected by vertex angles between CaCO3 crystals. The initial pH condition determines the solubility of CO32− and the supersaturation of the solution, which in turn affected the formed calcium carbonate crystal form.

Compound Weakly Supervised Clustering.

Clustering is a fundamental and important step in many image processing tasks, such as face recognition and image segmentation. The performance of clustering can be largely enhanced if relevant weak supervision information is appropriately exploited. To achieve this goal, in this paper, we propose the Compound Weakly Supervised Clustering (CSWC) method. Concretely, CSWC incorporates two types of widely available and easily accessed weak supervision information from the label and feature aspects, respectively. To be specific, at the label level, the pairwise constraints are utilized as a kind of typical weak label supervision information. At the feature level, the partial instances collected from multiple perspectives have internal consistency and they are regarded as weak structure supervision information. To achieve a more confident clustering partition, we learn a unified graph with its similarity matrix to incorporate the above two types of weak supervision. On one hand, this similarity matrix is constructed by self-expression across the partial instances collected from multiple perspectives. On the other hand, the pairwise constraints, i.e., must-links and cannot-links, are considered by formulating a regularizer on the similarity matrix. Finally, the clustering results can be directly obtained according to the learned graph, without performing additional clustering techniques. Besides evaluating CSWC on 7 benchmark datasets, we also apply it to the application of face clustering in video data since it has vast application potentiality. Experimental results demonstrate the effectiveness of our algorithm in both incorporating compound weak supervision and identifying faces in real applications.

The application of convolutional neural networks in face detection

Face detection is a popular and challenging issue which is widely studied in the past few decades. Its application includes the identity authentication, human machine interaction, security surveillance and social network. To have a better insight of the application of one of the typical deep learning algorithms called Convolutional Neural Network (CNN) in this field, this paper aims to analyze the current literature and progress about the face detection of low image quality and face detection optimization. The literature of Convolutional Neural Network from 2015 was included in this paper. Past research topics of face detection includes the occlusion, scale, small face cluster, speed, precision and multi-task region proposal network. The comparison between various deep learning-based methods in terms of the performance indicated that there is still no high robustness solution to all problems. The future research agendas of face detection based on the Convolutional Neural Network was also summarized.

Progressive structure enhancement graph convolutional network for face clustering

Face clustering, a technique for automatically annotating large-scale face data, has made significant advancements with the advent of graph convolutional networks (GCNs). Despite their success, GCNs can suffer from decreased performance due to conflicting information passed along noisy edges of a graph. To address this issue, we propose a novel framework named progressive structure enhancement GCN (PSE-GCN), which combines graph structure learning with graph-guided feature aggregation. Our PSE-GCN framework includes a dynamic graph construction (DGC) module that enhances local relationships and suppresses global noise, thereby improving the quality of the graph. By stacking multiple DGCs, PSE-GCN progressively refines the graph quality and yields discriminative features for various clustering tasks. Additionally, we introduce a subgraph-based neighborhood re-ranking (SNR) mechanism that improves graph homogeneity by rearranging the candidate neighbors of each face based on structural similarity at the subgraph level. Our experimental results, conducted on several popular benchmarks, not only demonstrate the effectiveness of PSE-GCN, but also show that it outperforms state-of-the-art methods, e.g., 93.50% in pairwise F-score on the MS-Celeb-1M dataset.

Plane-based clustering with asymmetric distribution loss

Ramp-based twin support vector clustering (RampTWSVC) is a powerful clustering method, which measures the within-cluster and between-cluster scatter by the bounded ramp functions. However, the ramp loss is a symmetric function and it does not consider the data distribution, which makes the RampTWSVC not robust enough to deal with noisy datasets. In this paper, we construct an asymmetric distribution loss function and propose a novel plane-based clustering with asymmetric distribution loss (ADPC). The asymmetric distribution loss function combined the ramp loss with first-order and second-order statistics, it inherits the advantages of ramp loss function and captures the data distribution precisely, which makes ADPC more robust to the samples far from the cluster center plane. Moreover, two adjusting parameters t1 and t2 are introduced to flexibly control the form of the asymmetric distribution loss function and improve the performance of the algorithm. In addition, its nonlinear clustering formation is also provided by using the kernel trick. Some experiments on noise-corrupted benchmark UCI and artificial datasets have been provided, and the detailed comparison results show that the proposed ADPC has better performance on most datasets. Further experiments and analysis on the face clustering have been done to demonstrate the effectiveness and feasibility of the proposed ADPC.

Similar Graph Face Clustering Based on Graph Convolutional Neural Network

Face clustering is primarily a method for grouping a large number of face images and has important applications in the fields of label-free face image annotation and image management. Traditional machine learning clustering algorithms do not work well on face image data and are unable to learn effectively on complex face image features. Recent research has turned to the use of graph convolutional neural networks (GCNs) to learn contextual information from neighbourhood features between face images for inference, which can significantly improve performance. Unlike the conventional link prediction of face data points and confidence prediction between vertices by using GCNs, for this paper a similarity graph based face clustering algorithm is proposed. Construction of subgraphs by K-nearest neighbour algorithm, Learn the similarity between the k-nearest neighbor subgraphs of each face instance by constructing subgraphs with the k-nearest neighbor algorithm. Using a bottom-up clustering strategy to merge subgraphs to complete the clustering, this algorithm improves the clustering results on complex face features by 20% and 7% compared to traditional machine learning methods such as K-means and DBSCAN respectively, improving the clustering accuracy and reducing the computational complexity.

Adapt-Infomap: Face clustering with adaptive graph refinement in infomap

Face clustering is a critical task in computer vision due to the increasing number of applications such as augmented reality or photo album management. The primary challenge in this task arises from the imperfections in image feature representations. Given image features extracted from an existing pre-trained representation model, it remains an unresolved problem that how to leverage the inherent characteristics of similarities among unlabelled images to improve the clustering performance. In order to solve face clustering in an unsupervised manner, we develop an effective and robust framework named as Adapt-Infomap. First, we reformulate face clustering as a process of non-overlapping community detection. Specially, Adapt-Infomap achieves face clustering by minimizing the entropy of information flows (also known as the map equation) on an affinity graph of images. Since the affinity graph of images might contain noisy edges, we develop an outlier detection strategy in Adapt-Infomap to adaptively refine the affinity graph. Experiments with ablation studies demonstrate that Adapt-Infomap significantly outperforms existing methods and achieves new state-of-the-arts on three popular large-scale datasets for face clustering, e.g., an absolute improvement of more than 10% and 3% comparing with prior unsupervised and supervised methods respectively in terms of average of Pairwise F-score.

Truth-value unconstrained face clustering for identity resolution in a distributed environment of criminal police information systems

The work of criminal police in contemporary society is characterized by the proliferation of data and information to be processed, a more significant limitation of access to personal data, increased public monitoring, and higher expectations in the efficiency of identifying perpetrators. Also, all information collected during investigations is distributed through police systems separated by different levels and organization units, and often there are insufficient human and material resources. Resolving the perpetrator’s identity in those circumstances is a complex task, and police decision support systems must group all available evidence related to specific persons. For that purpose, this paper proposes a new approach to unconstrained pairwise clustering of face feature vectors extracted from the histogram of oriented gradients descriptor, named Truth-value clustering (TVC), based on non-axiomatic logic and graphs. The clustering approach was experimentally tested with six different face image databases. They were created to simulate unconstrained conditions like IARPA Janus Benchmark-B Face Dataset (IJB-B), IMDb-Wiki, Labeled Faces in the Wild, and YouTube Faces. The results of the proposed solution are compared with other state-of-the-art methods, showing that the approach gives, in summary, significantly better results. Application of the IJB-B protocol created for testing face clustering showed that the new approach gives better results by an average of 8.25% (σ=4.2). The main advantage over other methods is the possibility of utilizing mechanisms from non-axiomatic logic such as revision, which can then acquire new knowledge based on information from different nodes of the distributed environment consisting of various police information systems.

Structure-enhanced pairwise feature learning for face clustering

Face clustering groups massive unlabeled face images according to their underlying identities and has proven to be a valuable tool for data analysis. Most recent studies have utilized graph convolutional networks (GCNs) to explore the structural properties of faces, thereby effectively achieving improved clustering performance. However, these methods usually suffer from computational intractability for large-scale graphs and tend to be sensitive to some postprocessing thresholds that serve to purify the clustering results. To address these issues, in this paper, we consider each pairwise relationship between two samples as a learning unit and infer clustering assignments by evaluating a group of pairwise connections. Specifically, we propose a novel clustering framework, named structure-enhanced pairwise feature learning (SEPFL), which mixes neighborhood information to adaptively produce pairwise representations for cluster identification. In addition, we design a combined density strategy to select representative pairs, thus ensuring training effectiveness and inference efficiency. The extensive experimental results show that SEPFL achieves better performance than other advanced face clustering techniques.

Self-Supervised Video-Centralised Transformer for Video Face Clustering.

This article presents a novel method for face clustering in videos using a video-centralised transformer. Previous works often employed contrastive learning to learn frame-level representation and used average pooling to aggregate the features along the temporal dimension. This approach may not fully capture the complicated video dynamics. In addition, despite the recent progress in video-based contrastive learning, few have attempted to learn a self-supervised clustering-friendly face representation that benefits the video face clustering task. To overcome these limitations, our method employs a transformer to directly learn video-level representations that can better reflect the temporally-varying property of faces in videos, while we also propose a video-centralised self-supervised framework to train the transformer model. We also investigate face clustering in egocentric videos, a fast-emerging field that has not been studied yet in works related to face clustering. To this end, we present and release the first large-scale egocentric video face clustering dataset named EasyCom-Clustering. We evaluate our proposed method on both the widely used Big Bang Theory (BBT) dataset and the new EasyCom-Clustering dataset. Results show the performance of our video-centralised transformer has surpassed all previous state-of-the-art methods on both benchmarks, exhibiting a self-attentive understanding of face videos.

STAR-FC: Structure-Aware Face Clustering on Ultra-Large-Scale Graphs.

Face clustering is a promising method for annotating unlabeled face images. Recent supervised approaches have boosted the face clustering accuracy greatly, however their performance is still far from satisfactory. These methods can be roughly divided into global-based and local-based ones. Global-based methods suffer from the limitation of training data scale, while local-based ones are inefficient for inference due to the use of numerous overlapped subgraphs. Previous approaches fail to tackle these two challenges simultaneously. To address the dilemma of large-scale training and efficient inference, we propose the STructure-AwaRe Face Clustering (STAR-FC) method. Specifically, we design a structure-preserving subgraph sampling strategy to explore the power of large-scale training data, which can increase the training data scale from 105 to 107. On this basis, a novel hierarchical GCN training paradigm is further proposed for better capturing the dynamic local structure. During inference, the STAR-FC performs efficient full-graph clustering with two steps: graph parsing and graph refinement. And the concept of node intimacy is introduced in the second step to mine the local structural information, where a calibration module is further proposed for fairer edge scores. The STAR-FC gets 93.21 pairwise F-score on standard partial MS1M within 312 seconds, which far surpasses the state-of-the-arts while maintaining high inference efficiency. Furthermore, we are the first to train on an ultra-large-scale graph with 20M nodes, and achieve superior inference results on 12M testing data. Overall, as a simple and effective method, the proposed STAR-FC provides a strong baseline for large-scale face clustering. Code is available in https://github.com/sstzal/STAR-FC.

A Novel Hybrid Multi-Modal Deep Learning for Detecting Hashtag Incongruity on Social Media.

Hashtags have been an integral element of social media platforms over the years and are widely used by users to promote, organize and connect users. Despite the intensive use of hashtags, there is no basis for using congruous tags, which causes the creation of many unrelated contents in hashtag searches. The presence of mismatched content in the hashtag creates many problems for individuals and brands. Although several methods have been presented to solve the problem by recommending hashtags based on the users' interest, the detection and analysis of the characteristics of these repetitive contents with irrelevant hashtags have rarely been addressed. To this end, we propose a novel hybrid deep learning hashtag incongruity detection by fusing visual and textual modality. We fine-tune BERT and ResNet50 pre-trained models to encode textual and visual information to encode textual and visual data simultaneously. We further attempt to show the capability of logo detection and face recognition in discriminating images. To extract faces, we introduce a pipeline that ranks faces based on the number of times they appear on Instagram accounts using face clustering. Moreover, we conduct our analysis and experiments on a dataset of Instagram posts that we collect from hashtags related to brands and celebrities. Unlike the existing works, we analyze these contents from both content and user perspectives and show a significant difference between data. In light of our results, we show that our multimodal model outperforms other models and the effectiveness of object detection in detecting mismatched information.

Hierarchical Deep Multitask Learning With the Attention Mechanism for Similarity Learning

Similarity learning is often adopted as an auxiliary task of deep multitask learning methods to learn discriminant features. Most existing approaches only use the single-layer features extracted by the last fully connected layer, which ignores the abundant information of feature channels in lower layers. Besides, small cliques are the most commonly used methods in similarity learning tasks to model the correlation of data, which can lead to the limited relation learning. In this article, we present an end-to-end hierarchical deep multitask learning framework for similarity learning which can learn more discriminant features by sharing information from different layers of network and dealing with complex correlation. Its main task is graph similarity inference. We build focus graphs for each sample. Then, an attention mechanism and a node feature enhancing model are introduced into backbone network to extract the abundant and important channel information from multiple layers of network. In the similarity inference task, a relation enhancing mechanism is applied to graph convolutional network to leverage the crucial relation in channels, which can effectively facilitate the learning ability of the whole framework. The extensive experiments have been conducted to demonstrate the effectiveness of the proposed method on person reidentification and face clustering applications.

Individual differences in looking at persons in scenes.

Individuals freely viewing complex scenes vary in their fixation behavior. The most prominent and reliable dimension of such individual differences is the tendency to fixate faces. However, much less is known about how observers distribute fixations across other body parts of persons in scenes and how individuals may vary in this regard. Here, we aimed to close this gap. We expanded a popular annotated stimulus set (Xu, Jiang, Wang, Kankanhalli, & Zhao, 2014) with 6,365 hand-delineated pixel masks for the body parts of 1,136 persons embedded in 700 complex scenes, which we publish with this article (https://osf.io/ynujz/). This resource allowed us to analyze the person-directed fixations of 103 participants freely viewing these scenes. We found large and reliable individual differences in the distribution of fixations across person features. Individual fixation tendencies formed two anticorrelated clusters, one for the eyes, head, and the inner face and one for body features (torsi, arms, legs, and hands). Interestingly, the tendency to fixate mouths was independent of the face cluster. Finally, our results show that observers who tend to avoid person fixations in general, particularly do so for the face region. These findings underscore the role of individual differences in fixation behavior and reveal underlying dimensions. They are further in line with a recently proposed push-pull relationship between cortical tuning for faces and bodies. They may also aid the comparison of special populations to general variation.

Subspace clustering via joint [formula omitted] and [formula omitted] norms

Some of the data collected from practical applications are usually heavily corrupted. In subspace clustering, the common method is to use the specific regularization strategy to correct these corrupted data by virtue of the prior knowledge, which could result in a suboptimal clustering solution. To alleviate the problem, we develop a novel formulation named subspace clustering via joint ℓ1,2 and ℓ2,1 (L12-21) norms (SCJL12-21). Specifically, we identify and exclude the heavily corrupted data points (unimportant data points) from participating in the linear representation of other points by imposing the ℓ1,2 norm on the representation matrix, and improve the robustness to outliers by enforcing the ℓ2,1 constraint on the error matrix. The joint ℓ1,2 and ℓ2,1 minimization leads to a good representation matrix which enhances the clustering performance. Related ℓ1,2 and ℓ2,1 norm constrained optimization problem is solved by utilizing the augmented Lagrange multiplier method. The effectiveness of the proposed method is demonstrated through experiments on the constructed datasets as well as the two practical problems of motion segmentation and face clustering.

A portable clustering algorithm based on compact neighbors for face tagging

We focus on the following problem: Given a collection of unlabeled facial images, group them into the individual identities where the number of subjects is not known. To this end, a Portable clustering algorithm based on Compact Neighbors called PCN is proposed. (1) Benefiting from the compact neighbor, the local density of each sample can be determined automatically and only one user-specified parameter, the number of nearest neighbors k, is involved in our model. (2) More importantly, the performance of PCN is not sensitive to the number of nearest neighbors. Therefore this parameter is relatively easy to determine in practical applications. (3) The computational overhead of PCN is O(nk(k2+log(nk))) that is nearly linear with respect to the number of samples, which means it is easily scalable to large-scale problems. In order to verify the effectiveness of PCN on the face clustering problem, extensive experiments based on a two-stage framework (extracting features using a deep model and performing clustering in the feature space) have been conducted on 16 middle- and 5 large-scale benchmark datasets. The experimental results have shown the efficiency and effectiveness of the proposed algorithm, compared with state-of-the-art methods. [code]

Efficient large-scale face clustering using an online Mixture of Gaussians

In recent years, the number of applications demanding real-time face clustering algorithms has increased, especially for security and surveillance purposes. However, state-of-the-art face clustering methods are offline, they need to repeat the whole clustering process every time new data arrives, and thus, they are not suitable for real-time applications. On the other hand, online clustering methods are highly dependent on the order and the size of the data, and they are less accurate than offline methods. To overcome these limitations, we present an online gaussian mixture-based clustering method (OGMC). The key idea of this method is the proposal that an identity can be represented by more than just one distribution or cluster. Using feature vectors extracted from the incoming faces, OGMC generates clusters that may be connected to others depending on their proximity and their robustness, and updates their connections every time their parameters are updated. With this approach, we reduce the dependency of the clustering process on the order and the size of the data and we are able to deal with complex data distributions. Experimental results show that OGMC outperforms state-of-the-art clustering methods on large-scale face clustering benchmarks not only in accuracy, but also in efficiency and scalability.