Intra-class Samples Research Articles

Deep low-rank feature learning and encoding for cross-age face recognition

Cross-age face recognition (CAFR) is a challenging task, due to significant intra-personal variations. Furthermore, the training and testing data may contain random noise components. To address these issues, this paper proposes a deep low-rank feature learning and encoding method. Firstly, our method employs manifold learning in the low-rank optimization, which preserves the global and local structure of the data samples, while learning the clean low-rank features. Secondly, we encode the low-rank features using our locality-constrained feature encoding method, which learns an age-insensitive codebook from training data, and enables the intra-class samples to share the same local bases in a codebook. In the testing stage, the gallery and probe features are encoded by the learned codebook, which represents the images of the same identity by similar codewords for recognition. Furthermore, the periocular region of human faces is investigated for CAFR. Extensive experiments on five datasets demonstrate the effectiveness of our method.

Large Margin Graph Embedding-Based Discriminant Dimensionality Reduction

Discriminant graph embedding-based dimensionality reduction methods have attracted more and more attention over the past few decades. These methods construct an intrinsic graph and penalty graph to preserve the intrinsic geometry structures of intraclass samples and separate the interclass samples. However, the marginal samples cannot be accurately characterized only by penalty graphs since they treat every sample equally. In practice, these marginal samples often influence the classification performance, which needs to be specially tackled. In this study, the near neighbors’ hypothesis margin of marginal samples has been further maximized to separate the interclass samples and improve the discriminant ability by integrating intrinsic graph and penalty graph. A novel discriminant dimensionality reduction named LMGE-DDR has been proposed. Several experiments on public datasets have been conducted to verify the effectiveness of the proposed LMGE-DDR such as ORL, Yale, UMIST, FERET, CMIU-PIE09, and AR. LMGE-DDR performs better than other compared methods, and the corresponding standard deviation of LMGE-DDR is smaller than others. This demonstrates that the evaluation method verifies the effectiveness of the introduced method.

Dual-Constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior

Nonnegativity based matrix factorization is usually powerful for learning the parts-based “shallow” representation, however it fails to discover deep hidden information within both the basis concept and representation spaces. In this paper, we therefore propose a new dual-constrained deep semi-supervised coupled factorization network (DS2CF-Net) for learning hierarchical representations. DS2CF-Net is formulated as the joint partial-label and structure-constrained deep factorization network using multi-layers of linear transformations, which coupled updates the basic concepts and new representations in each layer. An error correction mechanism with feature fusion strategy is also integrated between consecutive layers to improve the representation ability of features. To improve the discriminating abilities of both representation and coefficients in feature space, we clearly consider how to enrich the prior knowledge by the coefficients-based label prediction, and incorporate the enriched prior knowledge as the additional label and structure constraints. To be specific, the label constraint enables the intra-class samples to have the same coordinate in the feature space, while the structure constraint forces the coefficients in each layer to be block-diagonal so that the enriched prior knowledge are more accurate. Besides, we integrate the adaptive dual-graph learning to retain the locality structures of both the data manifold and feature manifold in each layer. Finally, a fine-tuning process is performed to refine the structure-constrained matrix and data weight matrix in each layer using the predicted labels for more accurate representations. Extensive simulations on public databases show that our method can obtain state-of-the-art performance.

Spectral-Locational-Spatial Manifold Learning for Hyperspectral Images Dimensionality Reduction

Dimensionality reduction (DR) plays an important role in hyperspectral image (HSI) classification. Unsupervised DR (uDR) is more practical due to the difficulty of obtaining class labels and their scarcity for HSIs. However, many existing uDR algorithms lack the comprehensive exploration of spectral-locational-spatial (SLS) information, which is of great significance for uDR in view of the complex intrinsic structure in HSIs. To address this issue, two uDR methods called SLS structure preserving projection (SLSSPP) and SLS reconstruction preserving embedding (SLSRPE) are proposed. Firstly, to facilitate the extraction of SLS information, a weighted spectral-locational (wSL) datum is generated to break the locality of spatial information extraction. Then, a new SLS distance (SLSD) excavating the SLS relationships among samples is designed to select effective SLS neighbors. In SLSSPP, a new uDR model that includes a SLS adjacency graph based on SLSD and a cluster centroid adjacency graph based on wSL data is proposed, which compresses intraclass samples and approximately separates interclass samples in an unsupervised manner. Meanwhile, in SLSRPE, for preserving the SLS relationship among target pixels and their nearest neighbors, a new SLS reconstruction weight was defined to obtain the more discriminative projection. Experimental results on the Indian Pines, Pavia University and Salinas datasets demonstrate that, through KNN and SVM classifiers with different classification conditions, the classification accuracies of SLSSPP and SLSRPE are approximately 4.88%, 4.15%, 2.51%, and 2.30%, 5.31%, 2.41% higher than that of the state-of-the-art DR algorithms.

Metric learning for image-based flower cultivars identification

BackgroundThe study of plant phenotype by deep learning has received increased interest in recent years, which impressive progress has been made in the fields of plant breeding. Deep learning extremely relies on a large amount of training data to extract and recognize target features in the field of plant phenotype classification and recognition tasks. However, for some flower cultivars identification tasks with a huge number of cultivars, it is difficult for traditional deep learning methods to achieve better recognition results with limited sample data. Thus, a method based on metric learning for flower cultivars identification is proposed to solve this problem.ResultsWe added center loss to the classification network to make inter-class samples disperse and intra-class samples compact, the script of ResNet18, ResNet50, and DenseNet121 were used for feature extraction. To evaluate the effectiveness of the proposed method, a public dataset Oxford 102 Flowers dataset and two novel datasets constructed by us are chosen. For the method of joint supervision of center loss and L2-softmax loss, the test accuracy rate is 91.88%, 97.34%, and 99.82% across three datasets, respectively. Feature distribution observed by T-distributed stochastic neighbor embedding (T-SNE) verifies the effectiveness of the method presented above.ConclusionsAn efficient metric learning method has been described for flower cultivars identification task, which not only provides high recognition rates but also makes the feature extracted from the recognition network interpretable. This study demonstrated that the proposed method provides new ideas for the application of a small amount of data in the field of identification, and has important reference significance for the flower cultivars identification research.

Partial-Label and Structure-constrained Deep Coupled Factorization Network

In this paper, we technically propose an enriched prior guided framework, called Dual-constrained Deep Semi-Supervised Coupled Factorization Network (DS2CF-Net), for discovering hierarchical coupled data representation. To extract hidden deep features, DS2CF-Net is formulated as a partial-label and geometrical structure-constrained framework. Specifically, DS2CF-Net designs a deep factorization architecture using multilayers of linear transformations, which can coupled update both the basis vectors and new representations in each layer. To enable learned deep representations and coefficients to be discriminative, we also consider enriching the supervised prior by joint deep coefficients-based label prediction and then incorporate the enriched prior information as additional label and structure constraints. The label constraint can enable the intra-class samples to have same coordinate in feature space, and the structure constraint forces the coefficients in each layer to be block-diagonal so that the enriched prior using the self-expressive label propagation are more accurate. Our network also integrates the adaptive dual-graph learning to retain the local structures of both data and feature manifolds in each layer. Extensive experiments on image datasets demonstrate the effectiveness of DS2CF-Net for representation learning and clustering.

Robust Video-Based Person Re-Identification by Hierarchical Mining

Video-based person re-identification (Re-ID) aims at retrieving the person through the video sequences across non-overlapping cameras. Some characteristics of pedestrians are not consecutive across frames due to the variations of viewpoints, postures, and occlusions over time. However, existing methods ignore such data peculiarity and the networks tend to only learn those salient consecutive characteristics among frames in video sequences. As a result, the learned representations fail to cover all the characteristics of pedestrians, thus lacking integrity and discrimination. To tackle this problem, we present a novel deep architecture termed Hierarchical Mining Network (HMN), which mines as many pedestrians’ characteristics by referring to the temporal and intra-class knowledge. It consists of a novel Attentive Temporal Module (ATM) and a Dynamic Supervising Branch (DSB), with a Balancing Triplet Loss (BTL) assisting the training. The proposed ATM, with pedestrian perceiving capacity, is capable of evaluating each activation of features through temporal analysis, so that the temporally scattered characteristics of pedestrians can be better aggregated and the contaminated ones can be eliminated. Then, the DSB along with the BTL further enhances the integrity of representations by multiple supervision. Specifically, the DSB perceives the diversities of intra-class samples in each mini-batch and generates targeted supervising signals for them, in which process the BTL guarantees the signals with smaller intra-class variations and larger inter-class variations. Comprehensive experiments on two video-based datasets, i.e., MARS, and DukeMTMC-VideoReID, demonstrate the contribution of each component and the superiority of the proposed HMN over the state-of-the-arts. Benchmarking our model on three popular image-based datasets, i.e., Market1501, DukeMTMC-Reid, and MSMT17 additionally verifies the promising generalizability of the proposed DSB and BTL.

Intra-class low-rank regularization for supervised and semi-supervised cross-modal retrieval

Cross-modal retrieval aims to retrieve related items across different modalities, for example, using an image query to retrieve related text. The existing deep methods ignore both the intra-modal and inter-modal intra-class low-rank structures when fusing various modalities, which decreases the retrieval performance. In this paper, two deep models (denoted as ILCMR and Semi-ILCMR) based on intra-class low-rank regularization are proposed for supervised and semi-supervised cross-modal retrieval, respectively. Specifically, ILCMR integrates the image network and text network into a unified framework to learn a common feature space by imposing three regularization terms to fuse the cross-modal data. First, to align them in the label space, we utilize semantic consistency regularization to convert the data representations to probability distributions over the classes. Second, we introduce an intra-modal low-rank regularization, which encourages the intra-class samples that originate from the same space to be more relevant in the common feature space. Third, an inter-modal low-rank regularization is applied to reduce the cross-modal discrepancy. To enable the low-rank regularization to be optimized using automatic gradients during network back-propagation, we propose the rank-r approximation and specify the explicit gradients for theoretical completeness. In addition to the three regularization terms that rely on label information incorporated by ILCMR, we propose Semi-ILCMR in the semi-supervised regime, which introduces a low-rank constraint before projecting the general representations into the common feature space. Extensive experiments on four public cross-modal datasets demonstrate the superiority of ILCMR and Semi-ILCMR over other state-of-the-art methods.

Deep center-based dual-constrained hashing for discriminative face image retrieval

With the advantages of low storage cost and extremely fast retrieval speed, deep hashing methods have attracted much attention for image retrieval recently. However, large-scale face image retrieval with significant intra-class variations is still challenging. Neither existing pairwise/triplet labels-based nor softmax classification loss-based deep hashing works can generate compact and discriminative binary codes. Considering these issues, we propose a center-based framework integrating end-to-end hashing learning and class centers learning simultaneously. The framework minimizes the intra-class variance by clustering intra-class samples into a learnable class center. To strengthen inter-class separability, it additionally imposes a novel regularization term to enlarge the Hamming distance between pairwise class centers. Moreover, a simple yet effective regression matrix is introduced to encourage intra-class samples to generate the same binary codes, which further enhances the hashing codes compactness. Experiments on four large-scale datasets show the proposed method outperforms state-of-the-art baselines under various code lengths and commonly-used evaluation metrics.

Discriminant analysis based on reliability of local neighborhood

To obtain a compact and effective low-dimensional representation, recently, most existing discriminant manifold learning methods have integrated manifold learning into discriminant analysis (DA) for extracting the intrinsic structure of data. These methods learn two kinds of adjacency graphs, such as intrinsic graph and penalty graph, to characterize the similarity between samples from intraclass and the pseudo similarity of interclass. However, they treat every sample equally, which results in the following defects: (1) These methods cannot accurately characterize the marginal region among different classes only through penalty graphs. (2) They can not identify the noisy and outlier samples which reduce the robustness of these methods. To address these problems, we introduce an adaptive adjacency factor to perform the discriminative based reliability analysis for each sample. By integrating the adjacency factor into discriminant manifold learning methods, we propose a novel method for DA namely discriminant analysis based on reliability of local neighborhood (DA-RoLN). We mainly have three contributions in this paper: (1) By the introduction of adjacency factor, sample points can be divided into three parts: intraclass samples, marginal samples, and outliers. Therefore, DA-RoLN emphasizes the effect of valid samples and filters the influence of outliers. (2) We adaptively calculate the adjacency factor in low-dimensional space, thus, the margin between different classes in low-dimensional space is emphasized. (3) An iterative algorithm is developed to solve the objective function of DA-RoLN, and it is easy to solve with a low computational cost. Extensive experimental results show the effectiveness of DA-RoLN.

Discriminative dictionary learning algorithm with pairwise local constraints for histopathological image classification.

Histopathological image contains rich pathological information that is valued for the aided diagnosis of many diseases such as cancer. An important issue in histopathological image classification is how to learn a high-quality discriminative dictionary due to diverse tissue pattern, a variety of texture, and different morphologies structure. In this paper, we propose a discriminative dictionary learning algorithm with pairwise local constraints (PLCDDL) for histopathological image classification. Inspired by the one-to-one mapping between dictionary atom and profile, we learn a pair of discriminative graph Laplacian matrices that are less sensitive to noise or outliers to capture the locality and discriminating information of data manifold by utilizing the local geometry information of category-specific dictionaries rather than input data. Furthermore, graph-based pairwise local constraints are designed and incorporated into the original dictionary learning model to effectively encode the locality consistency with intra-class samples and the locality inconsistency with inter-class samples. Specifically, we learn the discriminative localities for representations by jointly optimizing both the intra-class locality and inter-class locality, which can significantly improve the discriminability and robustness of dictionary. Extensive experiments on the challenging datasets verify that the proposed PLCDDL algorithm can achieve a better classification accuracy and powerful robustness compared with the state-of-the-art dictionary learning methods. Graphical abstract The proposed PLCDDL algorithm. 1) A pair of graph Laplacian matrices are first learned based on the class-specific dictionaries. 2) Graph-based pairwise local constraints are designed to transfer the locality for coding coefficients. 3) Class-specific dictionaries can be further updated.

Cosmos-Loss: A Face Representation Approach With Independent Supervision

Cost function is one of the most important topics in face recognition. Classic methods based on anchor-positive-negative sample pairs directly or indirectly have been proved to be effective. Taking advantage of information from sample pair with labels is the key point to make intra-class samples more centralized and inter-class ones more scattered. However, there are some problems to constrain model in these classical approaches, such as sample-pair-combination and the poor information used in parameter updating. This paper proposes a novel method named Cosmos-Loss based on the principle of feature center. It divides the feature space into many subspaces. There is a one-to-one match between each subspace and face identity. All samples belonged to the same subspace decide their feature center. This proposed method uses the information between sample and its own feature center to make inner features more similar and the information between sample and other center to make intra features more dissimilar, instead of the poor information between only two samples. This supervision method could use the information of all samples which belonged to the same center indirectly and make feature distribution smoother. And all experiments show that this proposed method is an effective way to supervise model training independently.

A weighted exponential discriminant analysis through side-information for face and kinship verification using statistical binarized image features

Side-information based exponential discriminant analysis (SIEDA) is more efficient than side-information based linear discriminant analysis (SILDA) in computing the discriminant vectors because it maximizes the Fisher criterion function. In this paper, we develop a novel criterion, named side-information based weighted exponential discriminant analysis (SIWEDA), that is based on the classical SIEDA method. We reformulate and generalize the classical Fisher criterion function in order to maximize it, with the property to pull as close as possible the intra-class samples (within-class samples), and push and repulse away as far as possible the inter-class samples (between-class samples). Thus, SIWEDA selects the eigenvalues of high significance and eliminate those with less discriminative information. To reduce the feature vector dimensionality and lighten the class intra-variability, we use SIWEDA and within class covariance normalization (WCCN) using the proposed statistical binarized image features (StatBIF). Moreover, we use score fusion strategy to extract the complementarity of different weighting scales of our StatBIF descriptor. We conducted experiments to evaluate the performance of the proposed method under unconstrained environment, using five datasets namely LFW, YTF, Cornell KinFace, UB KinFace and TSKinFace datasets, in the context of matching faces and kinship verification in the wild conditions. The experiments showed that the proposed approach outperforms the current state of the art. Very interestingly, our approach showed superior performance compared to methods based on deep metric learning.

Improved prototypical networks for few-Shot learning

Abstract Few-Shot Learning (FSL) aims at recognizing the target classes that only a few samples are available for training. The current approaches mostly address FSL by learning a generalized class-level metric while neglect the intra-class distribution information. In this work, we propose Improved Prototypical Networks (IPN) to address this issue. Inspired by the observation that the intra-class samples differ greatly in revealing the class distribution, we first propose an attention-analogous strategy to explore the class distribution information by distributing different weights to samples based on their representativeness. Besides, to further explore the discriminative information across classes, we propose a distance scaling strategy to reduce the intra-class difference while enlarge the inter-class difference. The experimental results on two benchmark datasets show the superiority of the proposed model against the state-of-the-art approaches.

Orthogonal neighborhood preserving discriminant analysis with patch embedding for face recognition

Intuitively, all facial images of a person are located on or near a manifold in the high-dimensional image space, and the process of face recognition can be regarded as the recovery process of multiple low-dimensional manifolds. To preserve the manifold structure information of intra-class samples after dimensionality reduction, we proposed a patch-based multi-manifold orthogonal neighborhood-preserving discriminant analysis algorithm, namely ONPDA. From the perspective of path alignment, we consider the intra-class compactness, intra-class structure and inter-class separability simultaneously. Moreover, we infuse intra-class structure information described by the sample reconstruction into intra-class compactness loss, considering the compactness of two reconstruction groups instead of sample pairs in the same class. By analyzing the relationship between the projection direction and the maximum inter-class margin, we select the samples that should participate in the inter-class separability on the patch. Meanwhile, a fast orthogonalization method is performed to obtain the orthogonal projection matrix. Besides, we perform ONPDA in reproducing kernel Hilbert space which gives rise to nonlinear maps, resulting in the kernel ONPDA (KONPDA). Experimental results compared with some state-of-the-art methods on a toy dataset and several benchmark face image databases demonstrate the effectiveness of ONPDA and KONPDA.

Compact convolutional autoencoder for SAR target recognition

Learning discriminative features is difficult for deep learning-based target recognition in synthetic aperture radar (SAR) images with small training samples. To achieve a better feature learning, this study proposes a new deep network, a compact convolutional autoencoder (CCAE) for SAR target recognition. CCAE minimises the reconstruction loss and the distance between intra-class samples simultaneously by imposing compactness constraint on the encoder, which results in a more discriminative feature representation. Furthermore, the pretrained CCAE encoder can be used to initialise the corresponding parameters of a convolutional neural network to facilitate the training of the end-to-end model. Experimental results using the moving and stationary target acquisition and recognition dataset show that the proposed method outperforms the existing deep learning-based methods in the case of small training samples.

Hyperspectral Image Dimensionality Reduction via Graph Embedding in Core Tensor Space

This letter tries to effectively reduce the dimension of hyperspectral images (HSIs) by jointly considering both the spectral redundancy and spatial continuity through a multilinear transformation with graph embedding in core tensor space. The whole process is constructed in the framework of Tucker decomposition (TD). Since the distance between intraclass samples should be relatively smaller than that of the interclass samples, the reduced tensor cores should maintain this property. To achieve this goal, a graph is embedded to the core tensor space during TD. Moreover, considering the unstability of solution of the previous works, we constrain the projected matrices by orthogonality so that the results can be more stable and the extracted features can be more discriminative. We further analyze the effect of different constrains to TD methods for HSI dimensionality reduction. Finally, the experimental results show the superiority of this method to many other tensor methods.

Dimensionality Reduction With Enhanced Hybrid-Graph Discriminant Learning for Hyperspectral Image Classification

Dimensionality reduction (DR) is an important way of improving the classification accuracy of a hyperspectral image (HSI). Graph learning, which can effectively reveal the intrinsic relationships of data, has been widely used in the case of HSIs. However, most of them are based on a simple graph to represent the binary relationships of data. An HSI contains complex high-order relationships among different samples. Therefore, in this article, we propose a hybrid-graph learning method to reveal the complex high-order relationships of the HSI, termed enhanced hybrid-graph discriminant learning (EHGDL). In EHGDL, an intraclass hypergraph and an interclass hypergraph are constructed to analyze the complex multiple relationships of a HSI. Then, a supervised locality graph is applied to reveal the binary relationships of a HSI which can form the complementarity of a hypergraph. Simultaneously, we also construct a weighted neighborhood margin model to boost the difference of samples from different classes. Finally, we design a DR model based on the intraclass hypergraph, the interclass hypergraph, the supervised locality graph, and the weighted neighborhood margin to improve the compactness of the intraclass samples and the separability of the interclass samples, and an optimal projection matrix can be achieved to extract the low-dimensional embedding features of the HSI. To demonstrate the effectiveness of the proposed method, experiments have been conducted on the Indian Pines, PaviaU, and HoustonU data sets. The experimental results show that EHGDL can generate better classification performance compared with some related DR methods. As a result, EHGDL can better reveal the complex intrinsic relationships of a HSI by the complementarity of different characteristics and enhance the discriminant performance of land-cover types.

Efficient and Accurate 3D Finger Knuckle Matching using Surface Key Points.

Contactless 3D finger knuckle is a new biometric identifier with a lot of potentials, which can provide an accurate, efficient and convenient alternative for the personal identification. The current 3D finger knuckle recognition methods are limited by computationally complex or inefficient matching algorithms, which attempt to compute the matching scores from all possible translational and rotational parameters for matching a pair of templates. The strength of such approach lies in its simplicity and reliability for accurately matching intra-class samples, but expensive computational time is required. Furthermore, attempting on excessive numbers of translational and rotational parameters can also degrade the overall recognition accuracy because the imposter matches can be increased. In fact, this conventional matching approach is commonly adopted in many biometric studies, but its drawbacks have not received adequate attention. This paper addresses such 3D finger knuckle recognition problem by developing a more efficient matching approach using surface key points extracted from 3D finger knuckle surfaces. Our comparative experimental results with the state-of-the art method on a publicly available 3D finger knuckle database indicates that our approach can offer over 23 times faster with performance improvement on the accuracy. Although the focus of our work is on 3D finger knuckle recognition, we also present the performance of our method on other publicly available databases with similar 3D biometric patterns including 3D palmprint and 3D fingerprint, to validate the effectiveness of the proposed approach.

Multi-Task Deep Metric Learning with Boundary Discriminative Information for Cross-Age Face Verification

Image based face verification has attracted extension attention in the fields of pattern recognition and intelligent vision. With difference in age, cross-age face verification from facial images remains a challenging work because of a large number of facial variations caused by shape, skin color and wrinkles and so on. This study proposes a multi-task deep metric learning with boundary discriminative information method called MDML-BDI. It learns a distance metric by exploring discriminative information among the interclass neighborhood samples, such that the distances between intraclass samples are as small as possible and that between interclass neighborhood samples are as far as possible. MDML-BDI learns hierarchical nonlinear transformations by integrating metric learning into the framework of multi-task deep neural network, such that a common shared layer shares the common transformation by multiple tasks, and the other independent layers learn individual task-special transformation for each task. Experimental results on FG-NET, CACD-VS and CALFW datasets show that MDML-BDI achieves satisfactory performance in terms of accuracy and receiver operating characteristic (ROC) curve.