Local Geometric Structure Research Articles

Class-driven nonnegative matrix factorization with manifold regularization for data clustering

Nonnegative matrix factorization (NMF) is an effective technique to extract the underlying low-dimensional structure of data by utilizing its parts-based representation, which has been widely used in feature extraction and machine learning. However, NMF is an unsupervised learning algorithm without utilizing the discriminative prior information. In this paper, we put forward a new class-driven NMF with manifold regularization (MCDNMF) algorithm, which incorporates both the local manifold regularization and the label information of data into the NMF model. Specifically, MCDNMF not only encodes the local geometrical structure of data space by using the manifold regularization, but also takes the available label information by introducing the class-driven constraint. This class-driven constraint forces the new representations of data points to be more similar within the same class while different between other classes. Therefore, the discriminative abilities of clustering are greatly boosted. Experimental results on several datasets validate the effectiveness of proposed MCDNMF in comparison with the other state-of-the-art methods.

Explicitly Perceiving and Preserving the Local Geometric Structures for 3D Point Cloud Attack

Deep learning models for point clouds have shown to be vulnerable to adversarial attacks, which have received increasing attention in various safety-critical applications such as autonomous driving, robotics, and surveillance. Existing 3D attack methods generally employ global distance losses to implicitly constrain the point-wise perturbations for optimization. However, these simple losses are quite difficult to accurately measure and restrict the proper 3D geometry as point clouds are highly structured. Although few recent works try to exploit additional shape-aware surface knowledge to globally constrain the point position, they still fail to preserve the detailed point-to-point geometric dependency in different local regions. To this end, in this paper, we propose a novel Multi-grained Geometry-aware Attack (MGA), which explicitly captures the local topology characteristics in different 3D regions for adversarial constraint. Specifically, we first develop multi-scale spectral local filter banks adapting to different 3D object shapes to explore potential geometric structures in local regions. Considering that objects may contain complex geometries, we then extend each filter bank into multi-layer ones to gradually capture the topology contexts of the same region in a coarse-to-fine manner. Hence, the focused local geometric structures will be highlighted in the coefficients calculated by the filtering process. At last, by restricting these coefficients between benign and adversarial samples, our MGA is able to properly measure and preserve the detailed geometry contexts in the whole 3D object with trivial perturbations. Extensive experiments demonstrate that our attack can achieve superior performance on various 3D classification models, with satisfying adversarial imperceptibility and strong resistance to different defense methods.

LSFSR: Local label correlation-based sparse multilabel feature selection with feature redundancy

In recent studies, existing multilabel feature selection models have focused on either considering the relationship between labels or the redundancy between features. Furthermore, they only use simple sparsity constraints to process high-dimensional data without the intrinsic relationships between features and labels. These issues can have a great impact on the classification effectiveness of feature selection. To address these limitations, this article describes a new local label correlation-based sparse multilabel feature selection approach with feature redundancy. First, a new loss function is established among the matrices of samples, label coefficients, and labels. Then, the Frobenius norm is imposed to investigate the potential relationships between features and labels. The weight matrix is sparsified by the l2,1 norm to ensure that the new loss function has high interpretability. Second, a manifold constraint is employed to capture the local geometric structure between labels and to delve deeper into the latent information among the local labels. Then manifold constraints and Laplacian scores are combined for embedding feature selection to guide the exploration of hidden latent label. Finally, by considering the differences between the feature scores and the redundancy between the samples, feature redundancy is analyzed via the modified cosine similarity, and a candidate feature subset with low redundancy is generated. The l2 norm is used to select features with low redundancy while preserving sparsity, and a novel objective function is developed to optimize this solution. Thus, a sparse feature selection algorithm via local label correlation and feature redundancy is designed, and has demonstrated remarkable classification effectiveness in comparative experiments conducted on 21 multilabel datasets.

Activity trends of Pd clusters supported on C2N for oxygen evolution and reduction reactions

Developing highly efficient electrocatalysts for the oxygen evolution reaction (OER) and reduction reaction (ORR) is crucial for future renewable energy technology. Here, we use first-principles calculations combined with genetic algorithm to determine the structures of various Pd clusters supported on experimentally available C2N monolayer and evaluate the OER and ORR performance. Our findings show that the activity of the supported Pd clusters is closely linked to the local geometrical and electronic structure of the active site. Furthermore, we establish the activity trends of the clusters based on the adsorption free energies of intermediates. In particular, C2N supported Pd7 and Pd8 clusters exhibit outstanding OER activity with low overpotentials. We identify a volcano relation for the OER on the clusters, suggesting that the high activity of the cluster is related to the moderate adsorption strength of intermediates. Mechanistic analysis indicates that the second water formation is the potential-determining step for ORR on the clusters due to the strong adsorption of *OH. Additionally, we identify a linear scaling relationship between the ORR overpotentials and adsorption free energies of *OH, demonstrating that reducing the adsorption strength of reaction intermediates on Pd clusters can improve the activity. This work unravels the activity trends of cluster catalysts and provides strategies for the rational design of highly efficient single-cluster catalysts for OER and ORR.

Deep Nonnegative Matrix Factorization with Joint Global and Local Structure Preservation

Deep Non-Negative Matrix Factorization (DNMF) methods provide an efficient low-dimensional representation of given data through their layered architecture. A limitation of such methods is that they cannot effectively preserve the local and global geometric structures of the data in each layer. Consequently, a significant amount of the geometrical information within the data, present in each layer of the employed deep framework, can be overlooked by the model. This can lead to an information loss and a subsequent drop in performance. In this paper, we propose a novel deep non-negative matrix factorization method, Deep Non-Negative Matrix Factorization with Joint Global and Local Structure Preservation (dubbed Dn2MFGL), that ensures the preservation of both global and local structures within the data space. Dn2MFGL performs representation learning through a sequential embedding procedure which involves both the global data structure by accounting for the data variance, and the local data relationships by utilizing information from neighboring data points. Moreover, a regularization term that promotes sparsity by utilizing the concept of the inner product is applied to the matrices representing the lower dimensions. This aims to retain the fundamental data structure while discarding less crucial features. Simultaneously, the residual matrix of Dn2MFGL is subjected to the L2,1 norm, which ensures the robustness of the model against noisy data samples. An effective and multiplicative updating process also facilitates Dn2MFGL in solving the employed objective function. The clustering performance of the proposed deep NMF method is explored across various benchmarks of face datasets. The results point to Dn2MFGL outperforming several existing classical and state-of-the-art NMF methods. The source code is available at https://github.com/FaridSaberi/Dn2MFGO.git.

One-step multi-view clustering via deep-level semantics exploiting

Multi-view clustering (MVC) has gained promising performance improvement compared with traditional signal-view clustering due to the complementary information of multiple views. However, existing MVC methods exploit clustering structure by utilising signal-layer mapping, such that they cannot exploit the underlying deep-level semantic information in complex and interleaved multi-view data. Moreover, existing methods usually conduct multi-view fusion and clustering separately, which results in unpromising performance. To address the above problems, one-step MVC via deep-level semantics exploiting (DLSE) is proposed to exploit deep-level semantic information and learn the indicator matrix using a one-step manner. To be specific, a novel deep matrix factorisation (DMF) paradigm is designed to exploit the hierarchical semantics via a layer-wise scheme, so that samples from the same clusters are forced to be closer in the low-dimensional space layer by layer. Furthermore, to make the learned representation preserve the local geometric structure of data, DLSE introduces a local preservation regularisation to guide DMF. Meanwhile, by employing spectral rotating fusion, the cluster indicator can be obtained directly. Extensive experiments demonstrate the superiority of DLSE in contrast with some state-of-the-art methods.

Deep Learning of Partially Labeled Data for Quality Prediction Based on Stacked Target-Related Laplacian Autoencoder.

Partially labeled data, which is common in industrial processes due to the low sampling rate of quality variables, remains an important challenge in soft sensor applications. In order to exploit the information from partially labeled data, a target-related Laplacian autoencoder (TLapAE) is proposed in this work. In TLapAE, a novel target-related Laplacian regularizer is developed, which aims to extract structure-preserving and quality-related features by preserving the feature-target mapping according to the local geometrical structure of the data. In addition, stacked TLapAE (STLapAE) is further constructed to extract deep feature representations of the data by hierarchically stacking TLapAE blocks. For model training, backward propagation equations are derived based on matrix calculus techniques to update the model parameters of the proposed TLapAE. The effectiveness of the proposed STLapAE is evaluated using the butane content prediction case in a debutanizer column, the silicon content prediction case in a blast furnace (BF) ironmaking process, and the ethane concentration prediction case in an ethylene fractionator. The results show that the proposed TLapAE model has significantly improved prediction accuracy compared to soft sensors using only labeled data and other partially labeled data modeling methods.

Relaxation linear regression with spectral–spatial constrained locality adaptive regularization for hyperspectral image classification

AbstractRecently, relaxation linear regression has received increasing attention in image analysis. However, the current relaxation linear regression methods fail to consider the local geometrical structure and spatial information while they are applied for hyperspectral image (HSI) classification. To address the above problems, this letter proposes a novel relaxation linear regression with spectral–spatial constrained locality adaptive regularization (SSLA‐RLR) method for HSI classification. The SSLA‐RLR method not only integrates the locality adaptive graph with relaxation linear regression to adaptively exploit the local geometrical structure for relieving the side effects of noise corruptions, but also takes the spatial correlations between each data point and its neighbours into consideration via spectral–spatial constraint. Finally, extensive experiments are conducted on three benchmark HSI datasets to demonstrate the effectiveness of the proposed method.

Joint Sparse Locality Preserving Regression for Discriminative Learning

Ridge Regression (RR) is a classical method that is widely used in multiple regression analysis. However, traditional RR does not take the local geometric structure of data into consideration for discriminative learning and it is sensitive to outliers as it is based on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$L_{2}$</tex-math></inline-formula> -norm. To address this problem, this article proposes a novel method called Joint Sparse Locality Preserving Regression (JSLPR) for discriminative learning. JSLPR not only applies <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$L_{2,1}$</tex-math></inline-formula> -norm on both loss function and regularization term but also takes the local geometric structure of the data into consideration. The use of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$L_{2,1}$</tex-math></inline-formula> -norm can guarantee the robustness to outliers or noises and the joint sparsity for effective feature selection. Taking the local geometric structure into consideration can improve the performance of the feature extraction and selection method when the data lie on a manifold. To solve the optimization problem of JSLPR, an iterative algorithm is proposed and the convergence of the algorithm is also proven. Experiments on four famous face databases are conducted and the results show the merit of the proposed JSLPR on feature extraction and selection.

Discriminative Regression With Adaptive Graph Diffusion.

In this article, we propose a new linear regression (LR)-based multiclass classification method, called discriminative regression with adaptive graph diffusion (DRAGD). Different from existing graph embedding-based LR methods, DRAGD introduces a new graph learning and embedding term, which explores the high-order structure information between four tuples, rather than conventional sample pairs to learn an intrinsic graph. Moreover, DRAGD provides a new way to simultaneously capture the local geometric structure and representation structure of data in one term. To enhance the discriminability of the transformation matrix, a retargeted learning approach is introduced. As a result of combining the above-mentioned techniques, DRAGD can flexibly explore more unsupervised information underlying the data and the label information to obtain the most discriminative transformation matrix for multiclass classification tasks. Experimental results on six well-known real-world databases and a synthetic database demonstrate that DRAGD is superior to the state-of-the-art LR methods.

Graph-Based Multicentroid Nonnegative Matrix Factorization.

Nonnegative matrix factorization (NMF) is a widely recognized approach for data representation. When it comes to clustering, NMF fails to handle data points located in complex geometries, as each sample cluster is represented by a centroid. In this article, a novel multicentroid-based clustering method called graph-based multicentroid NMF (MCNMF) is proposed. Because the method constructs the neighborhood connection graph between data points and centroids, each data point is represented by adjacent centroids, which preserves the local geometric structure. Second, because the method constructs an undirected connected graph with centroids as nodes, in which the centroids are divided into different centroid clusters, a novel data clustering method based on MCNMF is proposed. In addition, the membership index matrix is reconstructed based on the obtained centroid clusters, which solves the problem of membership identification of the final sample. Extensive experiments conducted on synthetic datasets and real benchmark datasets illustrate the effectiveness of the proposed MCNMF method. Compared with single-centroid-based methods, the MCNMF can obtain the best experimental results.

Multi-scale Part-based Feature Representation for 3D Domain Generalization and Adaptation.

Deep networks on 3D point clouds have achieved remarkable success in 3D classification, but they are vulnerable to geometric variations resulting from inconsistent data acquisition procedures. This leads to challenging 3D domain generalization and adaptation tasks, aiming to tackle the challenge that the performance of a model trained on a source domain will degrade on an out-of-distribution target domain. In this paper, we introduce a novel Multi-Scale Part-based feature Representation, dubbed MSPR, as a generalizable representation for point cloud domain generalization and adaptation. Rather than relying on the global point cloud feature representation, we align the part-level features of shapes at different scales to a set of learnable part-template features, which can encode local geometric structures shared between the source and the target domains. Specifically, we construct a part-template feature space shared between source and target domains. Shapes from different domains are organized into part-level features at various scales and then aligned to the part-template features. To leverage the generalization ability of small-scale parts and the discrimination ability of large-scale parts, we further design a cross-scale feature fusion module to exchange information between aligned part-based features at different scales. The fused part-based representations are finally aggregated by a part-based feature aggregation module. To improve the robustness of the aligned part-based representations and global shape representation to geometry variations, we further propose a Contrastive Learning framework on Shape Representation (CLSR), applied to both 3D domain generalization and adaptation tasks. We conduct experiments on 3D domain generalization and adaptation benchmarks for point cloud classification. Experimental results demonstrate the effectiveness of our proposed approach, outperforming the previous state-of-the-art methods for both domain generalization and adaptation tasks. Ablation studies confirm the effectiveness of the proposed components in our model.

A Method for Clustering and Analyzing Vessel Sailing Routes Efficiently from AIS Data Using Traffic Density Images

A vessel automatic identification system (AIS) provides a large amount of dynamic vessel information over a large coverage area and data volume. The AIS data are a typical type of big geo-data with high dimensionality, large noise, heterogeneous densities, and complex distributions. This poses a challenge for the clustering and analysis of vessel sailing routes. This study proposes an efficient vessel sailing route clustering and analysis method based on AIS data that uses traffic density images to transform the clustering problem of complex AIS trajectories into an image processing problem. First, a traffic density image is constructed based on the statistics of the preprocessed AIS data. Next, the main sea route regions of traffic density images are extracted based on local image features, geometric structures, and spatial features. Finally, the sailing trajectories are clustered using the extracted sailing patterns. Based on actual vessel AIS data, multimethod comparisons and performance analysis experiments are conducted to verify the feasibility and effectiveness of the proposed method. These experimental results reveal that the proposed method displays potential for the clustering task of challenging vessel sailing routes.

Multi-label feature selection via adaptive dual-graph optimization

As in single-label learning, multi-label learning (MLL) also suffers from the problem of “the curse of dimensionality” due to the redundancy of features in the original data. To address this problem, extensive research has been conducted to explore the various feature selection algorithms aiming to eliminate irrelevant and redundant features. Among these, the manifold learning based methods have gained significant attention for their ability to capture local label correlations. However, previous approaches have mainly focused on extracting information from data space, overlooking valuable information in feature space. In addition, these methods conduct manifold learning and feature selection independently, thereby degrading the learning performance. To tackle this problem, we propose a novel multi-label feature selection method based on adaptive dual-graph optimization (MFS-ADGO) to learn the geometric structure of the data and select discriminative feature subsets simultaneously. Specifically, a dual-graph manifold regularization is constructed first, to locally capture label correlations in the data and feature spaces for discriminative features selection. A constraint is then imposed on the similarity matrices to adaptively learn the local geometric structure of the data during the feature selection, thereby capturing more reliable information from the original data. To facilitate feature selection, an objective function incorporating the l2,1−2-norm is proposed, along with an iterative updating algorithm. Extensive experiments unequivocally demonstrated the efficacy of MFS-ADGO in enhancing state-of-the-art feature selection methods in the realm of MLL.

PDRLRR: A novel low-rank representation with projection distance regularization via manifold optimization for clustering

The low-rank representation (LRR) method has attracted widespread attention due to its excellent performance in pattern recognition and machine learning. LRR-based variants have been proposed to solve the three existing problems in LRR: (1) the projection matrix is permanently fixed when dimensionality reduction techniques are adopted; (2) LRR fails to capture the local geometric structure; and (3) the solution deviates from the real low-rank solution. To address these problems, this paper proposes a low-rank representation with projection distance regularization (PDRLRR) via manifold optimization for clustering. In detail, we first introduce a low-dimensional projection matrix and a projection distance regularization term to fit the projected data automatically and capture the local structure of the data, respectively. Consequently, the projection matrix and representation matrix are obtained jointly. Then, we obtain a more accurate low-rank solution by minimizing the Schatten-p norm instead of the nuclear norm. Next, the projection matrix is optimized through a generalized Stiefel manifold. Extensive experiments demonstrate that our proposed method outperforms the state-of-the-art methods.

Weakly Supervised 3D Point Cloud Segmentation via Multi-Prototype Learning

Addressing the annotation challenge in 3D Point Cloud segmentation has inspired research into weakly supervised learning. Existing approaches mainly focus on exploiting manifold and pseudo-labeling to make use of large unlabeled data points. A fundamental challenge here lies in the large intra-class variations of local geometric structure, resulting in subclasses within a semantic class. In this work, we leverage this intuition and opt for maintaining an individual classifier for each subclass. Technically, we design a multi-prototype classifier, each prototype serves as the classifier weights for one subclass. To enable effective updating of multi-prototype classifier weights, we propose two constraints respectively for updating the prototypes w.r.t. all point features and for encouraging the learning of diverse prototypes. Experiments on weakly supervised 3D point cloud segmentation tasks validate the efficacy of proposed method in particular at low-label regime. Our hypothesis is also verified given the consistent discovery of semantic subclasses at no cost of additional annotations.

Centric graph regularized log-norm sparse non-negative matrix factorization for multi-view clustering

Multi-view non-negative matrix factorization (NMF) provides a reliable method to analyze multiple views of data for low-dimensional representation. A variety of multi-view learning methods have been developed in recent years, demonstrating successful applications in clustering. However, existing methods in multi-view learning often tend to overlook the non-linear relationships among data and the significance of the similarity of internal views, both of which are essential in multi-view tasks. Meanwhile, the mapping between the obtained representation and the original data typically contains complex hidden information that deserves to be thoroughly explored. In this paper, a novel multi-view NMF is proposed that explores the local geometric structure among multi-dimensional data and learns the hidden representation of different attributes through centric graph regularization and pairwise co-regularization of the coefficient matrix. In addition, the proposed model is further sparsified with l2,log-(pseudo) norm to efficiently generate sparse solutions. As a result, the model obtains a better part-based representation, enhancing its robustness and applicability in complex noisy scenarios. An effective iterative update algorithm is designed to solve the proposed model, and the convergence of the algorithm is proven to be theoretically guaranteed. The effectiveness of the proposed method is verified by comparing it with nine state-of-the-art methods in clustering tasks of eight public datasets.

Collaborative representation based cross-domain semantic transfer for vehicle re-identification

Vehicle re-identification (re-ID) has received increasing attention due to its tremendous potential in practical intelligent transportation scenarios. The main difficulties of vehicle re-ID are generally induced by insufficient annotations, varying viewpoints and illuminations, and mismatched visual-semantic cues. Previous studies on vehicle re-ID have generally focused on exploring better visual representations of the target samples while ignoring semantic complementarity from external sources. In this paper, we propose an unsupervised semantic transfer method based on collaborative matrix representation (STCMR) for vehicle re-ID that obtains rich semantic representations by learning knowledge from external annotated data. STCMR contains three components: an asymmetrical collaborative matrix representation term, a low-rank regularized relaxed attribute term, and a dual-graph Laplacian regularization term. In particular, we first build two reliable feature matrices to better transfer external annotated semantics from the source domain to the target re-ID domain. We then propose a robust collaborative matrix representation term to decompose the feature matrices into user-defined, domain-shared, and domain-unique parts in an asymmetrical manner. To tackle samples with incomplete semantics, we design a low-rank regularized attribute relaxed term that fully exploits existing annotations to complete missing entries in a soft constraint manner. Finally, we exploit the local geometric structure consistency in source and target domains by a dual-graph Laplacian regularization term to adaptively estimate the relationships of source domain samples. In addition, we further propose an alternating iterative algorithm to solve the optimization problem. Experimental results conducted on the VehicleID and VeRi-776 datasets validate the effectiveness of our proposed method.

Spatio-temporal scale information fusion of Functional Near-Infrared Spectroscopy signal for depression detection

Depression, a prevalent and debilitating mental health disorder, necessitates early detection to enable effective intervention. Functional Near-Infrared Spectroscopy (fNIRS) has proven to be a promising technique for studying depressive symptoms due to its affordability, portability, high spatial resolution and non-invasiveness. However, existing fNIRS-based depression detection approaches often overlook signal intrinsic properties and intrinsic correlations among different feature types, as well as local spatial geometric structures of data. To overcome these limitations, we suggest a spatio-temporal scale information fusion framework that involves supervised local low-rank correlation (IFLRC) to detect depression. IFLRC incorporates a low-rank constraint, a sparse ℓ2,1-norm term, multiple Laplacian regularizations, and a sparse penalty projection matrix to capture global spatial functional information, handle errors to enhance robustness to noise, preserve local spatial geometric structures, exploit correlations among multiple information sets, and enhance discriminative ability. An iterative algorithm, based on the alternating direction method of multipliers (ADMM), has been developed to solve the suggested objective function. The experimental findings validate the efficacy of IFLRC, yielding favorable detection rates of 95.8%, 87.3%, and 97.2% in response to positive, neutral, and negative emotional stimuli, respectively. These findings exceed existing methods, demonstrating the potential of fNIRS and the effectiveness of IFLRC for detecting depression. IFLRC presents notable advancements in feature extraction and fusion techniques for depression detection. Its capacity to capture spatio-temporal scale information, leverage intrinsic correlations, and preserve local geometric structures translates into improved performance. This framework holds promise for early detection and intervention in depressive disorders, offering valuable insights for clinical applications.

Non-Negative Low-Rank Representation With Similarity Correction for Cell Type Identification in scRNA-Seq Data.

Single-cell RNA sequencing (scRNA-Seq) technology has emerged as a powerful tool to investigate cellular heterogeneity within tissues, organs, and organisms. One fundamental question pertaining to single-cell gene expression data analysis revolves around the identification of cell types, which constitutes a critical step within the data processing workflow. However, existing methods for cell type identification through learning low-dimensional latent embeddings often overlook the intercellular structural relationships. In this paper, we present a novel non-negative low-rank similarity correction model (NLRSIM) that leverages subspace clustering to preserve the global structure among cells. This model introduces a novel manifold learning process to address the issue of imbalanced neighbourhood spatial density in cells, thereby effectively preserving local geometric structures. This procedure utilizes a position-sensitive hashing algorithm to construct the graph structure of the data. The experimental results demonstrate that the NLRSIM surpasses other advanced models in terms of clustering effects and visualization experiments. The validated effectiveness of gene expression information after calibration by the NLRSIM model has been duly ascertained in the realm of relevant biological studies. The NLRSIM model offers unprecedented insights into gene expression, states, and structures at the individual cellular level, thereby contributing novel perspectives to the field.