Graph Laplacian Regularization Research Articles

Deep Unrolled Weighted Graph Laplacian Regularization for Depth Completion

Deep Unrolled Weighted Graph Laplacian Regularization for Depth Completion

A Color- and Geometric-Feature-Based Approach for Denoising Three-Dimensional Cultural Relic Point Clouds.

In the acquisition process of 3D cultural relics, it is common to encounter noise. To facilitate the generation of high-quality 3D models, we propose an approach based on graph signal processing that combines color and geometric features to denoise the point cloud. We divide the 3D point cloud into patches based on self-similarity theory and create an appropriate underlying graph with a Markov property. The features of the vertices in the graph are represented using 3D coordinates, normal vectors, and color. We formulate the point cloud denoising problem as a maximum a posteriori (MAP) estimation problem and use a graph Laplacian regularization (GLR) prior to identifying the most probable noise-free point cloud. In the denoising process, we moderately simplify the 3D point to reduce the running time of the denoising algorithm. The experimental results demonstrate that our proposed approach outperforms five competing methods in both subjective and objective assessments. It requires fewer iterations and exhibits strong robustness, effectively removing noise from the surface of cultural relic point clouds while preserving fine-scale 3D features such as texture and ornamentation. This results in more realistic 3D representations of cultural relics.

Sample-Level Cross-View Similarity Learning for Incomplete Multi-View Clustering

Incomplete multi-view clustering has attracted much attention due to its ability to handle partial multi-view data. Recently, similarity-based methods have been developed to explore the complete relationship among incomplete multi-view data. Although widely applied to partial scenarios, most of the existing approaches are still faced with two limitations. Firstly, fusing similarities constructed individually on each view fails to yield a complete unified similarity. Moreover, incomplete similarity generation may lead to anomalous similarity values with column sum constraints, affecting the final clustering results. To solve the above challenging issues, we propose a Sample-level Cross-view Similarity Learning (SCSL) method for Incomplete Multi-view Clustering. Specifically, we project all samples to the same dimension and simultaneously construct a complete similarity matrix across views based on the inter-view sample relationship and the intra-view sample relationship. In addition, a simultaneously learning consensus representation ensures the validity of the projection, which further enhances the quality of the similarity matrix through the graph Laplacian regularization. Experimental results on six benchmark datasets demonstrate the ability of SCSL in processing incomplete multi-view clustering tasks. Our code is publicly available at https://github.com/Tracesource/SCSL.

Adaptive graph Laplacian MTL L1, L2 and LS-SVMs

Abstract Multi-Task Learning tries to improve the learning process of different tasks by solving them simultaneously. A popular Multi-Task Learning formulation for SVM is to combine common and task-specific parts. Other approaches rely on using a Graph Laplacian regularizer. Here we propose a combination of these two approaches that can be applied to L1, L2 and LS-SVMs. We also propose an algorithm to iteratively learn the graph adjacency matrix used in the Laplacian regularization. We test our proposal with synthetic and real problems, both in regression and classification settings. When the task structure is present, we show that our model is able to detect it, which leads to better results, and we also show it to be competitive even when this structure is not present.

A Laplacian regularized graph neural network for predictive modeling of multiple chronic conditions

Background and Goals: One of the biggest difficulties facing healthcare systems today is the prevalence of multiple chronic diseases (MCC). Mortality and the development of new chronic illnesses are more likely in those with MCC. Pre-existing diseases and risk factors specific to the patient have an impact on the complex stochastic process that guides the evolution of MCC. This study's goal is to use a brand-new Graph Neural Network (GNN) model to examine the connections between specific chronic illnesses, patient-level risk factors, and pre-existing conditions.Methods: We propose a graph neural network model to analyze the relationship between five chronic conditions (diabetes, obesity, cognitive impairment, hyperlipidemia, and hypertension). The proposed model adds a graph Laplacian regularization term to the loss function, which aims to improve the parameter learning process and accuracy of the GNN based on the graph structure. For validation, we used historical data from the Cameron County Hispanic Cohort (CCHC).Results: Evaluating the Laplacian regularized GNN on data from 600 patients, we expanded our analysis from two chronic conditions to five chronic conditions. The proposed model consistently surpassed a baseline GNN model, achieving an average accuracy of ≥89% across all combinations. In contrast, the performance of the standard model declined more markedly with the addition of more chronic conditions. The Laplacian regularization provided consistent predictions for adjacent nodes, beneficial in cases with shared attributes among nodes.Conclusions: The incorporation of Laplacian regularization in our GNN model is essential, resulting in enhanced node categorization and better predictive performance by harnessing the graph structure. This study underscores the significance of considering graph structure when designing neural networks for graph data. Future research might further explore and refine this regularization method for various tasks using graph-structured data.

Learning Fair Representations via Distance Correlation Minimization.

As machine learning algorithms are increasingly deployed for high-impact automated decision-making, the presence of bias (in datasets or tasks) gradually becomes one of the most critical challenges in machine learning applications. Such challenges range from the bias of race in face recognition to the bias of gender in hiring systems, where race and gender can be denoted as sensitive attributes. In recent years, much progress has been made in ensuring fairness and reducing bias in standard machine learning settings. Among them, learning fair representations with respect to the sensitive attributes has attracted increasing attention due to its flexibility in learning the rich representations based on advances in deep learning. In this article, we propose graph-fair, an algorithmic approach to learning fair representations under the graph Laplacian regularization, which reduces the separation between groups and the clustering within a group by encoding the sensitive attribute information into the graph. We have theoretically proved the underlying connection between graph regularization and distance correlation and show that the latter can be regarded as a standardized version of the former, with an additional advantage of being scale-invariant. Therefore, we naturally adopt the distance correlation as the fairness constraint to decrease the dependence between sensitive attributes and latent representations, called dist-fair. In contrast to existing approaches using measures of dependency and adversarial generators, both graph-fair and dist-fair provide simple fairness constraints, which eliminate the need for parameter tuning (e.g., choosing kernels) and introducing adversarial networks. Experiments conducted on real-world corpora indicate that our proposed fairness constraints applied for representation learning can provide better tradeoffs between fairness and utility results than existing approaches.

Hyperspectral Image Denoising Using Superpixel Segmentation and Graph Laplacian Regularization

Hyperspectral Image Denoising Using Superpixel Segmentation and Graph Laplacian Regularization

Cauchy Nonnegative Matrix Factorization for Hyperspectral Unmixing Based on Graph Laplacian Regularization

Cauchy Nonnegative Matrix Factorization for Hyperspectral Unmixing Based on Graph Laplacian Regularization

Cross-modal missing time-series imputation using dense spatio-temporal transformer nets.

Due to irregular sampling or device failure, the data collected from sensor network has missing value, that is, missing time-series data occurs. To address this issue, many methods have been proposed to impute random or non-random missing data. However, the imputation accuracy of these methods are not accurate enough to be applied, especially in the case of complete data missing (CDM). Thus, we propose a cross-modal method to impute time-series missing data by dense spatio-temporal transformer nets (DSTTN). This model embeds spatial modal data into time-series data by stacked spatio-temporal transformer blocks and deployment of dense connections. It adopts cross-modal constraints, a graph Laplacian regularization term, to optimize model parameters. When the model is trained, it recovers missing data finally by an end-to-end imputation pipeline. Various baseline models are compared by sufficient experiments. Based on the experimental results, it is verified that DSTTN achieves state-of-the-art imputation performance in the cases of random and non-random missing. Especially, the proposed method provides a new solution to the CDM problem.

Impulsive Noise Suppression Based on ℓq-norm and Graph Laplacian Regularization for NOMA System

Impulsive Noise Suppression Based on ℓq-norm and Graph Laplacian Regularization for NOMA System

Learning Common and Task-Specific Radiomic Features via Graph Regularized NMF for the Joint Prediction of Multiple Clinical Indicators in Breast Cancer.

Assessments of multiple clinical indicators based on radiomic analysis of magnetic resonance imaging (MRI) are beneficial to the diagnosis, prognosis and treatment of breast cancer patients. Many machine learning methods have been designed to jointly predict multiple indicators for more accurate assessments while using original clinical labels directly without considering the noisy and redundant information among them. To this end, we propose a multilabel learning method based on label space dimensionality reduction (LSDR), which learns common and task-specific features via graph regularized nonnegative matrix factorization (CTFGNMF) for the joint prediction of multiple indicators in breast cancer. A nonnegative matrix factorization (NMF) is adopted to map original clinical labels to a low-dimensional latent space. The latent labels are employed to exploit task correlations by using a least square loss function with [Formula: see text]-norm regularization to identify common features, which help to improve the generalization performance of correlated tasks. Furthermore, task-specific features were retained by a multitask regression formulation to increase the discrimination power for different tasks. Common and task-specific features are incorporated by dynamic graph Laplacian regularization into a unified model to learn complementary features. Then, a multilabel classification is built to predict multiple clinical indicators including human epidermal growth factor receptor 2 (HER2), Ki-67, and histological grade. Experimental results show that CTFGNMF achieves AUCs of 0.823, 0.691 and 0.776 in the three indicator predictions, outperforming other counterparts that consider only task-independent features or common features. It indicates CTFGNMF is a promising application for multiple classification tasks in breast cancer.

PLPCA: Persistent Laplacian-Enhanced PCA for Microarray Data Analysis.

Over the years, Principal Component Analysis (PCA) has served as the baseline approach for dimensionality reduction in gene expression data analysis. Its primary objective is to identify a subset of disease-causing genes from a vast pool of thousands of genes. However, PCA possesses inherent limitations that hinder its interpretability, introduce class ambiguity, and fail to capture complex geometric structures in the data. Although these limitations have been partially addressed in the literature by incorporating various regularizers, such as graph Laplacian regularization, existing PCA based methods still face challenges related to multiscale analysis and capturing higher-order interactions in the data. To address these challenges, we propose a novel approach called Persistent Laplacian-enhanced Principal Component Analysis (PLPCA). PLPCA amalgamates the advantages of earlier regularized PCA methods with persistent spectral graph theory, specifically persistent Laplacians derived from algebraic topology. In contrast to graph Laplacians, persistent Laplacians enable multiscale analysis through filtration and can incorporate higher-order simplicial complexes to capture higher-order interactions in the data. We evaluate and validate the performance of PLPCA using ten benchmark microarray data sets that exhibit a wide range of dimensions and data imbalance ratios. Our extensive studies over these data sets demonstrate that PLPCA provides up to 12% improvement to the current state-of-the-art PCA models on five evaluation metrics for classification tasks after dimensionality reduction.

Sparse graph-regularized dictionary learning for full waveform inversion

Full waveform inversion is a nonlinear fitting technique that requires regularization methods to alleviate ill-posedness. Based on the newly introduced Graph-Laplacian regularization, which offers outstanding manifold learning, we propose a novel regularization method for FWI that can combine the graph Laplacian matrix with a sparse representation in a learning-based dictionaries. Besides employing an adaptive sparsity prior, the graph Laplacian matrix incorporates the nonlocal self-similarity prior into the FWI process, and consequently improve the stability of the inversion process and produce more accurate inversion results. The Alternating Direction Method of Multipliers (ADMM) is applied to solve the problem numerically. A series of experiments are performed on the Marmousi model and the BG Compass model. The experimental results verify the effectiveness of the proposed FWI method in both quantitative analysis and visual perception.

Graph laplacian regularization with sparse coding in secure image restoration and representation for Internet of Things

Image restoration (IR) attempts to recreate the original (ideal) scene from a degraded observation. The goal of image restoration is to avert the process of image deterioration that happens during image acquisition and processing. Blurring and noise are two common types of picture capture degradation. When the blurring function is unknown, the image restoration issue is referred to as blind image restoration. Existing methods of image restoration do not provide efficient results due to time complexity, computational complexity, large-scale scaling factor and limited input. Therefore, in order to overcome those problems, an Improved Graph Laplacian Regularization with Sparse Coding by integrating Internet of Things (IoT) is developed in this research. The process of removing de-noised portions of image to texture layer and cartoon layer is carried out by Morphological Component Analysis (MCA). An improved Graph Laplacian regularized method and Simultaneous Sparse Coding with Gaussian Scale Mixture (SSC-GSM) methods is performed on texture layer, cartoon layer and restored image. A Levin’s dataset and a real-time dataset such as industrial manufacturing products are analyzed in this research. The proposed Graph Laplacian algorithm and sparse coding model are better efficient in identifying sharper texture information and getting a restored image. The proposed method proves that it has better performance in terms of Peak Signal-to-Noise Ratio (PSNR) of 35.82 and Structural Similarity Index Measure (SSIM) of 0.94 than the existing methods.

Mutual structure learning for multiple kernel clustering

Multiple kernel clustering (MKC) has garnered considerable attention in recent years, aiming to obtain an optimal partition from multiple base kernels. Existing MKC methods typically focus on either learning the pairwise structure by constructing an optimal kernel from the base kernels or learning the cluster structure by integrating multiple base partitions into a consensus one for clustering. However, these previous approaches overlook the potential for mutual structure learning between the two aspects, i.e., pairwise and cluster structure, of clustering. To address this limitation, we propose a novel multiple kernel clustering method, referred to as MSL-MKC, which simultaneously learns pairwise and cluster structure to achieve an improved consensus partition matrix for clustering. Specifically, MSL-MKC extends the adaptive neighbor graph learning approach into kernel space to construct a discriminative similarity graph from multiple base kernels for pairwise structure learning. The consensus partition is formulated by aligning it with multiple base partitions for cluster structure learning. We utilize a Laplacian regularization term to preserve the pairwise structure in the consensus partition and inject the cluster structure into the discriminative similarity graph. The proposed method integrates similarity graph learning, partition fusion, and Laplacian regularization into a unified framework, optimized using an iterative algorithm. Experimental results on various benchmark datasets demonstrate the efficacy of MSL-MKC. The demo code of this work is publicly available at https://github.com/guanyuezhen/MSL-MKC.

Semi-parametric contextual bandits with graph-Laplacian regularization

Non-stationarity is ubiquitous in human behavior and addressing it in the contextual bandits is challenging. Several works have addressed the problem by investigating semi-parametric contextual bandits and warned that ignoring non-stationarity could harm performances. Another prevalent human behavior is social interaction which has become available in a form of a social network or graph structure. As a result, graph-based contextual bandits have received much attention. In this paper, we propose SemiGraphTS, a novel contextual Thompson-sampling algorithm for a graph-based semi-parametric reward model. Our algorithm is the first to be proposed in this setting. We derive an upper bound of the cumulative regret that can be expressed as a multiple of a factor depending on the graph structure and the order for the semi-parametric model without a graph. We evaluate the proposed and existing algorithms via simulation and real data example.

A Learnable Radial Basis Positional Embedding for Coordinate-MLPs

We propose a novel method to enhance the performance of coordinate-MLPs (also referred to as neural fields) by learning instance-specific positional embeddings. End-to-end optimization of positional embedding parameters along with network weights leads to poor generalization performance. Instead, we develop a generic framework to learn the positional embedding based on the classic graph-Laplacian regularization, which can implicitly balance the trade-off between memorization and generalization. This framework is then used to propose a novel positional embedding scheme, where the hyperparameters are learned per coordinate (i.e instance) to deliver optimal performance. We show that the proposed embedding achieves better performance with higher stability compared to the well-established random Fourier features (RFF). Further, we demonstrate that the proposed embedding scheme yields stable gradients, enabling seamless integration into deep architectures as intermediate layers.

RRNMF-MAGL: Robust regularization non-negative matrix factorization with multi-constraint adaptive graph learning for dimensionality reduction

In this paper, a new unsupervised dimensionality reduction method named robust regularization non-negative matrix factorization with multi-constraint adaptive graph learning (RRNMF-MAGL) is developed. Compared with the existing methods, RRNMF-MAGL can extract robust low-dimensional features and adaptively learn a manifold structure to well reflect the data distribution. Specifically, to alleviate the influence of outliers and noises, a robust non-negative matrix factorization (RNMF) approach with L21-norm loss function is first constructed. Then, a multi-constraint adaptive graph learning (MAGL) model is designed based on low-dimensional features to reduce the computational complexity and avoid the adverse influence of redundant information. Moreover, multiple constraints (i.e., sparsity and locality) are incorporated into graph learning for enhancing the discrimination ability of graph structure. Next, to make the learned low-dimensional features well maintain the local geometric structure information of data, a graph Laplacian regularization (GLR) is designed. Finally, an iterative update strategy is proposed to optimize RRNMF-MAGL and its convergence is verified by numerical experiments. The superior effectiveness and robustness of RRNMF-MAGL is demonstrated by extensive experiments on eight publicly available datasets.

Row-Column Overcomplete Structured Dictionary Learning for Enhanced Fault Detection and Isolation

To improve the monitoring performance of dictionary learning based methods, this article proposes a row-column overcomplete structured dictionary learning (RCOSDL) method for fault detection and isolation of industrial processes. Unlike conventional dictionary learning approaches which are overcomplete column-wise, the proposed method involves a dictionary that is overcomplete both row- and column-wise. The introduction of row-column overcomplete dictionary results in monitoring statistics that are more sensitive to incipient faults. In order to incorporate structured information, two graph Laplacian regularization terms, namely, manifold graph Laplacian and full graph Laplacian are considered. Whilst the inherent local geometric structure in the data is preserved in the sparse representation by the manifold graph Laplacian term, the correlation structure between process variables is preserved by using the full graph Laplacian term. Hence violation in the geometric or correlation structure will be promptly detected. To pinpoint faulty variables, a fault isolation method is developed by imposing the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$l_{1}$</tex-math></inline-formula> / <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$l_{2,1}$</tex-math></inline-formula> -norm constraint on the sparse coefficients. In addition, theoretical property involving the condition for guaranteed fault isolation is presented in Theorem 1. The contributions of this article include the introduction of monitoring statistics based on RCOSDL that are suitable for incipient faults, a new fault isolation scheme as well as theoretical analysis on the condition of guaranteed fault isolation. The better performance of the proposed method is illustrated by applications to numerical studies and practical industrial cases.

Learning Transferable Sparse Representations for Cross-Corpus Facial Expression Recognition

An assumption widely used in traditional facial expression recognition algorithms is that the training and testing are conducted on the same dataset. However, this assumption does not hold in practice, in which the training data and testing data are often from different datasets. In this scenario, directly deploying these algorithms would lead to severe information loss and performance degradation due to the domain shift. To address this challenging problem, in this paper, we propose a novel transferable sparse subspace representation method (TSSR) for cross-corpus facial expression recognition. Specifically, in order to reduce the cross-corpus mismatch, inspired by sparse subspace clustering, we advocate reconstructing the source and target samples using the source data points based on L1-norm sparse representation. Each data point in source and target corpora can be ideally represented as a combination of a few other source points from its own subspace. Moreover, we take into account the local geometrical information within the cross-corpus data by adopting a graph Laplacian regularizer, which can efficiently preserve the local manifold structure and better transfer knowledge between two corpora. Finally, experimental results on several facial expression datasets demonstrate the superiority of the proposed method over some state-of-the-art methods.