Laplacian Regularization Research Articles

Therapeutic peptides identification via kernel risk sensitive loss-based k-nearest neighbor model and multi-Laplacian regularization.

Therapeutic peptides are therapeutic agents synthesized from natural amino acids, which can be used as carriers for precisely transporting drugs and can activate the immune system for preventing and treating various diseases. However, screening therapeutic peptides using biochemical assays is expensive, time-consuming, and limited by experimental conditions and biological samples, and there may be ethical considerations in the clinical stage. In contrast, screening therapeutic peptides using machine learning and computational methods is efficient, automated, and can accurately predict potential therapeutic peptides. In this study, a k-nearest neighbor model based on multi-Laplacian and kernel risk sensitive loss was proposed, which introduces a kernel risk loss function derived from the K-local hyperplane distance nearest neighbor model as well as combining the Laplacian regularization method to predict therapeutic peptides. The findings indicated that the suggested approach achieved satisfactory results and could effectively predict therapeutic peptide sequences.

Structural graph learning method for hyperspectral band selection

ABSTRACT Recently, graph learning-based hyperspectral band selection algorithms illustrate impressive performance for hyperspectral image (HSI) processing, whose goal is to select an optimal band combination containing less redundancy through the learned graph matrix. However, most of the previous methods work in single spectral domain, which neglects the rich image spatial information. Moreover, they typically model the graph matrix from a global perspective while ignoring the differences in spatial distribution that exist in diverse pixel and band regions. Based on the above considerations, to take full account of structure information in spatial and spectral domains, a structural graph learning method for hyperspectral band selection (SGLM) is designed. Specifically, SGLM constructs two matrices using the correlation between pixels and bands under the local perspective, which can capture the image structure information reasonably. Since the proposed method is modelled in both spatial and spectral dimensions, it is conducive to subsequent band subset generation task. Meanwhile, in order to guarantee local spatial consistency among bands, a Laplacian regularization term associated with the error matrix is introduced to the self-representation model. Additionally, considering that the importance of a band is consistent with its capability to reconstruct the entire band set, an adjacent bands reconstruction strategy is adopted to obtain the ultimate band combination, which can assess the significance of each band effectively. The comprehensive experimental analysis on four datasets demonstrates that the proposed SGLM model has outstanding superiority in comparison with several band selection algorithms.

Joint consensus kernel learning and adaptive hypergraph regularization for graph-based clustering

Recent advancements in multiple kernel learning-based graph clustering methods have demonstrated significant promise in effectively learning a consensus kernel matrix from various candidate kernel matrices. This approach enables the creation of a low-dimensional representation in the kernel space of high-dimensional data through self-expressiveness. However, a key challenge remains in capturing the latent geometric properties embedded within different kernel matrices to enhance data representation. In this paper, we propose a novel method called joint consensus kernel learning and adaptive hypergraph regularization for graph-based clustering (JKHR). Our approach integrates an innovative adaptive hypergraph Laplacian regularizer, which is characterized by the fusion of multiple nearest neighbor kernel graphs, into the multiple kernel learning-based graph clustering framework. JKHR jointly and adaptively optimizes both the consensus kernel matrix and the hypergraph Laplacian regularizer to achieve a low-dimensional representation that effectively preserves the intrinsic geometry of the data. Experimental results on both synthetic and real benchmark datasets demonstrate that JKHR outperforms state-of-the-art self-expressiveness-based graph clustering methods as well as traditional clustering techniques.

Impulsive Noise Suppression Based on ℓq -norm and Graph Laplacian Regularization for NOMA System

Impulsive Noise Suppression Based on <i>ℓ_q </i>-norm and Graph Laplacian Regularization for NOMA System

Deep Unrolled Weighted Graph Laplacian Regularization for Depth Completion

Deep Unrolled Weighted Graph Laplacian Regularization for Depth Completion

Multi-source partial multi-label learning via tensor decomposition and nonconvex regularization

Multi-source partial multi-label learning (MSPMLL) aims to learn a multi-label classifier from training examples with multi-source features and multiple labels. Most of the existing MSPMLL models tackle each source independently to derive classifiers. Those methods cannot efficiently reveal the intrinsic structure and relationships among instances across different sources, nor deal with inconsistent labels caused by noise. To address those challenges, this paper proposes a tensor decomposition and nonconvex regularization approach to multi-source partial multi-label learning, nominated as MSPML-TLR. In this model, a hypergraph based affinity graph is constructed from each of multi-source data in the feature space. A tensor representation of multi-source multi-label data is proposed based on the affinity graphs. A multi-linear projection is learned to map the feature affinity space to a latent ground-truth label space via tensor robust principal component analysis. Furthermore, a notion called the parameterized tensorial logarithmic rank is designed to approximate the rank of the multi-linear projection and the latent ground-truth label tensor. Moreover, a hypergraph-based Laplacian regularization is imposed on each frontal slice of the latent ground-truth label tensor to preserve the topological structure of instances across multi-source feature spaces. An efficient algorithm is designed to implement MSPML-TLR and the convergence of the algorithm is rigorously proven. Extensive experiments on various real-world benchmark datasets demonstrate that the proposed model outperforms state-of-the-art methods in multi-label classification tasks.

An Improved Detail-Enhancement CycleGAN Using AdaLIN for Facial Style Transfer

The rise of comics and games has led to increased artistic processing of portrait photos. With growing commercial demand and advancements in deep learning, neural networks for rapid facial style transfer have become a key research area in computer vision. This involves converting face photos into different styles while preserving content. Face images are more complex than regular images, requiring extensive modification. However, current methods often face issues such as unnatural color transitions, loss of detail in highlighted areas, and noticeable artifacts along edges, resulting in low-quality stylized images. In this study, an enhanced generative adversarial network (GAN) is proposed, which is based on Adaptive Layer Instance Normalization (AdaLIN) + Laplacian. This approach incorporates the AdaLIN normalization method, allowing for the dynamic adjustment of Instance Normalization (IN) and Layer Normalization (LN) parameters’ weights during training. By combining the strengths of both normalization techniques, the model selectively preserves and alters content information to some extent, aiming to strike a balance between style and content. This helps address problems such as unnatural color transitions and loss of details in highlights that lead to color inconsistencies. Furthermore, the introduction of a Laplacian regularization term aids in denoising the image, preventing noise features from interfering with the color transfer process. This regularization also helps reduce color artifacts along the face’s edges caused by noise while maintaining the image’s contour information. These enhancements significantly enhance the quality of the generated face images. To compare our method with traditional CycleGAN and recent algorithms such as XGAN and CariGAN, both subjective and objective evaluations were conducted. Subjectively, our method demonstrates more natural color transitions and superior artifact elimination, achieving higher scores in Mean Opinion Score (MOS) evaluations. Objectively, experiments using our method yielded better scores across three metrics: FID, SSIM, and MS-SSIM. The effectiveness of the proposed methods is validated through both objective and subjective evaluations.

Suspicious behaviour detection in multilayer social networks using PF-KMA and SS-GAE techniques

Multilayer Social networks are an important part of human life to interact on different networks at the same time. Due to the openness of such networks, they become a platform for spammers to spread malicious behaviors. Hence, there is an urgent need for effective detection of malicious behaviors; thereby, enabling the networks to take mitigation actions to decrease the possibility to reward such activities. Detection of suspicious behaviors in previous works is challenging due to the problems of community detection, the large amount of feature corruption, and memory requirements. Thus, to deal with such problems, in this paper, an efficient clustering-based detection of malicious users in multilayer social networks is proposed. Initially, the input dataset is pre-processed and used for Exponential Distribution based Erdős–Rényi based graph construction. From the graph structure, two types of data, such as user representations and graph features are extracted for graph encoding using the Soft Sign activated Graph Auto Encoder model. Then, the decoding is done to predict the information diffusion level, thereby, ranking the users using the Laplace Regularization technique. Then, the ranked users are clustered into different groups using the Pareto Front based K-Means Clustering Algorithm technique. Finally, the experimental results were analyzed to demonstrate the efficacy of the proposed model to detect malicious users in multilayer social networks.

Laplacian regularization of linear regression model for semi-supervised industrial soft sensor development

With more and more data collected from modern industrial processes, data-based soft sensor modeling methods have become popular for online prediction of those difficult-to-measure key variables. As one of the most widely used methods, the linear regression model has been studied for many years. In this paper, a new semi-supervised form of data-based soft sensor is developed upon this simple but commonly used linear regression model, in order to make it more engineering applicable. Particularly, a semi-supervised model structure is formulated by introducing a Laplacian regularization term, through which a lot more unlabeled data samples can be well incorporated for modeling. With an improved estimation for the input data distribution, the real data features can be better captured and thus the model parameters are well restricted, in order to provide a better performance. In addition, the basic Laplacian regularized linear regression model is further extended to nonlinear counterparts by the introductions of basis function and kernel trick in the input data space. For performance evaluation, two industrial case studies are provided, based on which both feasibility and effectiveness of the developed method are confirmed.

A New Look and Convergence Rate of Federated Multitask Learning With Laplacian Regularization.

Non-independent and identically distributed (non-IID) data distribution among clients is considered as the key factor that degrades the performance of federated learning (FL). Several approaches to handle non-IID data, such as personalized FL and federated multitask learning (FMTL), are of great interest to research communities. In this work, first, we formulate the FMTL problem using Laplacian regularization to explicitly leverage the relationships among the models of clients for multitask learning. Then, we introduce a new view of the FMTL problem, which, for the first time, shows that the formulated FMTL problem can be used for conventional FL and personalized FL. We also propose two algorithms FedU and decentrali- zed FedU ( dFedU ) to solve the formulated FMTL problem in communication-centralized and decentralized schemes, respectively. Theoretically, we prove that the convergence rates of both algorithms achieve linear speedup for strongly convex and sublinear speedup of order 1/2 for nonconvex objectives. Experimentally, we show that our algorithms outperform the conventional algorithm FedAvg, FedProx, SCAFFOLD, and AFL in FL settings, MOCHA in FMTL settings, as well as pFedMe and Per-FedAvg in personalized FL settings.

Graph Laplace Regularization-based pressure sensor placement strategy for leak localization in the water distribution networks under joint hydraulic and topological feature spaces

Urban water distribution networks (WDNs) have wide range and intricate topology, which include leakage, pipe burst and other abnormal states during production and operation. With the continuous development of the Internet of Things (IoT) technology in recent years, the means of monitoring the WDNs by using wireless sensor network technology has gradually received attention and extensive research. Most of the existing researches select the deployment location of sensors according to the hydraulic state of the WDNs, but the connectivity and topology between the nodes of the WDNs are not fully considered and analyzed. In this study, a new method that can integrate the topological features and hydraulic model information of the WDN is proposed to solve the problem of optimal sensor placement. First, the method preprocesses the covariance matrix of the pressure sensitivity matrix of the water distribution network by a diffusion kernel-based data prefiltering method and obtains the new network topology weights and its Laplacian matrix under the constraints of the network topology through a data-based graphical Laplacian learning method. Then, the sensor placement problem is transformed into a matrix minimum eigenvalue constraint problem by the Graph Laplace Regularization (GLR)-based method, and finally the selection of sensor nodes is accomplished by the method based on Gershgorin Disc Alignment (GDA). The proposed strategy is tested on a passive Hanoi network, an active Net 3 network, and a larger network, PA2, and is compared with some existing methods. The results show that the proposed solution achieves good performance in three different leak localization methods.

Robust deep fuzzy [formula omitted]-means clustering for image data

Image clustering is a difficult task with important application value in computer vision. The key to this task is the quality of images features. Most of current clustering methods encounter the challenge. That is, the process of feature learning and clustering operates independently. To address this problem, several researchers have been dedicated to performing feature learning and deep clustering together. However, the obtained features lack discriminability to address high-dimensional data successfully. To deal with this issue, we propose a novel model named as robust deep fuzzy K-means clustering (RD-FKC), which efficiently projects image samples into a representative embedding space and precisely learns membership degrees into a combined framework. Specifically, RD-FKC introduces Laplacian regularization technique to preserve locality properties of data. Moreover, by using an adaptive loss function, the model becomes more robust to diverse types of outliers. Furthermore, to avoid the latent space being distorted and make the extracted features retain the original information as much as possible, the model introduces reconstruction error and adds regularization to network parameters. Finally, an effective algorithm is derived to solve the optimization model. Numerous experiments have been conducted, illustrating the advantages and superiority of RD-FKC over existing clustering approaches.

A Color- and Geometric-Feature-Based Approach for Denoising Three-Dimensional Cultural Relic Point Clouds.

In the acquisition process of 3D cultural relics, it is common to encounter noise. To facilitate the generation of high-quality 3D models, we propose an approach based on graph signal processing that combines color and geometric features to denoise the point cloud. We divide the 3D point cloud into patches based on self-similarity theory and create an appropriate underlying graph with a Markov property. The features of the vertices in the graph are represented using 3D coordinates, normal vectors, and color. We formulate the point cloud denoising problem as a maximum a posteriori (MAP) estimation problem and use a graph Laplacian regularization (GLR) prior to identifying the most probable noise-free point cloud. In the denoising process, we moderately simplify the 3D point to reduce the running time of the denoising algorithm. The experimental results demonstrate that our proposed approach outperforms five competing methods in both subjective and objective assessments. It requires fewer iterations and exhibits strong robustness, effectively removing noise from the surface of cultural relic point clouds while preserving fine-scale 3D features such as texture and ornamentation. This results in more realistic 3D representations of cultural relics.

Robust multiple subspaces transfer for heterogeneous domain adaptation

Heterogeneous domain adaptation (HDA) aims to execute knowledge transfer from a source domain to a heterogeneous target domain. Previous works typically inject knowledge from the source and target domain into a common subspace. However, this may lead to the ineffectiveness of knowledge transfer due to the existence of heterogeneity. To overcome this drawback, in this paper, we propose a robust multiple subspaces transfer method for heterogeneous domain adaptation. Specifically, knowledge of two domains is projected into a union of multiple subspaces via a self-expressive model, in which joint distribution alignment and dynamic Laplacian regularization on self-repressive coefficients are included in the loss for characterizing transferability. Moreover, we provide a comprehensive analysis of stability, complexity, generalization, and convergence guarantee for the proposed method. Experiments on benchmark vision and Language datasets verify effectiveness of the proposed approach for heterogeneous domain adaptation.

Sample-Level Cross-View Similarity Learning for Incomplete Multi-View Clustering

Incomplete multi-view clustering has attracted much attention due to its ability to handle partial multi-view data. Recently, similarity-based methods have been developed to explore the complete relationship among incomplete multi-view data. Although widely applied to partial scenarios, most of the existing approaches are still faced with two limitations. Firstly, fusing similarities constructed individually on each view fails to yield a complete unified similarity. Moreover, incomplete similarity generation may lead to anomalous similarity values with column sum constraints, affecting the final clustering results. To solve the above challenging issues, we propose a Sample-level Cross-view Similarity Learning (SCSL) method for Incomplete Multi-view Clustering. Specifically, we project all samples to the same dimension and simultaneously construct a complete similarity matrix across views based on the inter-view sample relationship and the intra-view sample relationship. In addition, a simultaneously learning consensus representation ensures the validity of the projection, which further enhances the quality of the similarity matrix through the graph Laplacian regularization. Experimental results on six benchmark datasets demonstrate the ability of SCSL in processing incomplete multi-view clustering tasks. Our code is publicly available at https://github.com/Tracesource/SCSL.

Adaptive graph Laplacian MTL L1, L2 and LS-SVMs

Abstract Multi-Task Learning tries to improve the learning process of different tasks by solving them simultaneously. A popular Multi-Task Learning formulation for SVM is to combine common and task-specific parts. Other approaches rely on using a Graph Laplacian regularizer. Here we propose a combination of these two approaches that can be applied to L1, L2 and LS-SVMs. We also propose an algorithm to iteratively learn the graph adjacency matrix used in the Laplacian regularization. We test our proposal with synthetic and real problems, both in regression and classification settings. When the task structure is present, we show that our model is able to detect it, which leads to better results, and we also show it to be competitive even when this structure is not present.

Enhancing generalized spectral clustering with embedding Laplacian graph regularization

AbstractAn enhanced generalised spectral clustering framework that addresses the limitations of existing methods by incorporating the Laplacian graph and group effect into a regularisation term is presented. By doing so, the framework significantly enhances discrimination power and proves highly effective in handling noisy data. Its versatility enables its application to various clustering problems, making it a valuable contribution to unsupervised learning tasks. To optimise the proposed model, the authors have developed an efficient algorithm that utilises the standard Sylvester equation to compute the coefficient matrix. By setting the derivatives to zero, computational efficiency is maintained without compromising accuracy. Additionally, the authors have introduced smoothing strategies to make the non‐convex and non‐smooth terms differentiable. This enables the use of an alternative iteration re‐weighted procedure (AIwRP), which distinguishes itself from other first‐order optimisation algorithms by introducing auxiliary variables. The authors provide a provable convergence analysis of AIwRP based on the iteration procedures of unconstrained problems to support its effectiveness. Extensive numerical tests have been conducted on synthetic and benchmark databases to validate the superiority of their approaches. The results demonstrate improved clustering performance and computational efficiency compared to several existing spectral clustering methods, further reinforcing the advantages of their proposed framework. The source code is available at https://github.com/ZhangHengMin/LGR_LSRLRR.

Deep Learning of Partially Labeled Data for Quality Prediction Based on Stacked Target-Related Laplacian Autoencoder.

Partially labeled data, which is common in industrial processes due to the low sampling rate of quality variables, remains an important challenge in soft sensor applications. In order to exploit the information from partially labeled data, a target-related Laplacian autoencoder (TLapAE) is proposed in this work. In TLapAE, a novel target-related Laplacian regularizer is developed, which aims to extract structure-preserving and quality-related features by preserving the feature-target mapping according to the local geometrical structure of the data. In addition, stacked TLapAE (STLapAE) is further constructed to extract deep feature representations of the data by hierarchically stacking TLapAE blocks. For model training, backward propagation equations are derived based on matrix calculus techniques to update the model parameters of the proposed TLapAE. The effectiveness of the proposed STLapAE is evaluated using the butane content prediction case in a debutanizer column, the silicon content prediction case in a blast furnace (BF) ironmaking process, and the ethane concentration prediction case in an ethylene fractionator. The results show that the proposed TLapAE model has significantly improved prediction accuracy compared to soft sensors using only labeled data and other partially labeled data modeling methods.

Q-space imaging based on Gaussian radial basis function with Laplace regularization.

To introduce the diffusion signal characteristics presented by spherical harmonics (SH) basis into the q-space imaging method based on Gaussian radial basis function (GRBF) to robustly reconstruct ensemble average diffusion propagator (EAP) in diffusion MRI (dMRI). We introduced the Laplacian regularization of the signal into the dMRI imaging method based on GRBF, and derived the relevant indicators of microstructure imaging and the orientation distribution function (ODF) providing fiber bundle direction information based on EAP. In addition, this method is combined with a multi-compartment model to calculate the diameter of fiber bundle axons. The evaluation of the results included qualitative comparisons and quantitative assessments of the signal fitting. The results show that the proposed method achieves the more significant accuracy improvement in reconstructing signal. Meanwhile, ODFs estimated by the proposed method show the sharper profiles and less spurious peaks, even under the sparse and noisy conditions. In the 36 sets of axon diameter estimation experiments, 34 and 30 sets of results showed that the proposed method reduced the mean and SD of axon diameter estimates, respectively. Moreover, compared with the current state-of-the-art method, the mean and SD of axon diameter estimated by the proposed method are mostly lower, with 32 and 29 of 36 groups. The proposed method outperforms the GRBF regarding signal fitting and the estimation of the EAP and ODF with multi-shell sparse samples. Moreover, it shows the potential to recover important features of microstructures with less uncertainty by using proposed method together with multi-compartment models.

A Laplacian regularized graph neural network for predictive modeling of multiple chronic conditions

Background and Goals: One of the biggest difficulties facing healthcare systems today is the prevalence of multiple chronic diseases (MCC). Mortality and the development of new chronic illnesses are more likely in those with MCC. Pre-existing diseases and risk factors specific to the patient have an impact on the complex stochastic process that guides the evolution of MCC. This study's goal is to use a brand-new Graph Neural Network (GNN) model to examine the connections between specific chronic illnesses, patient-level risk factors, and pre-existing conditions.Methods: We propose a graph neural network model to analyze the relationship between five chronic conditions (diabetes, obesity, cognitive impairment, hyperlipidemia, and hypertension). The proposed model adds a graph Laplacian regularization term to the loss function, which aims to improve the parameter learning process and accuracy of the GNN based on the graph structure. For validation, we used historical data from the Cameron County Hispanic Cohort (CCHC).Results: Evaluating the Laplacian regularized GNN on data from 600 patients, we expanded our analysis from two chronic conditions to five chronic conditions. The proposed model consistently surpassed a baseline GNN model, achieving an average accuracy of ≥89% across all combinations. In contrast, the performance of the standard model declined more markedly with the addition of more chronic conditions. The Laplacian regularization provided consistent predictions for adjacent nodes, beneficial in cases with shared attributes among nodes.Conclusions: The incorporation of Laplacian regularization in our GNN model is essential, resulting in enhanced node categorization and better predictive performance by harnessing the graph structure. This study underscores the significance of considering graph structure when designing neural networks for graph data. Future research might further explore and refine this regularization method for various tasks using graph-structured data.