Laplacian Regularization Research Articles

Laplacian Welsch Regularization for Robust Semisupervised Learning.

Semisupervised learning (SSL) has been widely used in numerous practical applications where the labeled training examples are inadequate while the unlabeled examples are abundant. Due to the scarcity of labeled examples, the performances of the existing SSL methods are often affected by the outliers in the labeled data, leading to the imperfect trained classifier. To enhance the robustness of SSL methods to the outliers, this article proposes a novel SSL algorithm called Laplacian Welsch regularization (LapWR). Specifically, apart from the conventional Laplacian regularizer, we also introduce a bounded, smooth, and nonconvex Welsch loss which can suppress the adverse effect brought by the labeled outliers. To handle the model nonconvexity caused by the Welsch loss, an iterative half-quadratic (HQ) optimization algorithm is adopted in which each subproblem has an ideal closed-form solution. To handle the large datasets, we further propose an accelerated model by utilizing the Nyström method to reduce the computational complexity of LapWR. Theoretically, the generalization bound of LapWR is derived based on analyzing its Rademacher complexity, which suggests that our proposed algorithm is guaranteed to obtain satisfactory performance. By comparing LapWR with the existing representative SSL algorithms on various benchmark and real-world datasets, we experimentally found that LapWR performs robustly to outliers and is able to consistently achieve the top-level results.

Group Guided Fused Laplacian Sparse Group Lasso for Modeling Alzheimer’s Disease Progression

As the largest cause of dementia, Alzheimer's disease (AD) has brought serious burdens to patients and their families, mostly in the financial, psychological, and emotional aspects. In order to assess the progression of AD and develop new treatment methods for the disease, it is essential to infer the trajectories of patients' cognitive performance over time to identify biomarkers that connect the patterns of brain atrophy and AD progression. In this article, a structured regularized regression approach termed group guided fused Laplacian sparse group Lasso (GFL-SGL) is proposed to infer disease progression by considering multiple prediction of the same cognitive scores at different time points (longitudinal analysis). The proposed GFL-SGL simultaneously exploits the interrelated structures within the MRI features and among the tasks with sparse group Lasso (SGL) norm and presents a novel group guided fused Laplacian (GFL) regularization. This combination effectively incorporates both the relatedness among multiple longitudinal time points with a general weighted (undirected) dependency graphs and useful inherent group structure in features. Furthermore, an alternating direction method of multipliers- (ADMM-) based algorithm is also derived to optimize the nonsmooth objective function of the proposed approach. Experiments on the dataset from Alzheimer's Disease Neuroimaging Initiative (ADNI) show that the proposed GFL-SGL outperformed some other state-of-the-art algorithms and effectively fused the multimodality data. The compact sets of cognition-relevant imaging biomarkers identified by our approach are consistent with the results of clinical studies.

Laplacian Regularized Nonnegative Representation for Clustering and Dimensionality Reduction

Self-representation methods, such as low-rank representation (LRR), sparse subspace clustering (SSC) and their variants may generate negative coding coefficients since there is no explicit nonnegative constraint. These negative coefficients lack physical meaning. To be specific, it is unreasonable to allow a query sample encoded by heterogeneous samples to cancel each other out with subtractions. In this paper, we propose a novel model named Laplacian regularized nonnegative representation (LapNR). The new model improves its physical interpretability by ensuring that the query sample should be approximated from homogeneous samples and irrelevant to heterogeneous ones. More importantly, it captures the geometric information of input data by imposing the graph Laplacian to the nonnegative representations. As a result, the representation matrix generated by our LapNR model becomes sparse and discriminative. Based on the alternating direction method of multipliers (ADMM), an efficient optimization procedure is developed for LapNR. The extensive experiments on clustering and dimensionality reduction tasks show the effectiveness and efficiency of our LapNR.

Regularization on a rapidly varying manifold

The presence of inflection points on data manifold or rapid variation makes it difficult for the second order derivative based graph Laplacian and Hessian regularization techniques to accurately approximate the marginal distribution parameters. Moreover, in general, function over-fitting on seen unlabeled instances due to lack of extrapolation power which makes graph Laplacian regularization based solution biased towards constant. Hessian solves this problem by opting a generic function based on the function’s divergence in more than one direction. However, due to the presence of inflection points in the dense region, the function remains unpenalized by Hessian manifold regularization. We propose a Jerk based manifold regularization (JR) for dense, oscillating and manifolds with inflection points. JR approximates the rate of change of curvature from the underlying manifold which appropriately identifies the unpenalized geodesic deviating functions and accurately penalizes them. It also helps in identifying the optimal function in the presence of inflection points. Extensive experiments on synthetic and real-world datasets show that the proposed JR technique approximates accurate and generic input space geometrical constraints to outperform existing state-of-the-art manifold regularization techniques by a significant margin.

Drug-target interaction prediction using Multi Graph Regularized Nuclear Norm Minimization

The identification of potential interactions between drugs and target proteins is crucial in pharmaceutical sciences. The experimental validation of interactions in genomic drug discovery is laborious and expensive; hence, there is a need for efficient and accurate in-silico techniques which can predict potential drug-target interactions to narrow down the search space for experimental verification. In this work, we propose a new framework, namely, Multi-Graph Regularized Nuclear Norm Minimization, which predicts the interactions between drugs and target proteins from three inputs: known drug-target interaction network, similarities over drugs and those over targets. The proposed method focuses on finding a low-rank interaction matrix that is structured by the proximities of drugs and targets encoded by graphs. Previous works on Drug Target Interaction (DTI) prediction have shown that incorporating drug and target similarities helps in learning the data manifold better by preserving the local geometries of the original data. But, there is no clear consensus on which kind and what combination of similarities would best assist the prediction task. Hence, we propose to use various multiple drug-drug similarities and target-target similarities as multiple graph Laplacian (over drugs/targets) regularization terms to capture the proximities exhaustively. Extensive cross-validation experiments on four benchmark datasets using standard evaluation metrics (AUPR and AUC) show that the proposed algorithm improves the predictive performance and outperforms recent state-of-the-art computational methods by a large margin. Software is publicly available at https://github.com/aanchalMongia/MGRNNMforDTI.

Semi-supervised learning quantization algorithm with deep features for motor imagery EEG Recognition in smart healthcare application

This paper depicts a novel semi-supervised classification model with convolutional neural networks (CNN) for EEG Recognition. The performance of popular machine learning algorithm usually rely on the number of labeled training samples, such as the deep learning approaches,sparse classification approaches and supervised learning approaches. However, the labeled samples are very difficulty to get for electroencephalography(EEG) data. In addition, most deep learning algorithms are usually time-consuming in the process of training. Considering these problems, in this article, a novel semi-supervised quantization algorithm based on the cartesian K-means algorithm is proposed, which named it as the semi-supervised cartesian K-means (SSCK), we use the CNN models pre-trained on motor imagery samples to create deep features, and then we applied it for motor imagery (MI) data classification. Unlike the traditional semi-supervised learning models that labeled information can be directly casted into the model training, label information can only be implicitly used in the semi-supervised learning strategy, in the semi-supervised learning algorithm, supervised information is integrated into the quantization algorithm by resorting a supervised constructed laplacian regularizer. Experimental results over four popular EEG datasets substantiate the efficiency and effectiveness of our proposed semi-supervised cartesian K-means.

Saliency detection for strip steel surface defects using multiple constraints and improved texture features

Surface defect detection of strip steel is still a challenging task for its complex variances, e.g., intra-class defects exist large differences in appearance while inter-class defects contain similar parts. To address these issues, we regard the defect object as the salient part of the image and propose a novel, effective saliency propagation algorithm based on multiple constraints and improved texture features (MCITF). Firstly, we deliberately design 83-dim texture features that are used to generate label matrix (among which the label information viewed as the important basis of diffusion process) in the framework of multiple-instance learning. Then we resort to Laplacian regularization viewed as smoothness constraint for enlarging the gap between defect objects and background, and high-level prior (background, object, and mid-level feature) constraints for constraining the label information propagation process locally in order to uniformly highlight the complete defect objects while effectively suppress the non-salient background. Finally, we observe that the superpixel segmentation algorithm based on spectral clustering can adequately capture the edge information of defects, thus promoting to yield high-quality pixel-level saliency maps. Experimental results implemented on the real challenging strip steel benchmark database demonstrate that our MCITF model outperforms state-of-the-art methods with large margins and strong robustness in terms of eight evaluation metrics.

A Novel lncRNA-Disease Association Prediction Model Using Laplacian Regularized Least Squares and Space Projection-Federated Method

Recent studies indicated that numerous long noncoding RNAs (lncRNAs) are closely related to human diseases and can serve as potential biomarkers and drug targets for complex diseases. Therefore, identifying lncRNAs associated with diseases through computational methods is conducive to the exploration of disease pathogenesis. Most previous studies had shortcomings, such as low prediction accuracy, the need for negative samples, and weak generalization. Such studies established shallow prediction models and failed to fully capture the complex relationships among lncRNA-disease associations, lncRNA similarity, and disease similarity. LRLSSP, a new computational method based on Laplacian regularized least squares (LRLS) and space projection was used to predict candidate disease lncRNAs in this study. LRLSSP deeply integrates information on lncRNA similarity, disease similarity, and known lncRNA-disease associations. The estimated score of lncRNA-disease association was obtained through LRLS, and network projection was utilized to reliably predict disease-related lncRNAs. Leave-one-out cross validation(LOOCV) was implemented to evaluate the prediction performance of LRLSSP. Results showed that LRLSSP performed was better than other state-of-the-art methods in predicting lncRNA-disease associations. In addition, case studies conducted on melanoma,cervical cancer, ovarian cancer and breast cancer indicated that LRLSSP can discover potential and novel lncRNA-disease associations. Overall, the results demonstrated that LRLSSP may serve as a reliable and effective computational tool for disease-related lncRNAs prediction.

Fabric defect fetection via weighted low-rank decomposition and Laplacian regularization

Low-rank decomposition models have potential for fabric defect detection, where a feature matrix is decomposed into a low-rank matrix that corresponding to repeated texture structure and a sparse matrix that represent defective regions. Two limitations, however, still exist. First, previous work might fail to detect some large homogeneous defective block. Second, when the background and defective regions are relatively coherent or the texture of fabric image is complex, it is difficult to use previous methods to separate them. To deal with these problems, a new weighted low-rank decomposition model with Laplace regularization (WLRL) is proposed in this paper: (1) a weighted low-rank decomposition model that can decompose the original image into background and defective regions, and (2) a Laplace regularization that can enlarge the distance between the background and the defective regions. The performance of the proposed method WLRL is evaluated on the box- and star-patterned fabric databases, and superior results are shown compared with state-of-the-art methods, that is, 98.70% ACC (accuracy) and 72.83% TPR (true positive rate) for box-patterned fabrics, 99.09% ACC (accuracy) and 83.63% TPR (true positive rate) for star-patterned fabrics.

Video Object Segmentation by Latent Outcome Regression

This paper presents a novel algorithm for unsupervised video object segmentation (UVOS) in unconstrained scenarios. Although a large variety of methods have been proposed in the literature, segmenting generic objects is still challenging because different methods often perform well in different situations, and no single method can outperform the others in all cases. To address this, we propose to solve the problem of UVOS in a crowd-sourcing setting. We claim that one can achieve superior results by aggregating the predictions of multiple imperfect methods in a reasonable way. Specifically, we propose a latent regression algorithm for ensemble-based segmentation by jointly labelling pixels in a sequence and learning an adaptive weight for each single method in an ensemble. The pixel labellings offer the outcome (pseudo groundtruth) for regression and thus promote the procedure of weight learning, while the learnt weights could provide better shape priors for labelling, resulting in more accurate segmentation. Besides, Laplacian regularization is introduced into the regression to facilitate a stable learning of the weights. The most distinct feature of our algorithm is that it adaptively learns the contributions of different single methods for each test sequence, thus is capable of capturing the advantages of those methods while avoiding their weaknesses. In the experiments, our algorithm is built on 14 non-deep learning segmentation methods which are based on handcrafted features and require no training data. Experimental results on popular benchmarks show that our algorithm achieves compelling performance, even in comparison with deep learning-based methods. Furthermore, benefiting from the adaptive weight learning mechanism, our algorithm can achieve good flexibility and usability by choosing the most complementary single methods without losing too much performance.

Reliable Multi-kernel Subtask Graph Correlation Tracker.

Many astonishing correlation filter trackers pay limited concentration on the tracking reliability and locating accuracy. To solve the issues, we propose a reliable and accurate cross correlation particle filter tracker via graph regularized multi-kernel multi-subtask learning. Specifically, multiple non-linear kernels are assigned to multi-channel features with reliable feature selection. Each kernel space corresponds to one type of reliable and discriminative features. Then, we define the trace of each target subregion with one feature as a single view, and their multi-view cooperations and interdependencies are exploited to jointly learn multi-kernel subtask cross correlation particle filters, and make them complement and boost each other. The learned filters consist of two complementary parts: weighted combination of base kernels and reliable integration of base filters. The former is associated to feature reliability with importance map, and the weighted information reflects different tracking contribution to accurate location. The second part is to find the reliable target subtasks via the response map, to exclude the distractive subtasks or backgrounds. Besides, the proposed tracker constructs the Laplacian graph regularization via cross similarity of different subtasks, which not only exploits the intrinsic structure among subtasks, and preserves their spatial layout structure, but also maintains the temporal-spatial consistency of subtasks. Comprehensive experiments on five datasets demonstrate its remarkable and competitive performance against state-of-the-art methods.

Text detection in natural scene images using morphological component analysis and Laplacian dictionary

Text in natural scene images usually carries abundant semantic information. However, due to variations of text and complexity of background, detecting text in scene images becomes a critical and challenging task. In this paper, we present a novel method to detect text from scene images. Firstly, we decompose scene images into background and text components using morphological component analysis &#x0028 MCA &#x0029 , which will reduce the adverse effects of complex backgrounds on the detection results. In order to improve the performance of image decomposition, two discriminative dictionaries of background and text are learned from the training samples. Moreover, Laplacian sparse regularization is introduced into our proposed dictionary learning method which improves discrimination of dictionary. Based on the text dictionary and the sparse-representation coefficients of text, we can construct the text component. After that, the text in the query image can be detected by applying certain heuristic rules. The results of experiments show the effectiveness of the proposed method.

Unsupervised Spectral Feature Selection with Dynamic Hyper-graph Learning

Unsupervised Spectral Feature Selection (USFS) methods may output interpretable and discriminative results by integrating a Laplacian regularizer into the feature selection framework to keep the similarity of the training samples. To do this, USFS methods usually construct the Laplacian matrix using either a general graph or a Hyper-graph on the original data. Since the construction of the Laplacian matrix is separated from the process of feature selection and the original data usually contain noise, the performance of feature selection will be influenced. Based on previous observation that a hyper-graph measuring the relationship among more than two samples is able to characterize more complex (i.e. high-order) relationship of the samples than a general graph measuring the relationship between two samples, in this paper, we propose a novel feature selection method by dynamically learning a hyper-graph based Laplacian matrix to measure the relationship among the samples for selecting the features. Experimental results on real datasets showed that our proposed method outperforms the state-of-the-art feature selection methods in terms of both clustering and segmentation performance.

A Manifold Laplacian Regularized Semi-Supervised Sparse Image Classification Method With a Variant Trace Lasso Norm

Since the cost of labeling data is getting higher and higher, we hope to make full use of the large amount of unlabeled data and improve image classification effect through adding some unlabeled samples for training. In addition, we expect to uniformly realize two tasks, namely the clustering of the unlabeled data and the recognition of the query image. We achieve the goal by designing a novel sparse model based on manifold assumption, which has been proved to work well in many tasks. Based on the assumption that images of the same class lie on a sub-manifold and an image can be approximately represented as the linear combination of its neighboring data due to the local linear property of manifold, we proposed a sparse representation model on manifold. Specifically, there are two regularizations, i.e., a variant Trace lasso norm and the manifold Laplacian regularization. The first regularization term enables the representation coefficients satisfying sparsity between groups and density within a group. And the second term is manifold Laplacian regularization by which label can be accurately propagated from labeled data to unlabeled data. Augmented Lagrange Multiplier (ALM) scheme and Gauss Seidel Alternating Direction Method of Multiplier (GS-ADMM) are given to solve the problem numerically. We conduct some experiments on three human face databases and compare the proposed work with several state-of-the-art methods. For each subject, some labeled face images are randomly chosen for training for those supervised methods, and a small amount of unlabeled images are added to form the training set of the proposed approach. All experiments show our method can get better classification results due to the addition of unlabeled samples.

Fast Graph Sampling Set Selection Using Gershgorin Disc Alignment

Graph sampling set selection, where a subset of nodes are chosen to collect samples to reconstruct a smooth graph signal, is a fundamental problem in graph signal processing (GSP). Previous works employ an unbiased least-squares (LS) signal reconstruction scheme and select samples via expensive extreme eigenvector computation. Instead, we assume a biased graph Laplacian regularization (GLR) based scheme that solves a system of linear equations for reconstruction. We then choose samples to minimize the condition number of the coefficient matrix---specifically, maximize the smallest eigenvalue $\lambda_{\min}$. Circumventing explicit eigenvalue computation, we maximize instead the lower bound of $\lambda_{\min}$, designated by the smallest left-end of all Gershgorin discs of the matrix. To achieve this efficiently, we first convert the optimization to a dual problem, where we minimize the number of samples needed to align all Gershgorin disc left-ends at a chosen lower-bound target $T$. Algebraically, the dual problem amounts to optimizing two disc operations: i) shifting of disc centers due to sampling, and ii) scaling of disc radii due to a similarity transformation of the matrix. We further reinterpret the dual as an intuitive disc coverage problem bearing strong resemblance to the famous NP-hard set cover (SC) problem. The reinterpretation enables us to derive a fast approximation scheme from a known SC error-bounded approximation algorithm. We find an appropriate target $T$ efficiently via binary search. Extensive simulation experiments show that our disc-based sampling algorithm runs substantially faster than existing sampling schemes and outperforms other eigen-decomposition-free sampling schemes in reconstruction error.

Fitting Laplacian Regularized Stratified Gaussian Models

We consider the problem of jointly estimating multiple related zero-mean Gaussian distributions from data. We propose to jointly estimate these covariance matrices using Laplacian regularized stratified model fitting, which includes loss and regularization terms for each covariance matrix, and also a term that encourages the different covariances matrices to be close. This method `borrows strength' from the neighboring covariances, to improve its estimate. With well chosen hyper-parameters, such models can perform very well, especially in the low data regime. We propose a distributed method that scales to large problems, and illustrate the efficacy of the method with examples in finance, radar signal processing, and weather forecasting.

Graph-Based Non-Convex Low-Rank Regularization for Image Compression Artifact Reduction.

Block transform coded images usually suffer from annoying artifacts at low bit-rates, because of the independent quantization of DCT coefficients. Image prior models play an important role in compressed image reconstruction. Natural image patches in a small neighborhood of the high-dimensional image space usually exhibit an underlying sub-manifold structure. To model the distribution of signal, we extract sub-manifold structure as prior knowledge. We utilize graph Laplacian regularization to characterize the sub-manifold structure at patch level. And similar patches are exploited as samples to estimate distribution of a particular patch. Instead of using Euclidean distance as similarity metric, we propose to use graph-domain distance to measure the patch similarity. Then we perform low-rank regularization on the similar-patch group, and incorporate a non-convex lp penalty to surrogate matrix rank. Finally, an alternatively minimizing strategy is employed to solve the non-convex problem. Experimental results show that our proposed method is capable of achieving more accurate reconstruction than the state-of-the-art methods in both objective and perceptual qualities.

An Active Learning Methodology for Efficient Estimation of Expensive Noisy Black-Box Functions Using Gaussian Process Regression

Estimation of black-box functions often requires evaluating an extensive number of expensive noisy points. Learning algorithms can actively compare the similarity between the evaluated and unevaluated points to determine the most informative subsequent points for efficient estimation of expensive functions in a sequential procedure. In this paper, we propose an active learning methodology based on the integration of Laplacian regularization and active learning - Cohn (ALC) measure for identification of the most informative points for efficient estimation of noisy black-box functions using Gaussian processes. We propose two simple greedy search algorithms for sequential optimization of the tuning parameters and determination of subsequent points based on the information from the previously evaluated points. We also enhance the graph Laplacian with the information of both the predictor and response variables to capture the similarity between the points more effectively. The proposed methodology is particularly suited for problems involving estimation of expensive black-box functions with a high level of noise and plenty of unevaluated points. Using a case study for analysis of the kinematics of pitching in baseball as well as simulation experiments, we demonstrate the performance of the proposed methodology against existing methods in the literature in terms of estimation error.

Point Cloud Denoising via Feature Graph Laplacian Regularization.

Point cloud is a collection of 3D coordinates that are discrete geometric samples of an object's 2D surfaces. Imperfection in the acquisition process means that point clouds are often corrupted with noise. Building on recent advances in graph signal processing, we design local algorithms for 3D point cloud denoising. Specifically, we design a signal-dependent feature graph Laplacian regularizer (SDFGLR) that assumes surface normals computed from point coordinates are piecewise smooth with respect to a signal-dependent graph Laplacian matrix. Using SDFGLR as a signal prior, we formulate an optimization problem with a general 'p-norm fidelity term that can explicitly remove only two types of additive noise: small but non-sparse noise like Gaussian (using '2 fidelity term) and large but sparser noise like Laplacian (using '1 fidelity term). To establish a linear relationship between normals and 3D point coordinates, we first perform bipartite graph approximation to divide the point cloud into two disjoint node sets (red and blue). We then optimize the red and blue nodes' coordinates alternately. For '2-norm fidelity term, we iteratively solve an unconstrained quadratic programming (QP) problem, efficiently computed using conjugate gradient with a bounded condition number to ensure numerical stability. For '1-norm fidelity term, we iteratively minimize an '1-'2 cost function using accelerated proximal gradient (APG), where a good step size is chosen via Lipschitz continuity analysis. Finally, we propose simple mean and median filters for flat patches of a given point cloud to estimate the noise variance given the noise type, which in turn is used to compute a weight parameter trading off the fidelity term and signal prior in the problem formulation. Extensive experiments show state-of-the-art denoising performance among local methods using our proposed algorithms.

3D Point Cloud Denoising Using Graph Laplacian Regularization of a Low Dimensional Manifold Model.

3D point cloud-a new signal representation of volumetric objects-is a discrete collection of triples marking exterior object surface locations in 3D space. Conventional imperfect acquisition processes of 3D point cloud-e.g., stereo-matching from multiple viewpoint images or depth data acquired directly from active light sensors-imply non-negligible noise in the data. In this paper, we extend a previously proposed low-dimensional manifold model for the image patches to surface patches in the point cloud, and seek self-similar patches to denoise them simultaneously using the patch manifold prior. Due to discrete observations of the patches on the manifold, we approximate the manifold dimension computation defined in the continuous domain with a patch-based graph Laplacian regularizer, and propose a new discrete patch distance measure to quantify the similarity between two same-sized surface patches for graph construction that is robust to noise. We show that our graph Laplacian regularizer leads to speedy implementation and has desirable numerical stability properties given its natural graph spectral interpretation. Extensive simulation results show that our proposed denoising scheme outperforms state-of-the-art methods in objective metrics and better preserves visually salient structural features like edges.