Manifold Regularization Framework Research Articles (Page 1)

With the rapid advancement of 3D scanning technology, point clouds have become a crucial data type in computer vision and machine learning. However, learning robust representations for point clouds remains a significant challenge due to their irregularity and sparsity. In this paper, we propose a novel Dual Manifold Regularization (DMR) framework that makes full use of the properties of positive and negative curvature in manifolds to improve the representation of point clouds. Specifically, we leverage DMR based on hyperbolic and hyperspherical manifolds to address the limitations of traditional single-manifold regularization techniques, including inadequate generalization ability and adaptability to data diversity, as well as the difficulty of capturing complex relationships between data. To begin, we utilize the tree-like structure of the hyperbolic manifold to model the part-whole hierarchical relationships within point clouds. This allows for a more comprehensive representation of the data, improving the model's capability to understand complex shapes. Additionally, we construct positive samples through topological consistency augmentation and employ contrastive learning techniques in the hyperspherical manifold to capture more discriminative features within the data. Our experimental results show that our method outperforms traditional supervised learning and single-manifold regularization techniques in point cloud analysis. Specifically, for shape classification, DMR achieves a new State-Of-The-Art (SOTA) performance with 94.8% Overall Accuracy (OA) on ModelNet40 and 90.7% OA on ScanObjectNN, surpassing the recent SOTA model without increasing the baseline parameters.

Semi-Supervised Stochastic Configuration Networks Based on Manifold Regularization Framework

Cross-modal and multimodal data analysis based on functional mapping of spectral descriptors and manifold regularization

Discriminative sparse subspace learning with manifold regularization

Transductive transfer broad learning for cross-domain information exploration and multigrade soft sensor application

This paper aims to propose a framework for manifold regularization (MR) based distributed semi-supervised learning (DSSL) using single layer feed-forward neural network (SLFNN). The proposed framework, denoted as DSSL-SLFNN is based on the SLFNN, MR framework, and distributed optimization strategy. Then, a series of algorithms are derived to solve DSSL problems. In DSSL problems, data consisting of labeled and unlabeled samples are distributed over a communication network, where each node has only access to its own data and can only communicate with its neighbors. In some scenarios, DSSL problems cannot be solved by centralized algorithms. According to the DSSL-SLFNN framework, each node over the communication network exchanges the initial parameters of the SLFNN with the same basis functions for semi-supervised learning (SSL). All nodes calculate the global optimal coefficients of the SLFNN by using distributed datasets and local updates. During the learning process, each node only exchanges local coefficients with its neighbors rather than raw data. It means that DSSL-SLFNN based algorithms work in a fully distributed fashion and are privacy preserving methods. Finally, several simulations are presented to show the efficiency of the proposed framework and the derived algorithms.

Recently, broad learning system (BLS) has been introduced to solve industrial fault diagnosis problems and has achieved impressive performance. As a flat network, BLS enjoys a simple linear structure, which enables BLS to train and update the model efficiently in an incremental manner, and it potentially has better generalization capacity than deep learning methods when training data are limited. The basic BLS is a supervised learning method that requires all the training data to be labeled. However, in many practical industrial scenarios, data labels are usually difficult to obtain. Existing semisupervised variant uses manifold regularization framework to capture the information of unlabeled data, however, such a method will sacrifice the incremental learning capacity of BLS. Considering that in many practical applications, training data are sequentially generated, in this article, an online semisupervised broad learning system (OSSBLS) is proposed for fault diagnosis in these cases. The proposed method not only can efficiently construct and incrementally update the model, but also can take advantage of unlabeled data to improve the model's diagnostic performance. Experimental results on the Tennessee Eastman process and a real-world air compressor working process demonstrate the superiority of OSSBLS in terms of both diagnostic performance and time consumption.

Electroencephalography (EEG) signal classification is a crucial part in motor imagery brain–computer interface (BCI) system. Traditional supervised learning methods have performed well pleasing in EEG classification. Unfortunately, the unlabeled samples are easier to collect than labeled samples. In addition, recent studies have shown that it may degenerate performance of semi-supervised learning by exploiting unlabeled samples without selection. To address these issues, a novel semi-supervised broad learning system with transfer learning (TSS-BLS) is proposed in this paper. First, the pseudo-labels of unlabeled samples are obtained using the joint distribution adaptation algorithm. TSS-BLS is then constructed by an improved manifold regularization framework containing both labeled and pseudo-label information. Finally, the effectiveness of the proposed TSS-BLS is evaluated on three BCI competition datasets and four benchmark datasets from UCI repository and compared with seven state-of-the-art algorithms, including ELM, SS-ELM, HELM, SVM, LapSVM, BLS and GSS-BLS. Experimental results show that the performance of TSS-BLS is superior to BLS and GSS-BLS on average. It is thereby shown that TSS-BLS is safe and efficient for EEG classification.

Hyperspectral image classification with discriminative manifold broad learning system

Improving the predictions of soil properties from VNIR–SWIR spectra in an unlabeled region using semi-supervised and active learning

Artificial neural networks, consisting of many levels of nonlinearities, have been widely used to deal with various supervised learning tasks. At present, the most popular and effective training method is back-propagation algorithm (BP). Inspired by manifold regularization framework, we introduce a novel regularization framework, which aims at preserving the inter-object-distance of the data. Then a refined BP algorithm (IOD-BP) is proposed by imposing the proposed regularization framework into the objective function of BP algorithm. Comparative experiments on various benchmark classification tasks show that the new regularization BP method significantly improves the performance of BP algorithm in terms of classification accuracy.

Broad Learning System (BLS) are widely used in many fields because of its strong feature extraction ability and high computational efficiency. However, the BLS is mainly used in supervised learning, which greatly limits the applicability of the BLS. And the obtained data is less labeled data, but is a large number of unlabeled data. Therefore, the BLS is extended based on the semi-supervised learning of manifold regularization framework to propose a semi-supervised broad learning system (SS-BLS). Firstly, the features are extracted from labeled and unlabeled data by building feature nodes and enhancement nodes. Then the manifold regularization framework is used to construct Laplacian matrix. Next, the feature nodes, enhancement nodes and Laplacian matrix are combined to construct the objective function, which is effectively solved by ridge regression in order to obtain the output coefficients. Finally, the validity of the SS-BLS is verified by three different complex data of G50C, MNIST, and NORB, respectively. The experiment result show that the SS-BLS can achieve higher classification accuracy for different complex data, takes on fast operation speed and strong generalization ability.

Manifold regularization based distributed semi-supervised learning algorithm using extreme learning machine over time-varying network

This paper frames causal structure estimation as a machine learning task. The idea is to treat indicators of causal relationships between variables as 'labels' and to exploit available data on the variables of interest to provide features for the labelling task. Background scientific knowledge or any available interventional data provide labels on some causal relationships and the remainder are treated as unlabelled. To illustrate the key ideas, we develop a distance-based approach (based on bivariate histograms) within a manifold regularization framework. We present empirical results on three different biological data sets (including examples where causal effects can be verified by experimental intervention), that together demonstrate the efficacy and general nature of the approach as well as its simplicity from a user's point of view.

Video salient object detection aims at distinguishing the salient objects from the complex background and highlighting them uniformly in the spatiotemporal domain, which still suffers from the interference of the complicated dynamic background in unconstrained videos. To address this problem, we propose a novel coarse-to-fine spatiotemporal salient object detection method. Specifically, we first model a novel motion energy to exclude the motion noise by exploiting the motion magnitude and motion orientation. Then, a supervoxel-level inter-frame graph model is constructed for each pair of adjacent frames independently, and a robust graph clustering-based saliency seed generation method is proposed to produce a coarse saliency map. Furthermore, the supervoxel-level inter-frame graph model is reconstructed by considering the regional spatiotemporal consistency constraint based on the coarse saliency map. The prior information obtained from pixel clustering is also taken into account to optimize the weight of the inter-frame graph model. Finally, a multi-graphs saliency propagation method is exploited under the manifold regularization framework by fusing the motion energy and appearance feature to refine the coarse saliency map. The extensive experiments on two widely used datasets validate the effectiveness and superiority of the proposed method against 13 state-of-the-art methods in terms of PR-curves, scores of S-measure, $F_{\beta }$ , and MAE.

For semi-supervised learning, we propose the Laplacian embedded multiple kernel regression model. As we incorporate the multiple kernel occasion into manifold regularization framework, the models we proposed are flexible in many kinds of datasets and have a solid theoretical foundation. The proposed model can solve the two problems, which are the computation cost of manifold regularization framework and the difficulty in dealing with multi-source or multi-attribute datasets. Though manifold regularization is a convex optimization formulation, it often leads to dense matrix inversion with computation cost. Laplacian embedded method we adopted can solve the problem, however it lacks the proper ability to process complex datasets. Therefore, we further use multiple kernel learning as a part of the proposed model to strengthen its ability. Experiments on several datasets compared with the state-of-the-art methods show the effectiveness and efficiency of the proposed model.

Updating initial labels from spectral graph by manifold regularization for saliency detection

Data labeling is a tedious and subjective task that can be time consuming and error-prone; however, most learning algorithms are sensitive to noisy labels. This problem raises the need to develop algorithms that can exploit large amount of unlabeled data and also be robust to noisy label information. In this letter, we propose a novel semi-supervised classification framework that is robust to noisy labels, named self-paced manifold regularization. The proposed framework naturally integrates self-paced learning regime into the manifold regularization framework for selecting labeled training samples in a theoretically sound manner, and utilizes locally linear reconstructions to control the smoothness of the classifier with respect to the manifold structure of data. Finally, the alternative search strategy is adopted for the proposed framework to obtain the classifier. The proposed method can not only suppress the negative effect of noisy initial labels in semi-supervised learning, but also obtain an explicit multiclass classifier for newly coming data points. Experimental results demonstrate the effectiveness of the proposed method.

Manifold regularization (MR) has become one of the most widely used approaches in the semi-supervised learning field. It has shown superiority by exploiting the local manifold structure of both labeled and unlabeled data. The manifold structure is modeled by constructing a Laplacian graph and then incorporated in learning through a smoothness regularization term. Hence the labels of labeled and unlabeled data vary smoothly along the geodesics on the manifold. However, MR has ignored the discriminative ability of the labeled and unlabeled data. To address the problem, we propose an enhanced MR framework for semi-supervised classification in which the local discriminative information of the labeled and unlabeled data is explicitly exploited. To make full use of labeled data, we firstly employ a semi-supervised clustering method to discover the underlying data space structure of the whole dataset. Then we construct a local discrimination graph to model the discriminative information of labeled and unlabeled data according to the discovered intrinsic structure. Therefore, the data points that may be from different clusters, though similar on the manifold, are enforced far away from each other. Finally, the discrimination graph is incorporated into the MR framework. In particular, we utilize semi-supervised fuzzy c-means and Laplacian regularized Kernel minimum squared error for semi-supervised clustering and classification, respectively. Experimental results on several benchmark datasets and face recognition demonstrate the effectiveness of our proposed method.

In this paper, we propose a block-polynomial mapping for image feature learning, which can be efficiently represented by the matrix Khatri-Rao product. The block-polynomial mapping not only captures the local discriminative information within the image structure, but is also much more efficient than the traditional kernel mapping. Moreover, we embed the proposed mapping into the manifold regularization framework for semi-supervised image classification. Experimental results demonstrate that, while maintaining a comparable classification accuracy, the proposed algorithm performs much more efficient than the state-of-the-art methods.