Local Structure Learning Research Articles

PSL: An Algorithm for Partial Bayesian Network Structure Learning

Learning partial Bayesian network (BN) structure is an interesting and challenging problem. In this challenge, it is computationally expensive to use global BN structure learning algorithms, while only one part of a BN structure is interesting, local BN structure learning algorithms are not a favourable solution either due to the issue of false edge orientation. To address the problem, this article first presents a detailed analysis of the false edge orientation issue with local BN structure learning algorithms and then proposes PSL, an efficient and accurate P artial BN S tructure L earning (PSL) algorithm. Specifically, PSL divides V-structures in a Markov blanket (MB) into two types: Type-C V-structures and Type-NC V-structures, then it starts from the given node of interest and recursively finds both types of V-structures in the MB of the current node until all edges in the partial BN structure are oriented. To further improve the efficiency of PSL, the PSL-FS algorithm is designed by incorporating F eature S election (FS) into PSL. Extensive experiments with six benchmark BNs validate the efficiency and accuracy of the proposed algorithms.

Multi-view Spectral Clustering with Adaptive Graph Learning and Tensor Schatten p-norm

Although the promising clustering performance, existing graph-based multi-view clustering methods still have the following deficiencies. (1) They cannot well exploit the salient difference between graphs of different views, resulting in suboptimal graphs. (2) They minimize the divergence between graphs of different views by one-dimensional, element by element metric model. Thus, they cannot well exploit the complementary information embedded in graphs. (3) They fail to simultaneously take local and global intrinsic geometric structures into account, resulting in suboptimal clustering performance. To handle the aforementioned problems, we propose Multi-view Spectral Clustering with Adaptive Graph Learning and Tensor Schatten p-norm. Specifically, we present an adaptive weighted strategy that directly takes into account the contribution among different views for clustering. To well learn a good graph, which well characterizes cluster structure, we integrate local and global structure learning into a unified framework and leverage tensor Schatten p-norm regularizer to minimize the divergence between graphs of different views. Extensive experiments on benchmark datasets have demonstrated the superiority of the proposed method.

Consensus Affinity Graph Learning for Multiple Kernel Clustering.

Significant attention to multiple kernel graph-based clustering (MKGC) has emerged in recent years, primarily due to the superiority of multiple kernel learning (MKL) and the outstanding performance of graph-based clustering. However, many existing MKGC methods design a fat model that poses challenges for computational cost and clustering performance, as they learn both an affinity graph and an extra consensus kernel cumbersomely. To tackle this challenging problem, this article proposes a new MKGC method to learn a consensus affinity graph directly. By using the self-expressiveness graph learning and an adaptive local structure learning term, the local manifold structure of the data in kernel space is preserved for learning multiple candidate affinity graphs from a kernel pool first. After that, these candidate affinity graphs are synthesized to learn a consensus affinity graph via a thin autoweighted fusion model, in which a self-tuned Laplacian rank constraint and a top- k neighbors sparse strategy are introduced to improve the quality of the consensus affinity graph for accurate clustering purposes. The experimental results on ten benchmark datasets and two synthetic datasets show that the proposed method consistently and significantly outperforms the state-of-the-art methods.

Simultaneous Global and Local Graph Structure Preserving for Multiple Kernel Clustering.

Multiple kernel learning (MKL) is generally recognized to perform better than single kernel learning (SKL) in handling nonlinear clustering problem, largely thanks to MKL avoids selecting and tuning predefined kernel. By integrating the self-expression learning framework, the graph-based MKL subspace clustering has recently attracted considerable attention. However, the graph structure of data in kernel space is largely ignored by previous MKL methods, which is a key concept of affinity graph construction for spectral clustering purposes. In order to address this problem, a novel MKL method is proposed in this article, namely, structure-preserving multiple kernel clustering (SPMKC). Specifically, SPMKC proposes a new kernel affine weight strategy to learn an optimal consensus kernel from a predefined kernel pool, which can assign a suitable weight for each base kernel automatically. Furthermore, SPMKC proposes a kernel group self-expressiveness term and a kernel adaptive local structure learning term to preserve the global and local structure of the input data in kernel space, respectively, rather than the original space. In addition, an efficient algorithm is proposed to solve the resulting unified objective function, which iteratively updates the consensus kernel and the affinity graph so that collaboratively promoting each of them to reach the optimum condition. Experiments on both image and text clustering demonstrate that SPMKC outperforms the state-of-the-art MKL clustering methods in terms of clustering performance and computational cost.

Towards Efficient Local Causal Structure Learning

Local causal structure learning aims to discover and distinguish direct causes (parents) and direct effects (children) of a variable of interest from data. While emerging successes have been made, existing methods need to search a large space to distinguish direct causes from direct effects of a target variable T. To tackle this issue, we propose a novel Efficient Local Causal Structure learning algorithm, named ELCS. Specifically, we first propose the concept of N-structures, then design an efficient Markov Blanket (MB) discovery subroutine to integrate MB learning with N-structures to learn the MB of T and simultaneously distinguish direct causes from direct effects of T. With the proposed MB subroutine, ELCS starts from the target variable, sequentially finds MBs of variables connected to the target variable and simultaneously constructs local causal structures over MBs until the direct causes and direct effects of the target variable have been distinguished. Using eight Bayesian networks the extensive experiments have validated that ELCS achieves better accuracy and efficiency than the state-of-the-art algorithms.

Semisupervised Band Selection With Graph Optimization for Hyperspectral Image Classification

Semisupervised band selection (BS) technique plays an important role in processing hyperspectral images (HSIs) because of its superiority of using the limited labeled data and plentiful unlabeled data to select the discriminative and informative feature, which copes with the problem of high-dimensional and scare labeled samples of HSIs. Among semisupervised BS models, graph-based methods are superior to others in many situations and have received more and more attention. However, traditional graph-based models construct a similarity matrix and select valuable bands independently. The similarity matrix remains constant, which will damage the local manifold structure and lead to a suboptimal result. To solve this problem, we propose a semisupervised band selection with an optimal graph (BSOG) approach, which performs BS and local structure learning simultaneously. Instead of fixing the input similarity matrix, the similarity matrix is updated constantly to learn a better local structure. Besides, the learned similarity matrix is adaptive. Then, the optimal band subset can be selected by analyzing the obtained projection matrix <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol W$ </tex-math></inline-formula> . An efficient and simple optimization algorithm is proposed to solve this model. Experiments on four real HSIs data validate the effectiveness of the proposed model.

Simultaneous learning coefficient matrix and affinity graph for multiple kernel clustering

Due to the effectiveness of handling non-linear data and avoiding kernel customization, multiple kernel clustering (MKC) has been widely investigated and achieved promising results for challenging non-linear clustering tasks. Generally, the existing MKC methods mainly consist of first learning a coefficient matrix by leveraging multiple kernel learning (MKL) and sample self-expressiveness property, and then constructing an affinity graph relying on this coefficient matrix to accomplish spectral clustering. However, the quality of the affinity matrix (graph) is largely determined by the learned coefficient matrix, thus the two independent steps are not conducive to learn an optimal affinity graph. To tackle this problem, in this paper, we propose a new MKC method that uses one-step learning scheme rather than two-step learning scheme to learn an affinity graph, termed SLMKC. Specifically, SLMKC bridges the relationship between the affinity matrix and coefficient matrix by an adaptive local structure learning strategy, so it can simultaneously learn both in a mutual promotion manner. Furthermore, a self-weighted MKL strategy is introduced to learn an optimal consensus kernel, which can avoid selecting a specific kernel function and tuning its associated parameters. Extensive experiments validate that our SLMKC outperforms the state-of-the-art MKC competitors significantly.

Local Structure Preservation for Nonlinear Clustering

In this paper, we propose a new nonlinear clustering method to preserve local structure of the features. Specifically, our method applies the gaussian kernel function to achieve high dimensional projection so as to make the original data linearly separable. Our method establishes the similarity matrix of data features in low-dimensional space to conduct local structure learning, as a result, it can avoid the divergence of sample sets and retain the original nearest neighbor structural relations. Furthermore, our method uses the sparse learning to remove the redundant features to make the model more robust in the process of learning. Experimental results on eight benchmark datasets show that our proposed method was superior to the state-of-the-art clustering methods in terms of clustering performance.

Using Feature Selection for Local Causal Structure Learning

Local causal structure learning aims to discover and distinguish the direct causes and direct effects of a target variable. However, the state-of-the-art local causal structure learning algorithms need to perform an exhaustive subset search within the currently selected variables for PC (i.e., parents and children) discovery. In this article, we propose an efficient local causal structure learning algorithm around a target variable, called LCS-FS (Local Causal Structure learning by Feature Selection). First, to construct the local causal skeleton of the target, we employ feature selection for finding PC without searching for conditioning sets to speed up PC discovery, leading to improve the skeleton construction efficiency. Second, to orient edges in this local causal skeleton, we propose an efficient method to find separating sets from the subsets of PC for identifying V-structures. With the integration of feature selection and the new way of finding separating sets, LCS-FS recursively finds the spouses of Markov blankets in local causal skeleton for edge orientations, until the direct causes and direct effects of the target are distinguished. The experiments on five benchmark Bayesian networks with the number of variables from 35 to 801 validate that our algorithm achieves higher efficiency and better accuracy than the state-of-the-art local causal structure learning algorithms.

Structured optimal graph based sparse feature extraction for semi-supervised learning

Graph-based feature extraction is an efficient technique for data dimensionality reduction, and it has gained intensive attention in various fields such as image processing, pattern recognition, and machine learning. However, conventional graph-based dimensionality reduction algorithms usually depend on a fixed weight graph called similarity matrix, which seriously affects the subsequent feature extraction process. In this paper, a novel structured optimal graph based sparse feature extraction (SOGSFE) method for semi-supervised learning is proposed. In the proposed method, the local structure learning, sparse representation, and label propagation are simultaneously framed to perform data dimensionality reduction. In particular the similarity matrix and the projection matrix are obtained by an iterative calculation manner. The experimental results on several public image datasets demonstrate the robustness and effectiveness of the proposed method.

Prior-Knowledge-Driven Local Causal Structure Learning and Its Application on Causal Discovery Between Type 2 Diabetes and Bone Mineral Density

Type 2 diabetes (T2DM), one of the most prevalent chronic diseases, affects the glucose metabolism of the human body, which decreases the quantity of life and brings a heavy burden on social medical care. Patients with T2DM are more likely to suffer bone fragility fracture as diabetes affects bone mineral density (BMD). However, the discovery of the determinant factors of BMD in a medical way is expensive and time-consuming. In this paper, we propose a novel algorithm, Prior-Knowledge-driven local Causal structure Learning (PKCL), to discover the underlying causal mechanism between BMD and its factors from the clinical data. Since there exist limited data but redundant prior knowledge for medicine, PKCL adequately utilize the prior knowledge to mine the local causal structure for the target relationship. Combining the medical prior knowledge with the discovered causal relationships, PKCL can achieve more reliable results without long-standing medical statistical experiments. Extensive experiments are conducted on a newly provided clinical data set. The experimental study of PKCL on the data is proved to highly corresponding with existing medical knowledge, which demonstrates the superiority and effectiveness of PKCL. To illustrate the importance of prior knowledge, the result of the algorithm without prior knowledge is also investigated.

Unsupervised feature selection based on local structure learning

Unsupervised feature selection has become a significant and ambitious issue due to vast amounts of high-dimensional unlabeled data in machine learning. Traditional unsupervised feature selection algorithms usually make use of the similarity matrix for feature selection, and they heavily rely on the learned structure. However, a large amount of actual data always contains many noise samples or features that may make the similarity matrix obtained from the original data unreliable. Using Ideal Local Structure Learning (LSL) method ,we propose a novel unsupervised feature selection to perform feature selection and local structure learning at the same time in this paper. In order that we can earn more exactly structure information, an ideal local structure with precisely c connected components of data (c is the number of clusters) is utilized to refine the similarity matrix. Moreover, in order to optimize our algorithm, an effectual and plain iterative algorithm is developed. Experiments on multiple public baseline datasets, including biomedical data, letter recognition digit data and face image data, reveals the outstanding performance of our algorithms in the most advanced aspects.

Regularized nonnegative matrix factorization with adaptive local structure learning

Due to the effectiveness of Nonnegative Matrix Factorization (NMF) and its graph regularized extensions, these methods have been received much attention from various researchers. Generally, these methods are performed in two separate steps including Laplacian graph construction and the subsequent matrix decomposition.However, the similarity measurement for Laplacian graph is challenging since it’s often affected by several factors such as the neighborhood size, choice of similarity metric, etc. As a result, the learned graph may be not suitable, let alone the subsequent matrix decomposition. In this paper, we propose adaptive graph regularized NMF. Different from existing methods, the similarity matrix is automatically learned from the data. The proposed model can simultaneously performs matrix decomposition and similarity learning. By balancing the interactions between both of the two subtasks in our model, each subtask is improved iteratively based on the result of another. Experimental results on benchmark data sets illustrate the effectiveness of our model.

PEnBayes: A Multi-Layered Ensemble Approach for Learning Bayesian Network Structure from Big Data.

Discovering the Bayesian network (BN) structure from big datasets containing rich causal relationships is becoming increasingly valuable for modeling and reasoning under uncertainties in many areas with big data gathered from sensors due to high volume and fast veracity. Most of the current BN structure learning algorithms have shortcomings facing big data. First, learning a BN structure from the entire big dataset is an expensive task which often ends in failure due to memory constraints. Second, it is quite difficult to select a learner from numerous BN structure learning algorithms to consistently achieve good learning accuracy. Lastly, there is a lack of an intelligent method that merges separately learned BN structures into a well structured BN network. To address these shortcomings, we introduce a novel parallel learning approach called PEnBayes (Parallel Ensemble-based Bayesian network learning). PEnBayes starts with an adaptive data preprocessing phase that calculates the Appropriate Learning Size and intelligently divides a big dataset for fast distributed local structure learning. Then, PEnBayes learns a collection of local BN Structures in parallel using a two-layered weighted adjacent matrix-based structure ensemble method. Lastly, PEnBayes merges the local BN Structures into a global network structure using the structure ensemble method at the global layer. For the experiment, we generate big data sets by simulating sensor data from patient monitoring, transportation, and disease diagnosis domains. The Experimental results show that PEnBayes achieves a significantly improved execution performance with more consistent and stable results compared with three baseline learning algorithms.

Sparse Nonlinear Feature Selection Algorithm via Local Structure Learning

In this paper, we propose a new unsupervised feature selection algorithm by considering the nonlinear and similarity relationships within the data. To achieve this, we apply the kernel method and local structure learning to consider the nonlinear relationship between features and the local similarity between features. Specifically, we use a kernel function to map each feature of the data into the kernel space. In the high-dimensional kernel space, different features correspond to different weights, and zero weights are unimportant features (e.g. redundant features). Furthermore, we consider the similarity between features through local structure learning, and propose an effective optimization method to solve it. The experimental results show that the proposed algorithm achieves better performance than the comparison algorithm.

Evaluation of Causal Effects and Local Structure Learning of Causal Networks

Causal effect evaluation and causal network learning are two main research areas in causal inference. For causal effect evaluation, we review the two problems of confounders and surrogates. The Yule-Simpson paradox is the idea that the association between two variables may be changed dramatically due to ignoring confounders. We review criteria for confounders and methods of adjustment for observed and unobserved confounders. The surrogate paradox occurs when a treatment has a positive causal effect on a surrogate endpoint, which, in turn, has a positive causal effect on a true endpoint, but the treatment may have a negative causal effect on the true endpoint. Some of the existing criteria for surrogates are subject to the surrogate paradox, and we review criteria for consistent surrogates to avoid the surrogate paradox. Causal networks are used to depict the causal relationships among multiple variables. Rather than discovering a global causal network, researchers are often interested in discovering the causes and effects of a given variable. We review some algorithms for local structure learning of causal networks centering around a given variable.

Improving Bayesian network local structure learning via data-driven symmetry correction methods

Learning the structure of a Bayesian network (BN) from data is NP-hard. To efficiently handle high-dimensional datasets, many BN local structure learning algorithms are proposed. These learning algorithms can be categorized into two types: constraint-based and score-based. These learning algorithms learn the local structures separately for each node. As a result, asymmetric pairs of neighbors and Markov blankets create conflicts between the local structures. To resolve the conflicts, symmetry correction is required. The commonly used AND-rule symmetry correction method, which simply drops nodes in asymmetric pairs from the neighbor sets and Markov blankets of both nodes, may result in loss of information in learning the BN. In this paper, we propose a hybrid framework that combines a local structure learning algorithm of a particular type (either constraint-based or score-based) with a data-driven symmetry correction method of the other type. The score-based symG method and the constraint-based symC method are proposed to be used in the hybrid framework. Empirical results show that performances of constraint-based learning algorithms are improved by using the proposed score-based symG method. Similarly, the performance of score-based local learning algorithm is better when symC is used, compared to using symG.

Structured Graph Optimization for Unsupervised Feature Selection

Unsupervised feature selection has attracted more and more attention due to the rapid growth of the large amount of unlabelled and high-dimensional data. The performance of traditional spectral-based unsupervised methods always depends on the quality of constructed similarity matrix. However, real world data always contain a large number of noise samples and features that make the similarity matrix created by original data cannot be fully relied. We propose an unsupervised feature selection method which conducts feature selection and local structure learning simultaneously. Moreover, we add an important constraint on the similarity matrix to allow it to capture more accurate information of the data structure. To perform feature selection, orthogonal constraint and $\ell _{2,p}$ l 2 , p -norm are adopted on the projection matrix. An efficient and simple algorithm is derived to tackle the problem. We conduct comprehensive experiments on various benchmark data sets, including handwritten digit, face image, and biomedical data, to validate the effectiveness of the proposed approach.

Local structure learning of chain graphs with the false discovery rate control

Chain graphs (CGs) containing both directed and undirected edges, offer an elegant generalisation of both Markov networks and Bayesian networks. In this paper, we propose an algorithm for local structure learning of CGs. It works by first learning adjacent nodes of each variable for skeleton identification and then orienting the edges of the complexes of the graph. To control the false discovery rate (FDR) of edges when learning a CG, FDR controlling procedure is embedded in the algorithm. Algorithms for skeleton identification and complexes recovery are presented. Experimental results demonstrate that the algorithm with the FDR controlling procedure can control the false discovery rate of the skeleton of the recovered graph under a user-specified level, and the proposed algorithm is also a viable alternative to learn the structure of chain graphs.

Unsupervised feature selection via local structure learning and sparse learning

Feature self-representation has become the backbone of unsupervised feature selection, since it is almost insensitive to noise data. However, feature selection methods based on feature self-represe...