Graph-based Dimensionality Reduction Method Research Articles

Multi-View Projection Learning via Adaptive Graph Embedding for Dimensionality Reduction

In order to explore complex structures and relationships hidden in data, plenty of graph-based dimensionality reduction methods have been widely investigated and extended to the multi-view learning field. For multi-view dimensionality reduction, the key point is extracting the complementary and compatible multi-view information to analyze the complex underlying structure of the samples, which is still a challenging task. We propose a novel multi-view dimensionality reduction algorithm that integrates underlying structure learning and dimensionality reduction for each view into one framework. Because the prespecified graph derived from original noisy high-dimensional data is usually low-quality, the subspace constructed based on such a graph is also low-quality. To obtain the optimal graph for dimensionality reduction, we propose a framework that learns the affinity based on the low-dimensional representation of all views and performs the dimensionality reduction based on it jointly. Although original data is noisy, the local structure information of them is also valuable. Therefore, in the graph learning process, we also introduce the information of predefined graphs based on each view feature into the optimal graph. Moreover, assigning the weight to each view based on its importance is essential in multi-view learning, the proposed GoMPL automatically allocates an appropriate weight to each view in the graph learning process. The obtained optimal graph is then adopted to learn the projection matrix for each individual view by graph embedding. We provide an effective alternate update method for learning the optimal graph and optimal subspace jointly for each view. We conduct many experiments on various benchmark datasets to evaluate the effectiveness of the proposed method.

Discriminative and Geometry-Preserving Adaptive Graph Embedding for dimensionality reduction

Learning graph embeddings for high-dimensional data is an important technology for dimensionality reduction. The learning process is expected to preserve the discriminative and geometric information of high-dimensional data in a new low-dimensional subspace via either manual or automatic graph construction. Although both manual and automatic graph constructions can capture the geometry and discrimination of data to a certain degree, they working alone cannot fully explore the underlying data structure. To learn and preserve more discriminative and geometric information of the high-dimensional data in the low-dimensional subspace as much as possible, we develop a novel Discriminative and Geometry-Preserving Adaptive Graph Embedding (DGPAGE). It systematically integrates manual and adaptive graph constructions in one unified graph embedding framework, which is able to effectively inject the essential information of data involved in predefined graphs into the learning of an adaptive graph, in order to achieve both adaptability and specificity of data. Learning the adaptive graph jointly with the optimized projections, DGPAGE can generate an embedded subspace that has better pattern discrimination for image classification. Results derived from extensive experiments on image data sets have shown that DGPAGE outperforms the state-of-the-art graph-based dimensionality reduction methods. The ablation studies show that it is beneficial to have an integrated framework, like DGPAGE, that brings together the advantages of manual/adaptive graph construction.

Convolutional Graph Thermography for Subsurface Defect Detection in Polymer Composites

Infrared thermography for quality assessment of polymer composites has gained increasing attention with the development of various thermographic data analysis methods. However, low-quality thermal images containing noise and inhomogeneous backgrounds restrict the defect detection performance of these methods. In this work, a convolutional graph thermography (CGT) method is proposed for subsurface defect detection in polymer composites. With a developed convolutional autoencoder thermogram enhancer, the noise and inhomogeneous background of the thermal image can be reduced. Subsequently, a graph-based dimensionality reduction method performs feature extraction on the enhanced thermograms. Consequently, the low-dimensional intrinsic manifolds of the high-dimensional thermographic data are characterized to highlight the defects visually. Two case studies on carbon fiber reinforced polymers demonstrated the performance of the proposed method.

Local Preserving Graphs Using Intra-Class Competitive Representation for Dimensionality Reduction of Hyperspectral Image

As a key technique in hyperspectral image pre-processing, dimensionality reduction has received a lot of attention. However, most of the graph-based dimensionality reduction methods only consider a single structure in the data and ignore the interfusion of multiple structures. In this paper, we propose two methods for combining intra-class competition for locally preserved graphs by constructing a new dictionary containing neighbourhood information. These two methods explore local information into the collaborative graph through competing constraints, thus effectively improving the overcrowded distribution of intra-class coefficients in the collaborative graph and enhancing the discriminative power of the algorithm. By classifying four benchmark hyperspectral data, the proposed methods are proved to be superior to several advanced algorithms, even under small-sample-size conditions.

Adaptive Flexible Optimal Graph for Unsupervised Dimensionality Reduction

Graph-based dimensionality reduction methods are widely used in classification and clustering tasks due to their superior performance. The key to the performance of these methods is how to construct the graph. Although some existing methods can obtain an adaptive graph and a projection matrix simultaneously by combining manifold learning and graph construction in a unified framework, the linear constraint used in manifold learning is too hard . To solve this problem, we propose a novel method named adaptive flexible optimal graph (AFOG) for unsupervised dimensionality reduction. AFOG can obtain an adaptive flexible optimal graph by relaxing the linear constraint in the process of low-dimensional manifold mapping. At the same time, by using the principle of maximum separability, it can also obtain an effective projection matrix, which can solve out-ofsample problems. Experiments on six public benchmark data sets indicate that AFOG outperforms several other state-of-the-art methods.

Low-rank graph optimization for multi-view dimensionality reduction.

Graph-based dimensionality reduction methods have attracted substantial attention due to their successful applications in many tasks, including classification and clustering. However, most classical graph-based dimensionality reduction approaches are only applied to data from one view. Hence, combining information from different data views has attracted considerable attention in the literature. Although various multi-view graph-based dimensionality reduction algorithms have been proposed, the graph construction strategies utilized in them do not adequately take noise and different importance of multiple views into account, which may degrade their performance. In this paper, we propose a novel algorithm, namely, Low-Rank Graph Optimization for Multi-View Dimensionality Reduction (LRGO-MVDR), that overcomes these limitations. First, we construct a low-rank shared matrix and a sparse error matrix from the graph that corresponds to each view for capturing potential noise. Second, an adaptive nonnegative weight vector is learned to explore complementarity among views. Moreover, an effective optimization procedure based on the Alternating Direction Method of Multipliers scheme is utilized. Extensive experiments are carried out to evaluate the effectiveness of the proposed algorithm. The experimental results demonstrate that the proposed LRGO-MVDR algorithm outperforms related methods.

Discriminative globality and locality preserving graph embedding for dimensionality reduction

Graph embedding in dimensionality reduction has attracted much attention in the high-dimensional data analysis. Graph construction in graph embedding plays an important role in the quality of dimensionality reduction. However, the discrimination information and the geometrical distributions of data samples are not fully exploited for discovering the essential geometrical and discriminant structures of data and strengthening the pattern discrimination in graph constructions of graph embedding. To overcome the limitations of graph constructions, in this article we propose a novel graph-based dimensionality reduction method entitled discriminative globality and locality preserving graph embedding (DGLPGE) by designing the informative globality and locality preserving graph constructions. In the constructed graphs, bidirectional weights of edges are newly defined by considering both the geometrical distributions of each point of edges and the class discrimination. Using the adjacent weights of graphs, we characterize the intra-class globality preserving scatter, the inter-class globality preserving scatter and the locality preserving scatter to formulate the objective function of DGLPGE in order to optimize the projection of dimensionality reduction. Extensive experiments demonstrate that the proposed DGLPGE often outperforms the state-of-the-art dimensionality reduction methods.

A Semisupervised Spatial Spectral Regularized Manifold Local Scaling Cut With HGF for Dimensionality Reduction of Hyperspectral Images

Hyperspectral images (HSI) contain a wealth of information over hundreds of contiguous spectral bands, making it possible to classify materials through subtle spectral discrepancies. However, the classification of this rich spectral information is accompanied by the challenges like high dimensionality, singularity, limited training samples, lack of labeled data samples, heteroscedasticity and nonlinearity. To address these challenges, we propose a semi-supervised graph based dimensionality reduction method named `semi-supervised spatial spectral regularized manifold local scaling cut' (S3RMLSC). The underlying idea of the proposed method is to exploit the limited labeled information from both the spectral and spatial domains along with the abundant unlabeled samples to facilitate the classification task by retaining the original distribution of the data. In S3RMLSC, a hierarchical guided filter (HGF) is initially used to smoothen the pixels of the HSI data to preserve the spatial pixel consistency. This step is followed by the construction of linear patches from the nonlinear manifold by using the maximal linear patch (MLP) criterion. Then the inter-patch and intra-patch dissimilarity matrices are constructed in both spectral and spatial domains by regularized manifold local scaling cut (RMLSC) and neighboring pixel manifold local scaling cut (NPMLSC) respectively. Finally, we obtain the projection matrix by optimizing the updated semi-supervised spatial-spectral between-patch and total-patch dissimilarity. The effectiveness of the proposed DR algorithm is illustrated with publicly available real-world HSI datasets.

Fast and Flexible Large Graph Embedding Based on Anchors

Dimensionality reduction is one of the most fundamental topic in machine learning. A range of methods focus on dimensionality reduction have been proposed in various areas. Among the unsupervised dimensionality reduction methods, graph-based dimensionality reduction has begun to draw more and more attention due to its effectiveness. However, most existing graph-based methods have high computation complexity, which is not applicable to large-scale problems. To solve this problem, an unsupervised graph-based dimensionality reduction method called fast and flexible large graph embedding (FFLGE) based on anchors is proposed. FFLGE uses an anchor-based strategy to construct an anchor-based graph and design similarity matrix and then perform the dimensionality reduction efficiently. The computational complexity of the proposed FFLGE reduces to $O(ndm)$ , where $n$ is the number of samples, $d$ is the number of dimensions and $m$ is the number of anchors. Furthermore, it is interesting to note that locality preserving projection and principal component analysis are two special cases of FFLGE. In the end, the experiments based on several publicly large-scale datasets proves the effectiveness and efficiency of the method proposed.

Dimensionality Reduction of Hyperspectral Image Using Spatial Regularized Local Graph Discriminant Embedding

Dimensionality reduction (DR) is an important preprocessing step for hyperspectral image (HSI) classification. Recently, graph-based DR methods have been widely used. Among various graph-based models, the local graph discriminant embedding (LGDE) model has shown its effectiveness due to the complete use of label information. Besides spectral information, an HSI also contains rich spatial information. In this paper, we propose a regularization method to incorporate the spatial information into the LGDE model. Specifically, an oversegmentation method is first employed to divide the original HSI into nonoverlapping superpixels. Then, based on the observation that pixels in a superpixel often belong to the same class, intraclass graphs are constructed to describe such spatial information. Finally, the constructed superpixel-level intraclass graphs are used as a regularization term, which can be naturally incorporated into the LGDE model. Besides, to sufficiently capture the nonlinear property of an HSI, the linear LGDE model is further extended into its kernel counterpart. To demonstrate the effectiveness of the proposed method, experiments have been established on three widely used HSIs acquired by different hyperspectral sensors. The obtained results show that the proposed method can achieve higher classification performance than many state-of-the-art graph embedding models, and the kernel extension model can further improve the classification performance.

Patch Tensor-Based Sparse and Low-Rank Graph for Hyperspectral Images Dimensionality Reduction

Dimensionality reduction is an important aspect in hyperspectral images processing. Recently, graph-based dimensionality reduction methods have drawn much attention and achieved promising performance. In traditional graph methods, $k$ -nearest neighbors and $\varepsilon$ -ball neighborhood are the most commonly used methods for graph construction and the pairwise Euclidean distance is often chosen as the similarity between the corresponding data points. But these methods are sensitive to data noise, and their graph structures are unstable with additive noise. More recently, sparse graph and low-rank graph have been proposed to exploit local and global structures hidden in hyperspectral images. But these methods only consider part of the entire structural information and fail to capture the full intrinsic information of hyperspectral images. To overcome these drawbacks, a patch tensor-based sparse and low-rank graph (PT-SLG) is proposed for hyperspectral images dimensionality reduction in this paper. In PT-SLG, the sparsity and low-rankness properties are jointly considered to capture the local and global intrinsic structures hidden in hyperspectral data simultaneously. In addition, tensor analysis is utilized to preserve the spatial neighborhood information. A clustering strategy is used to exploit the nonlocal similarity information, which enhances the low-rank and sparse constraints and also reduces the computational cost. Moreover, a novel tensor-based graph construction method is presented, which considers the joint similarity along the two spatial domains across all the tensor samples and makes the resulting graph more informative. Experimental results on real hyperspectral datasets demonstrate the superiority of PT-SLG over the other state-of-the-art approaches.

Dimensionality Reduction via Multiple Locality-Constrained Graph Optimization

In recent years, graph-based dimensionality reduction methods became increasingly more significant since they have been successfully applied in various computer vision and machine learning problems. The key point in graph-based dimensionality reduction methods is how to construct an appropriate graph to reflect the underlying distribution of data. However, most existing methods usually consider graph construction and dimensionality reduction as two separate processes. To overcome this limitation, a multiple locality-constrained graph optimization for dimensionality reduction (MLGODR) algorithm is proposed in this paper. The proposed MLGODR possesses two characteristics. First, MLGODR integrates graph optimization and dimensionality reduction into a unified framework. Thus, a graph that characterizes the distribution of input data and a matrix that projects the input data into a low-dimensional subspace can be learned simultaneously. Second, to better exploit the local structure of input data, a locality constraint that adaptively combines multiple distance measurements is introduced into our objective function. Moreover, an effective updating algorithm is also designed to solve the proposed MLGODR. Extensive experiments are performed on four image databases and four UCI data sets. The experimental results demonstrate that our method outperforms the compared approaches in both classification and cluster tasks.

Locality constrained Graph Optimization for Dimensionality Reduction

Recently, graph-based dimensionality reduction methods have attracted much attention due to their widely applications in many practical tasks such as image classification and data clustering. However, an inappropriate graph which cannot accurately reflect the underlying structure and distribution of input data will dramatically deteriorate the performances of these methods. In this paper, we propose a novel algorithm termed Locality Constrained Graph Optimization Dimensionality Reduction (LC-GODR) to address the limitations of existing graph-based dimensionality reduction methods. Firstly, unlike most graph-based dimensionality reduction methods in which the graphs are constructed in advance and kept unchanged during dimensionality reduction, our LC-GODR combines the graph optimization and projection matrix learning into a joint framework. Therefore, the graph in the proposed algorithm can be adaptively updated during the procedure of dimensionality reduction. Secondly, through introducing the locality constraints into our LC-GODR, the local information of high-dimensional input data can be discovered and well preserved, which makes the proposed algorithm distinct from other graph optimization based dimensionality reduction methods. Moreover, an effective updating scheme is also provided to solve the proposed LC-GODR. Extensive experiments on two UCI and five image databases are conducted to demonstrate the effectiveness of our algorithm. The experimental results indicate that the proposed LC-GODR outperforms other related methods.

Orthogonal enhanced linear discriminant analysis for face recognition

From the intuition that natural face images lie on or near a low-dimensional submanifold, the authors propose a novel spectral graph based dimensionality reduction method, named orthogonal enhanced linear discriminant analysis (OELDA), for face recognition. OELDA is based on enhanced LDA (ELDA), which takes into account both the discriminative structure and geometrical structure of the face space, and generates non-orthogonal basis vectors. However, a significant fact is that eliminating the dependence of basis vectors can promote more effective recognition of unseen face images. For this purpose, the authors seek to improve the ELDA scheme by imposing orthogonal constraints on the basis vectors. Experimental results on real-world face datasets show that, benefitting from orthogonality, OELDA has more locality preserving power and discriminative power than LDA and ELDA, and achieves the highest recognition rates among compared methods.

Double adjacency graphs-based discriminant neighborhood embedding

Discriminant neighborhood embedding (DNE) is a typical graph-based dimensionality reduction method, and has been successfully applied to face recognition. By constructing an adjacency graph, aiming to keep the local structure for original data in the subspace, it is able to find the optimal discriminant direction effectively. Not for every sample does DNE set up a link between it and its heterogeneous samples when constructing the adjacency graph, which would result in a small between-class scatter. Motivated by this fact, we develop an extension of DNE, called double adjacency graphs-based discriminant neighborhood embedding (DAG-DNE) by introducing two adjacency graphs, or homogeneous and heterogeneous neighbor adjacency graphs. In DAG-DNE, neighbors belonging to the same class are compact while neighbors belonging to different classes become separable in the subspace. Thus, DAG-DNE could keep the local structure of a given data and find a good projection matrix for them. To investigate the performance of DAG-DNE, we compare it with the state-of-the-art dimensionality reduction techniques such as DNE and MFA on several publicly available datasets. Experimental results show the feasibility and effectiveness of the proposed method.

Extended dot product representations of graphs with application to radar image segmentation

Graph-based dimensionality reduction methods are popular in pattern recognition and machine learning. In contrast to the manifold learning approaches, the dot product representation of graphs (DPRG) seeks a solution to dimensionality reduction by assigning vectors to each node of a graph such that the dot product of every pair of nodes approximates the similarity between them. The DPRG has many potential applications, for the reason that there is no prior assumption of the data distribution. It has been found, however, that the DPRG tends to reduce the distances of the graph nodes represented in a low-dimensional space, which in turn degrades the performance of data clustering. Motivated by this observation, we propose an extended DPRG (EDPRG) model by simply employing negative similarity values. The theoretical analysis and experiments on synthetic data show that the modification is effective in increasing between-class distances. We demonstrate the effectiveness of the EDPRG model by experiments on synthetic aperture radar (SAR) image segmentation. The proposed image segmentation method has two steps. The first one presegments the image by the mean shift algorithm. The second merges the resulting regions by means of the EDPRG model.